Qdrant retrieval and the remaining DJ-analysis gap

The platform now has a first-class offline Rust analyzer and a separate Rust retrieval crate. MongoDB database prochain_aidj is authoritative. Qdrant is a rebuildable index of already computed embeddings; indexing and searching never decode audio or call an LLM.

Implemented pipeline

Symphonia → canonical 48 kHz stereo PCM → native DSP measurements
                        ↓
               Rubato → 16 kHz mono
                        ↓
             ONNX Runtime / YAMNet
                        ↓
      voice head + music heads + mean embedding
                        ↓
     versioned Mongo report and two-second timeline
                        ↓
     Qdrant payload filters + cosine retrieval (30)
                        ↓
       current Mongo identity/review revalidation
                        ↓
       similarity/tempo ordering → at most 10
                        ↓
       compact DJ summaries, ready for an agent

prochain-music-retrieval uses Qdrant's HTTP API through the existing Rust reqwest dependency. The tested cluster runs Qdrant 1.12.4. No Python runs inside production analysis or retrieval. The optional Python scripts remain offline benchmark tools.

The dedicated collection is prochain_aidj_tracks_v1, with 1,024-dimensional cosine vectors and UUID track IDs. Payload indexes cover embedding space, review status, track kind, BPM, energy and exact key. This follows Qdrant's filtered retrieval and payload indexing APIs.

Embedding space includes model hash, analyzer code hash, sample rates, pooling, model identity and dimensions. Different spaces are never compared even if their vector dimensions match. The code hash is deliberately conservative: unrelated analyzer changes can require a new compatible cohort. A future separately versioned embedding extractor can reduce unnecessary separation. Structure-only refinements retain the original embedding provenance.

Commands and API

Set BOOTH_QDRANT_URL and optional BOOTH_QDRANT_API_KEY_FILE. The collection override must start with prochain_aidj_; credentials are not accepted in URLs. HTTP calls have timeouts, redirects are disabled and authentication is not logged.

analyzerd index-init
analyzerd index-tracks TRACK_ID...
analyzerd similar CURRENT_TRACK_ID filters.json

Example filters.json:

{
  "bpm": 128,
  "energy_min": 0.7,
  "energy_max": 0.95,
  "diagnostic_unreviewed": false
}

An optional key is an exact native key string such as A minor, not a Camelot-neighbour filter. Tempo filtering permits a target/source ratio of 0.85–1.15. The authenticated POST /api/v1/library/{id}/similar accepts the same body. The old shortlist BPM remains a separate tempo-only diagnostic. The library UI has not yet gained a similarity-results panel.

Indexing accepts explicit batches of 1–1,000 IDs and upserts the current report by stable track UUID. It returns failures separately; the CLI fails a partially rejected batch. Source read failures do not erase existing index entries. Re-index after native review, changed PCM or a new musical report. There is no continuous Mongo change-stream synchronizer yet.

Every returned hit is reloaded from Mongo and checked against an identity/review/feature snapshot hash, the embedding space and current filter eligibility. Deleted, missing, stale or newly unreviewed tracks are dropped. The snapshot hash avoids rejecting otherwise identical payloads because of JSON floating-point round trips. Compact context must still refer to the checked report. Stale entries can reduce results below ten; the query does not scan beyond its bounded 30 hits to fill the list.

Production retrieval requires reviewed tracks. diagnostic_unreviewed: true is an explicit research mode; its output is marked diagnostic. The 13 benchmark embeddings were indexed, but those recordings remain unreviewed, so a reviewed-only query correctly returns no benchmark candidates.

Ordering is an explicit heuristic: 0.75 × max(cosine, 0) + 0.25 × tempo_fit. It is not a DJ compatibility score. Responses include similarity and required tempo adjustment, and set recommendation_ready: false. No raw vector or frame array is passed into the compact candidate response.

Distance from the intended AI DJ

Capability	Current implementation	Remaining gate
Native measurements	Decoding/resampling, spectrum, band energy, flux, RMS/crest, LUFS/true peak, chroma/key, onset/tempo estimates	Validate difficult recordings; confidence calibration and metrical ambiguity
Beat/phrase timing	Provisional beat grid, local tempo estimates, regular four-bar phrase candidates	Reliable downbeats, meter, variable-tempo alignment and actual phrase boundaries
Musical structure	Intro/groove/build/breakdown/outro and entry/drop hypotheses	Human-labelled section boundaries; verified verse/pre-chorus/chorus/bridge
Vocals	Music-specific two-second score timeline and benchmark evaluator	Calibrated presence, trustworthy vocal-free windows, separate dominance model
Instruments	Generic AudioSet group scores	Music-specific instrument evidence or stems; bass-band power is not a bass stem
Genre/mood/danceability	Optional research heads	Library validation and suitable model licensing before wider deployment
Retrieval	Versioned embeddings, filtered Qdrant search, Mongo revalidation, bounded summaries	Better music embeddings and learned/validated pair compatibility
Agent decision	Existing agent infrastructure and compact analysis context	Wire this retrieval path into the live next-track planner and evaluate choices
Transitions	Existing reusable recipes/attempts/reviews and virtual controls	Compile region-aware plans into beat-aligned controls; measure rendered audio and human preference
AI-vs-AI / AI-vs-human	Four-player/mixer prototype and earlier LAN/audio tests	Reliable end-to-end autonomous sessions; physical audio routing, timing and ownership validation

We have the data and retrieval foundation. We cannot yet reliably answer “mix this pair over the next 32 bars without vocal collision and land the drop.” The next work is: annotate standalone benchmark passages; improve and measure beat/downbeat and vocal timing; score specific outgoing/incoming regions; render and rank reusable transition scripts; then connect that validated loop to the live agents. Qdrant improves candidate retrieval, not the accuracy of the measurements it filters.

Implemented pipeline​

Commands and API​

Distance from the intended AI DJ​

Implemented pipeline

Commands and API

Distance from the intended AI DJ