DJ analyzer v2 — evidence and limitations
Version 2 improves the three-layer offline pipeline. It preserves native measurements, adds musical structure hypotheses, and optionally runs music-specific ONNX heads over cached YAMNet embeddings. No LLM is called to analyze a recording. Original audio remains unchanged and source-wide processing remains paused.
| Layer | Improvement | Remaining limit |
|---|---|---|
| Deterministic MIR | Explicit tempo prior; 16-second local tempo estimates every 8 seconds; 2-second bass/mid/high, flux, crest and silence timeline alongside existing loudness/key/chroma | Tempo prior can bias uncommon tempi; half/double-time and other metrical ambiguities remain; local evidence is not a verified variable-tempo grid |
| Musical structure | Feature novelty over 6/12/24 seconds; section, build/breakdown and bass/drum-entry hypotheses; phrase support curve | Verse/chorus/bridge, drop boundaries, meter and downbeats are unverified |
| Neural music understanding | Optional music-trained voice head with short context; genre, mood and danceability heads with whole-track context | Scores are uncalibrated; generic instrument predictions remain weak; no stems or instrument transcription |
| DJ context | Current section, upcoming event candidates, candidate regions, key/tempo/loudness and top music tags | No pairwise compatibility certification or live planner integration |
The optional heads come from Essentia's model repository, whose MTG weights have CC BY-NC-SA 4.0 terms with other licensing available from the authors. They are separate research assets, not bundled in Git or the runtime image. See the voice model metadata for its training context. Our library and temporal pooling need their own validation.
The full reports preserve hashes of audio, analyzer code, models, label metadata and the loaded ONNX runtime. The previous reports remain immutable. Missing models or uncertain measurements remain explicit rather than being filled with invented labels or confidence values.
How to use the evidence
The /library UI shows the new section table, local tempo evidence, phrase hypotheses, voice timeline and optional music tags. Selecting a section moves the inspection window. Listening preview supports checking the corresponding audio. The complete report and compact DJ context remain inspectable.
analyzerd summary TRACK_ID 120 returns cached context for the two-minute position. It includes at most three future event hypotheses and two mix-region candidates. Embeddings and frame arrays are omitted; music tags are capped at six heads and three classes per head. shortlist attaches context to reviewed candidates where a current report exists. The command does not invoke a model or approve a track for performance.
Segment-level listening annotations are still needed to measure vocal recall/false positives and beat/section timing errors. Whole-track agreement with a named benchmark is not sufficient to qualify vocal exclusion or phase-safe mixing. The next gate is measured accuracy on those annotations, followed by pairwise transition evaluation.
Six-recording comparison
These are model outputs, not accuracy scores. The vocal column is the mean music-head score with up to 4.815 seconds of context; it must not be interpreted as the fraction of the track containing vocals.
| Recording | Tempo estimate | Mean music voice score | Variable-tempo candidate |
|---|---|---|---|
| Dua Lipa — Don’t Start Now (VDJ JD edit) | 127.89 BPM | 0.888 | No |
| Eric Prydz — Opus | 126.29 BPM | 0.433 | Yes |
| deadmau5 — Strobe | 127.60 BPM | 0.365 | Yes |
| Daft Punk — Aerodynamic | 81.81 BPM | 0.515 | Yes |
| M.A.N.D.Y./Booka Shade — Body Language | 127.73 BPM | 0.367 | Yes |
| Bicep — Glue | 86.89 BPM | 0.522 | Yes |
The Dua Lipa edit moved from a 64.01 BPM estimate to 127.89 BPM. The new voice head provides strong vocal evidence on that recording, whereas the generic AudioSet vocal score was very low. This is evidence of improved detection on one reference, not calibration across the library.
Opus now has a build hypothesis from approximately 2:04 to 3:44 and a subsequent drop candidate at 3:44. Its local-tempo variation is flagged. Other detected entries remain bass/drum-entry candidates rather than being automatically named drops. These timings have not been checked against manual boundary annotations. Early quiet sections are kept distinct from breakdowns after an established groove; a fading section is not labelled entirely silent simply because its average level is low.
Several problems remain visible: Aerodynamic and Glue retain tempo ambiguity, five recordings trigger the variable-tempo heuristic, and instrumental or ambiguous textures still receive substantial voice scores. These prevent automatic acceptance for sync or vocal-collision avoidance. Genre/mood/danceability are available as research-model scores, but have not been measured against human reference labels. Generic instrument detection and verified downbeats remain open quality gaps.
All six base reports were computed and reused from cache. A subsequent structure-only refinement reused their audio/neural measurements, published new immutable reports, and returned cache hits on repetition. Position-specific summaries were checked for all six. The local comparison artifact is aidj-platform/data/benchmark-analysis-v2.json; immutable parent links preserve prior measurements. Originals and the 6,005-file source queue remain unchanged, with bulk processing paused.
Validation included 65 passing Rust workspace tests, the ONNX/TensorFlow fixture, focused structure tests after refinement, clean Clippy, UI production build, and browser checks for section navigation, model scores, compact context, audio preview and mobile layout. No paid LLM call or live booth deployment was performed.