From a music library to evaluated DJ decisions

The playable booth and the DJ's decision system are separate. The target booth is four CDJs and one shared mixer. The AI lab prepares and evaluates decisions before a future adapter schedules them on that booth. Its current renderer still alternates two virtual decks; the independent four-CDJ runtime is not driven by this learning loop yet.

The 2026-09-12 implementation adds measured audio descriptors, short-sample ingestion, versioned transition recipes, controlled auditions, persistent attempts/reviews, and evidence-aware candidate selection. It does not train a neural model or establish professional mixing quality.

flowchart LR
    Library[Tracks and samples] --> Analyze[Versioned audio analysis]
    Analyze --> Retrieve[Feasible next-track candidates]
    Memory[(Recipes, attempts and ratings)] --> Retrieve
    Retrieve --> DJ[Independent DJ actor]
    DJ --> Validate[Validate selected recipe and track]
    Validate --> Execute[Deterministic transition execution]
    Execute --> Listen[Rendered audio and control trace]
    Listen --> Auto[Automatic technical measurements]
    Listen --> Human[Human listening review]
    Auto --> Memory
    Human --> Memory

See the 2026-09-13 training and manual feedback update: reversible exact-context exclusions and an offline supervised baseline now extend this foundation. It remains separate from live automatic learning.

What the AI should know about music

Whole-track summaries help retrieval, but transitions happen at specific positions. Keep time windows, beat/phrase coordinates, provenance, confidence where meaningful, and the source PCM hash. Silence is not zero BPM; missing vocals are not evidence of an instrumental section. A manually confirmed regular grid is not a detected verse/drop structure.

Layer	Current measurements	Further analysis to evaluate
Identity and assets	UUID, exact source/PCM hashes, original file, tags, Chromaprint	Acoustic duplicate clustering across edits and remasters
Rhythm	Provisional BPM/confidence, reviewed regular beat/phrase grid	Downbeats, variable tempo, groove/swing, onset density, percussion activity, grid uncertainty
Tonality	Provisional/reviewed key; explicit harmonic compatibility heuristic	Local chroma/chords, tuning, key changes, harmonic tension and stem-aware clashes
Levels and dynamics	Integrated LUFS, true peak; RMS/sample peak/crest, silence and full-scale fractions	Short-term/momentary LUFS, loudness range and perceptually evaluated transition envelopes
Timbre and spectrum	Centroid, 85% roll-off, flatness, four band-power fractions, normalized positive spectral flux	MFCC, spectral contrast, learned timbral embeddings, instrument activity
Stereo	Channel DC, mean-centered correlation, side-energy fraction	Frequency-dependent phase compatibility and mono translation
Musical content	Unknown, explicitly listed as unavailable	Vocal activity/lyrics, sections, build/drop/breakdown, mood, genre, instruments, semantic embeddings
Samples	Explicit sample asset type, 50 ms minimum, same measured descriptors	One-shot/loop/stem type, attack/decay, loop seams, pitch and contextual placement

New imports compute the native descriptor set over the entire canonical 48 kHz stereo asset and consecutive one-second windows. Spectral measurements use non-overlapping 4096-frame Hann windows, zero-padding the last partial frame and summing channel powers. Band boundaries are 20/150/800/4000/20000 Hz. ZCR uses the mono sum; the spectrum preserves antiphase stereo content. Positive spectral flux is computed between normalized power spectra within each analyzed interval. Silence means both channels below -80 dBFS. Full-scale sample fraction uses absolute amplitude ≥1; it is not proof of a clipped source. Mathematical measurements do not receive invented classifier confidence scores. Null values represent undefined/unavailable measurements such as correlation of silence.

analysis.features.version, source_pcm_sha256, FFT/rate/window configuration and explicit unavailable families document provenance. Existing libraries remain readable; Refresh measurements computes features without changing reviewed timing/key values and increments the analysis revision. Analysis snapshots in each session preserve its inputs. Short samples are excluded from full-track review and selection; automatic sample placement is future work.

Essentia's extractor documents spectral, rhythm and tonal families and configurable time-series extraction. Its model catalogue is a reference for evaluating semantic analyzers. No Essentia binary or model is integrated in this iteration. Future workers should record model/checkpoint hashes, preprocessing, feature version and confidence/calibration; validate each family on the intended genres and check distribution terms before shipping model assets.

Reusable transition scripts

A recipe describes a reusable technique. An attempt binds it to two particular assets, cue positions, tempo, analysis revisions and an engine build. Human ratings belong to that attempt, not universally to the recipe or track pair.

The implemented JSON language is deliberately bounded: 8/16/32/64/96/128 beats, linear/equal-power channel-fader curves, a complementary low-EQ exchange, and an optional outgoing high-pass sweep. It contains no arbitrary code, shell commands or network addresses. Positions and durations are beats; faders and EQ endpoints are linear gains. Filter values are the current engine's normalized high-pass parameter, not Hz or a calibrated DJM-A9 control law. The compiler handles load, cue, key-locked tempo, play, ownership grants, ramps and outgoing stop.

{
  "id": "gentle-bass-exchange",
  "version": 1,
  "name": "Gentle 32-beat bass exchange",
  "length_beats": 32,
  "fader_curve": "equal_power",
  "bass_swap_at_beats": 16,
  "bass_swap_duration_beats": 8,
  "outgoing_filter": {"start_beat": 24, "end_value": 0.6}
}

Recipe ID/version pairs are immutable. Reuse requires choosing that version; editing requires a new version or ID. The store seeds the four historical baselines (blend/filter ×16/32) plus the structural set built by TransitionRecipe::structural — blend, bassswap, filterfade, dropswap and cut at 8/16/32/64 beats — deduplicated by key, currently 22 recipes. blend-16 and blend-32 are byte-identical to the historical baselines, so their recorded evidence still applies. The planner and the store share that one constructor: a recipe key must never describe two different scripts. Full recipe snapshots go into plans and attempts, so subsequent catalogue changes do not rewrite historical execution. Validate bounds before compilation. The opening track uses a short startup fade and is not recorded as a transition attempt.

/transitions in aidj-ui lets an operator inspect/adapt JSON recipes and audition a chosen pair/recipe. Auditions use reviewed first-beat cues, a 32-beat lead-in, selected target tempo and a short tail. Both tracks must fit the duration, reviewed phrase grid and ±15% key-locked tempo range. They create deterministic experiments, not fabricated AI decisions. Change one variable at a time to compare recipes. Excerpts play from the session audio with two seconds of surrounding context.

Automatic and human ranking

Automatic evaluation measures each complete overlap on pre-quantization stereo output. Two scores are kept deliberately separate.

Technical health (technical_score, unchanged formula): peak, clipped-sample fraction (>1), silent-frame fraction and the RMS difference between the overlap's first/last one-second regions.

score = clamp(1 - min(100 * clipped_fraction, 1)
                - 0.6 * silent_fraction
                - 0.2 * min(abs(rms_change_db) / 12, 1), 0, 1)

It detects a broken render and nothing else. Measured across 1,027 practice renders on 2026-09-14 its spread was 0.034 while the recipe factors being swept moved it by 0.0015 (fader curve), 0.0014 (filter depth) and 0.0007 (bass-swap position) — an order of magnitude under the noise. Only overlap length registered, in the degenerate direction where longer scored higher. Optimising against it could not, even in principle, tell a bass swap from a bass pile-up.

Transition quality (transition_quality, evaluator overlap-health-2): the renderer now hands each deck's processed signal to the evaluator before summing. Per-deck one-pole band splits in 50 ms blocks give the fraction of the overlap where both decks carry substantial bass at once (relative to each deck's own loudest block, so a quiet incoming track is not mistaken for a clean swap), the same for the top band, the deepest hole and loudest pile-up in the mix against its opening level, and the spectral lurch between the overlap's first and last fifth. These are exactly what a bass swap, a high-pass sweep and a fade curve change. The regression test renders two overlaps identical in every health measure that differ only in whether the outgoing bass is cut when the incoming arrives: health saw 0.0007 between them, transition quality more than 0.15. On the first two real practice renders health reported 0.994 and transition quality 0.70, for an 11.8 dB level dip during the overlap.

Neither score is taste. Rows scored before overlap-health-2 existed carry only the health score and are reported as such; they are not rescored. Do not present either as “mix quality” or use either as the sole optimization objective.

Human labels come in two forms. Reviews record overall preference, rhythm, harmony, energy and coherence (1–5), a reviewer label and notes; they are append-only and the newest rating from each label per attempt counts in ranking. Blind pairwise comparisons (added 2026-09-14) are the label the ranking model is actually fitted on: two renders of the same recordings, left/right shuffled per pair, the listener picks the cleaner handover or a tie, and only then sees which recipe was which. A comparison cancels both the reviewer's scale and the material, which a lone 1–5 score cannot; across a thousand practice renders the gap between two reviewers' idea of “4 out of 5” is larger than the entire measured spread. GET /api/v1/transitions/next-comparison orders candidate pairs by fewest differing recipe factors, then largest measured gap, so a few dozen answers teach the most. Since 2026-09-20 it offers only renders whose mix.wav is still on the data volume: practice retention removes the audio of unreviewed sessions after seven days (24 hours under disk pressure) but keeps the attempt rows, and before this change the queue kept serving pairs nobody could play. Ratings already recorded on such pairs cannot have been made by ear; the page counts them and offers POST /api/v1/transitions/preferences/discard-unheard, which moves them to a preferences_voided archive so the next training run leaves them out. A rating whose two sides both still play is provably sound, because retention only ever removes audio. Reviewer labels are still not verified identities. Verified raters and inter-rater agreement remain future work.

The actor sees a bounded feasible shortlist, entry/outgoing audio-window descriptors, available recipe snapshots and matching evidence. The baseline score considers tempo change, a named harmonic heuristic, target energy and novelty. Existing model-backed actors receive the same structured context and must select supplied IDs. Feasibility is checked before evidence is applied. The local planner remains a deterministic policy, not an LLM.

Evidence is pooled at two levels. Observations on the same recording pair — matching asset hashes, analysis revisions, recipe snapshot and engine fingerprint — are strongest and are used alone whenever they exist. Playhead seconds are deliberately not part of that match: an earlier revision required exact float equality of outgoing position and incoming cue, so in practice no two attempts ever matched. When no same-pair observation exists, observations of the same move on comparable material transfer at a 0.5 discount, keyed on a coarse musical bucket: outgoing and incoming section labels, overlap length, technique, and quantized energy delta, vocal collision and tempo. Rows at a superseded analysis revision of the same recordings are excluded rather than transferred, and rows without structural evidence never enter a bucket. Five neutral pseudo-observations reduce the influence of small samples. A bounded bonus combines measured evidence (weight 0.05; transition_quality where a render carries it, the health score otherwise) and human preference (weight 0.15). Only overall human preference currently affects this bonus; the other dimensions are retained for inspection and future models. transferable_evidence is an explicit cross-engine variant used only by the live booth, which reuses offline practice renders and says so. Bucketed transfer is a similarity heuristic, not online learning.

Attempts carry a provenance — practice, session or audition — in place of the earlier single diagnostic flag, which had meant both “rendered on an unapproved grid” and “never learn from this” and had silently excluded the entire practice corpus from ranking and training. Practice renders are measured evidence and count for both; what still approves nothing is any render of any provenance, enforced by the renderer's grid check.

Separately from pooled evidence, an approved preference model may add a bonus bounded to ±0.10 (tanh of its margin) in both rankers. analyzerd dj-learn fits the pairwise model on stored comparisons with a held-out split by connected recording/session group and stores it unapproved; dj-learn approve LABEL is the only way it becomes deployable, and is refused unless held-out accuracy beat the coin flip. Without an approved model the bonus is exactly zero. As of the 2026-09-14 rollout no comparisons and therefore no model exist.

Transition memory lives in MongoDB when configured, or library/transitions.sqlite in local mode. Both are durable data, not rebuildable caches. Back it up together with library sidecars/PCM and session audio. Collections/tables store recipes, attempts and review history. Rendering writes transitions.json alongside manifest.json, commands.json, booth.jsonl, decisions, metrics and audio. A replay verifies execution but does not insert more learning observations. The current listing/ranking reads the newest 5,000 attempts; storage retains older rows. Pagination and indexed contextual queries should replace this bound before larger deployments.

Path to AI versus AI, then human versus AI

1. Offline evaluated choices — implemented foundation: native descriptors, reviewed timing, reusable recipes, independent actor choices, auditions, ratings and deterministic replay. Build a listening corpus with real music across tempos, genres and transition techniques.
2. Richer perception: evaluated vocal/section/beat-grid models, time-local embeddings, sample/loop placement and better candidate cue points. Add optional heavy analyzers as versioned background jobs; the playback clock must not wait for them.
3. Learned ranking and search: compare multiple feasible pair/recipe/cue candidates, preserve failures, add bounded exploration and preference learning. Split evaluation by track/artist/session so reused audio does not leak into held-out quality results. Measure musical quality as well as clipping, latency and failure rate.
4. Live AI versus AI: prefetch/decode, validate resource leases, reserve lead time, translate beat plans into the four-player/shared-mixer runtime, and log observed execution. Keep recovery and fallback transitions local when model requests time out.
5. Human versus AI: observe the human's playhead, tempo, on-air channels and intent; the AI owns only assigned decks/channels. Support takeover/cancel and uncertainty-aware planning. AI audio feeds spare channels of the physical DJM-A9, which remains the final audio mixer. Existing protocol/MIDI discovery is not authorization to move physical mixer controls.

An “optimal” transition depends on the desired energy arc, style, novelty and listener preference. Keep those objectives explicit. The first learning loop should improve measurable outcomes without turning a loudness proxy into the definition of a good DJ.

Verified operator workflow

The synthetic integration check exercised imports, one-second metrics, a short sample, immutable recipe creation, a chosen-recipe audition, human review, exact replay and absence of duplicate learning rows. Browser checks exercised recipe editing, audition creation, listening-review submission and time-window inspection on desktop/mobile. These establish working execution and persistence; musical quality still requires listening tests with reviewed real music.

The displayed technical scores are synthetic test results, not endorsements of musical quality.

See shared metadata storage for the MongoDB schema, migration, multi-worker behavior and backup requirements.

Offline analysis and catalog

The music-analysis service now scans a source collection without immediately decoding it, stores native/model reports separately, and exposes searchable per-song metrics in the UI. Neural timelines remain a quality-gated pilot. They are not yet injected into the session planner as trusted vocal/phrase evidence.