From a music library to evaluated DJ decisions

The playable booth and the DJ's decision system are separate. The target booth is four CDJs and one shared mixer. The AI lab prepares and evaluates decisions before a future adapter schedules them on that booth. Its current renderer still alternates two virtual decks; the independent four-CDJ runtime is not driven by this learning loop yet.

The 2026-09-12 implementation adds measured audio descriptors, short-sample ingestion, versioned transition recipes, controlled auditions, persistent attempts/reviews, and evidence-aware candidate selection. It does not train a neural model or establish professional mixing quality.

flowchart LR
    Library[Tracks and samples] --> Analyze[Versioned audio analysis]
    Analyze --> Retrieve[Feasible next-track candidates]
    Memory[(Recipes, attempts and ratings)] --> Retrieve
    Retrieve --> DJ[Independent DJ actor]
    DJ --> Validate[Validate selected recipe and track]
    Validate --> Execute[Deterministic transition execution]
    Execute --> Listen[Rendered audio and control trace]
    Listen --> Auto[Automatic technical measurements]
    Listen --> Human[Human listening review]
    Auto --> Memory
    Human --> Memory

What the AI should know about music

Whole-track summaries help retrieval, but transitions happen at specific positions. Keep time windows, beat/phrase coordinates, provenance, confidence where meaningful, and the source PCM hash. Silence is not zero BPM; missing vocals are not evidence of an instrumental section. A manually confirmed regular grid is not a detected verse/drop structure.

Layer	Current measurements	Further analysis to evaluate
Identity and assets	UUID, exact source/PCM hashes, original file, tags, Chromaprint	Acoustic duplicate clustering across edits and remasters
Rhythm	Provisional BPM/confidence, reviewed regular beat/phrase grid	Downbeats, variable tempo, groove/swing, onset density, percussion activity, grid uncertainty
Tonality	Provisional/reviewed key; explicit harmonic compatibility heuristic	Local chroma/chords, tuning, key changes, harmonic tension and stem-aware clashes
Levels and dynamics	Integrated LUFS, true peak; RMS/sample peak/crest, silence and full-scale fractions	Short-term/momentary LUFS, loudness range and perceptually evaluated transition envelopes
Timbre and spectrum	Centroid, 85% roll-off, flatness, four band-power fractions, normalized positive spectral flux	MFCC, spectral contrast, learned timbral embeddings, instrument activity
Stereo	Channel DC, mean-centered correlation, side-energy fraction	Frequency-dependent phase compatibility and mono translation
Musical content	Unknown, explicitly listed as unavailable	Vocal activity/lyrics, sections, build/drop/breakdown, mood, genre, instruments, semantic embeddings
Samples	Explicit sample asset type, 50 ms minimum, same measured descriptors	One-shot/loop/stem type, attack/decay, loop seams, pitch and contextual placement

New imports compute the native descriptor set over the entire canonical 48 kHz stereo asset and consecutive one-second windows. Spectral measurements use non-overlapping 4096-frame Hann windows, zero-padding the last partial frame and summing channel powers. Band boundaries are 20/150/800/4000/20000 Hz. ZCR uses the mono sum; the spectrum preserves antiphase stereo content. Positive spectral flux is computed between normalized power spectra within each analyzed interval. Silence means both channels below -80 dBFS. Full-scale sample fraction uses absolute amplitude ≥1; it is not proof of a clipped source. Mathematical measurements do not receive invented classifier confidence scores. Null values represent undefined/unavailable measurements such as correlation of silence.

analysis.features.version, source_pcm_sha256, FFT/rate/window configuration and explicit unavailable families document provenance. Existing libraries remain readable; Refresh measurements computes features without changing reviewed timing/key values and increments the analysis revision. Analysis snapshots in each session preserve its inputs. Short samples are excluded from full-track review and selection; automatic sample placement is future work.

Essentia's extractor documents spectral, rhythm and tonal families and configurable time-series extraction. Its model catalogue is a reference for evaluating semantic analyzers. No Essentia binary or model is integrated in this iteration. Future workers should record model/checkpoint hashes, preprocessing, feature version and confidence/calibration; validate each family on the intended genres and check distribution terms before shipping model assets.

Reusable transition scripts

A recipe describes a reusable technique. An attempt binds it to two particular assets, cue positions, tempo, analysis revisions and an engine build. Human ratings belong to that attempt, not universally to the recipe or track pair.

The implemented JSON language is deliberately bounded: 16/32 beats, linear/equal-power channel-fader curves, a complementary low-EQ exchange, and an optional outgoing high-pass sweep. It contains no arbitrary code, shell commands or network addresses. Positions and durations are beats; faders and EQ endpoints are linear gains. Filter values are the current engine's normalized high-pass parameter, not Hz or a calibrated DJM-A9 control law. The compiler handles load, cue, key-locked tempo, play, ownership grants, ramps and outgoing stop.

{
  "id": "gentle-bass-exchange",
  "version": 1,
  "name": "Gentle 32-beat bass exchange",
  "length_beats": 32,
  "fader_curve": "equal_power",
  "bass_swap_at_beats": 16,
  "bass_swap_duration_beats": 8,
  "outgoing_filter": {"start_beat": 24, "end_value": 0.6}
}

Recipe ID/version pairs are immutable. Reuse requires choosing that version; editing requires a new version or ID. Four baseline recipes are seeded (blend/filter ×16/32). Full recipe snapshots go into plans and attempts, so subsequent catalogue changes do not rewrite historical execution. Validate bounds before compilation. The opening track uses a short startup fade and is not recorded as a transition attempt.

/transitions in aidj-ui lets an operator inspect/adapt JSON recipes and audition a chosen pair/recipe. Auditions use reviewed first-beat cues, a 32-beat lead-in, selected target tempo and a short tail. Both tracks must fit the duration, reviewed phrase grid and ±15% key-locked tempo range. They create deterministic experiments, not fabricated AI decisions. Change one variable at a time to compare recipes. Excerpts play from the session audio with two seconds of surrounding context.

Automatic and human ranking

Automatic evaluation measures each complete overlap on pre-quantization stereo output: peak, clipped-sample fraction (>1), silent-frame fraction and the RMS difference between its first/last one-second regions (shortened to a quarter of the overlap where needed). Partial overlaps do not receive a score. overlap-health-1 is a transparent diagnostic score:

score = clamp(1 - min(100 * clipped_fraction, 1)
                - 0.6 * silent_fraction
                - 0.2 * min(abs(rms_change_db) / 12, 1), 0, 1)

Undefined RMS change uses the maximum 0.2 penalty. This detects technical problems, not beat alignment, vocal collisions, phrase quality or taste. Silence can be intentional; raw measurements remain available for interpretation. Do not present this score as “mix quality” or use it as the sole optimization objective.

Human reviews record overall preference, rhythm, harmony, energy and coherence (1–5), a reviewer label and notes. Reviews are append-only; the newest rating from each label per attempt counts in ranking. Reviewer labels are not verified identities under the current single-token local API. The UI sorts attempts by human preference or technical health separately and displays review counts. Blind pairwise preference, randomized playback, verified raters and inter-rater agreement are future extensions.

The actor sees a bounded feasible shortlist, entry/outgoing audio-window descriptors, available recipe snapshots and matching evidence. The baseline score considers tempo change, a named harmonic heuristic, target energy and novelty. Existing model-backed actors receive the same structured context and must select supplied IDs. Feasibility is checked before evidence is applied. The local planner remains a deterministic policy, not an LLM.

Only matching asset hashes, analysis revisions, outgoing position, incoming cue, target BPM, recipe snapshot and engine fingerprint contribute to a choice. Five neutral pseudo-observations reduce the influence of small samples. A bounded bonus combines technical evidence (weight 0.05) and human preference (weight 0.15); an actor's preferred transition length adds 0.02 to recipe selection. Only overall human preference currently affects this bonus; the other dimensions are retained for inspection and future models. No cross-pair generalization or online model training is claimed.

Transition memory lives in MongoDB when configured, or library/transitions.sqlite in local mode. Both are durable data, not rebuildable caches. Back it up together with library sidecars/PCM and session audio. Collections/tables store recipes, attempts and review history. Rendering writes transitions.json alongside manifest.json, commands.json, booth.jsonl, decisions, metrics and audio. A replay verifies execution but does not insert more learning observations. The current listing/ranking reads the newest 5,000 attempts; storage retains older rows. Pagination and indexed contextual queries should replace this bound before larger deployments.

Path to AI versus AI, then human versus AI

1. Offline evaluated choices — implemented foundation: native descriptors, reviewed timing, reusable recipes, independent actor choices, auditions, ratings and deterministic replay. Build a listening corpus with real music across tempos, genres and transition techniques.
2. Richer perception: evaluated vocal/section/beat-grid models, time-local embeddings, sample/loop placement and better candidate cue points. Add optional heavy analyzers as versioned background jobs; the playback clock must not wait for them.
3. Learned ranking and search: compare multiple feasible pair/recipe/cue candidates, preserve failures, add bounded exploration and preference learning. Split evaluation by track/artist/session so reused audio does not leak into held-out quality results. Measure musical quality as well as clipping, latency and failure rate.
4. Live AI versus AI: prefetch/decode, validate resource leases, reserve lead time, translate beat plans into the four-player/shared-mixer runtime, and log observed execution. Keep recovery and fallback transitions local when model requests time out.
5. Human versus AI: observe the human's playhead, tempo, on-air channels and intent; the AI owns only assigned decks/channels. Support takeover/cancel and uncertainty-aware planning. AI audio feeds spare channels of the physical DJM-A9, which remains the final audio mixer. Existing protocol/MIDI discovery is not authorization to move physical mixer controls.

An “optimal” transition depends on the desired energy arc, style, novelty and listener preference. Keep those objectives explicit. The first learning loop should improve measurable outcomes without turning a loudness proxy into the definition of a good DJ.

Verified operator workflow

The synthetic integration check exercised imports, one-second metrics, a short sample, immutable recipe creation, a chosen-recipe audition, human review, exact replay and absence of duplicate learning rows. Browser checks exercised recipe editing, audition creation, listening-review submission and time-window inspection on desktop/mobile. These establish working execution and persistence; musical quality still requires listening tests with reviewed real music.

The displayed technical scores are synthetic test results, not endorsements of musical quality.

See shared metadata storage for the MongoDB schema, migration, multi-worker behavior and backup requirements.

Offline analysis and catalog

The music-analysis service now scans a source collection without immediately decoding it, stores native/model reports separately, and exposes searchable per-song metrics in the UI. Neural timelines remain a quality-gated pilot. They are not yet injected into the session planner as trusted vocal/phrase evidence.