Training, test sets and manual feedback
Updated 2026-09-13. Scoring has three distinct meanings: measured technical behavior, human preference, and a model prediction. Store their sources separately. Do not display an arbitrary weighted score as a probability of a good mix.
Manual review implemented
The Transition memory page auditions a rendered overlap and stores overall, rhythm, harmony, energy and coherence ratings from 1–5 plus notes. Low overall ratings reduce the exact-context planner bonus; positive technical health cannot cancel a negative human contribution. Lowest-ratings-first helps review bad results.
Avoid excludes that recipe from subsequent offline B2B planning for the exact source PCM hashes, analysis revisions, cue positions, target BPM and engine fingerprint. It does not ban the whole song or recipe globally. The latest review per reviewer label per attempt counts; earlier entries remain in history. Submit a new review with Avoid cleared to withdraw that reviewer's exclusion. Any other current exclusion still applies. All candidates excluded means no feasible choice; the planner must not silently restore an avoided option.
The API adds optional avoid: boolean to
POST /api/v1/transitions/{attempt_id}/reviews. Old records default to false.
Evidence.human_excluded is passed through the stored candidate evidence.
Storage uses the existing workspace Mongo/SQLite transition store.
Reviewer labels are still self-entered, not verified Keycloak identity or expert credentials. These are listener ratings, not certified professional labels. Live supervisor handoffs currently record events but do not create these rendered attempts; this review flow applies to the B2B lab. Whole-set, sample and overlay review forms still need implementation.
Offline training foundation implemented
Rust prochain-evaluation::learning exports rated attempts, validates a versioned
dataset, builds grouped train/validation/test partitions, fits a small supervised
recipe-rating regression model, and evaluates held-out mean absolute error against
a constant training-mean baseline. No external Python/ML service is required.
The baseline's six inputs are target tempo, recipe duration, bass-swap location and duration, filter depth and fader-curve type. They describe a recipe before rendering. Post-render faults and human ratings are targets/evidence, never input features. This intentionally limited model does not understand vocals, style or track compatibility and is not wired into live ranking.
Dataset dj-learning-1 stores feature names, source and rights declarations plus
examples with IDs, kind, normalized features, target, label source, provenance and
leakage keys. Kinds distinguish transition/mix/sample/overlay, but the current
export and CLI trainer only implement transition ratings. Synthetic and AI-critic
labels cannot train the human-preference baseline. Existing unreviewed attempts
are omitted, rather than invented as successful demonstrations.
Rows sharing any session/mix, source PCM or rendered-audio identity stay in one connected component, including transitive links. Components are hash-assigned approximately 80/10/10; small datasets may yield empty partitions and cannot train. A common track can connect many mixes into one component. This is intentional. For external data, also include canonical recording/song-family IDs across edits; PCM hashes alone cannot detect a remaster or another encoding of the same song.
A canonical SHA-256 fingerprints the complete dataset. Freeze the dataset and split together: adding labels creates a new revision, not a silent change to the held-out benchmark. The trainer checks the exact split, requires at least 20 training/5 validation/5 test examples of one kind/source, and uses fixed hyperparameters. Validation/test labels never update weights. The minimum is an engineering gate, not evidence of statistical sufficiency. Correlated examples remain correlated even when split correctly; uncertainty estimates and wider independent test sets remain needed.
Models record dataset hash, feature schema, weights, training mean, validation
errors and deployment_approved: false. Test reports compare held-out error with
the baseline. No command auto-promotes a model. Repeated experimentation against
the same test set can overfit it; freeze an additional final listening benchmark
before deployment and use validation for model selection.
From aidj-platform, with the normal Rust/native build environment:
cargo run -p prochain-evaluation --example learning -- export /path/to/library > dataset.json
cargo run -p prochain-evaluation --example learning -- split dataset.json > split.json
cargo run -p prochain-evaluation --example learning -- train dataset.json split.json > model.json
cargo run -p prochain-evaluation --example learning -- test dataset.json split.json model.json > test.json
Export honors the existing Mongo workspace environment when configured; otherwise it reads the local transition store. Keep snapshots private. The CLI is a developer example, not currently packaged as a deployed training worker. The regression checks use labelled synthetic fixtures to test the software, not to claim a trained professional DJ model.
Dataset research and what it can teach
| Source | Available evidence | Appropriate use and limitation |
|---|---|---|
| UnmixDB | Automatically generated mixes; source/cue/tempo/speed ground truth; underlying CC track material | Validate alignment and reconstruction of known mixing operations. It is synthetic, not expert taste labels. Preserve per-track terms and attribution. |
| DJ Mix Dataset | Human DJ mix/track metadata and acquisition tooling | Investigate authentic sequence/transition examples. Metadata availability does not grant blanket redistribution or training rights for linked recordings. |
| DJtransGAN | Research implementation, pretrained model and data-generation pipeline | Reference for learned fader/EQ control. Authors explicitly do not distribute their training dataset because of licensing issues. |
These are researched sources, not imported training data. Their annotations do not automatically supply “good versus bad,” professional ratings, lyrics/overlay placement or labelled sample opportunities. External adapters, asset rights review, recording identity mapping and benchmark imports remain work. The user's product specification defines the learning loop but is not itself a labelled dataset.
Next connection to the DJ GPS
Capture a stable recommendation ID, candidate set, analysis/model versions, actual choice, rendered outcome and later review. A skipped recommendation is weak behavioral evidence: the DJ may have been browsing or preparing another deck. Only explicit feedback is an explicit preference label. Separate private per-DJ preferences from a shared technical policy and obtain consent before using private session material in a shared corpus.
Use learned scores to rank permitted candidates/templates. A score never bypasses approved timing, current region coordinates, authority, headroom or human Avoid. Test challengers against the deployed baseline, archive comparisons, promote with a rollback path, then measure outcomes again. The idle practice scheduler, live attempt capture, authenticated expert reviews, deployment gate and model loading are not implemented by this offline baseline.