Skip to main content

Shared metadata storage

The platform supports MongoDB for shared library metadata and AI transition memory. The selected deployment database is prochain_aidj. Audio stays in the platform data directory; MongoDB does not store WAV/MP3 blobs.

InformationStorage
Track/sample UUIDs, tags, source/PCM hashes, current analysis referenceMongoDB tracks
Versioned analysis and per-window metricsanalyses, analysis_windows
Reusable immutable transition scriptsrecipes
Rendered attempts and automatic technical scoresattempts
Human review historyreviews, ordered by counters
Session status, decisions, reduced manifests, actor snapshotssessions, decisions, session_manifests, actors
Originals, decoded PCM, rendered mixes and complete replay artifactsFilesystem under BOOTH_DATA_DIR

BOOTH_STORAGE=mongodb selects this backend. BOOTH_MONGO_DATABASE defaults to prochain_aidj; BOOTH_MONGO_URI_FILE points to a mounted connection secret. The application account needs readWrite only on its database. Administrator credentials do not belong in runtime configuration or Git. The local helper aidj-platform/tools/run_with_storage.py loads non-secret storage settings from ignored .env.storage.json.

Without BOOTH_STORAGE=mongodb, the original local mode remains available: sidecar track.json files plus a rebuildable library.sqlite index, and durable transitions.sqlite for recipes/ratings. MongoDB mode does not keep these legacy metadata files synchronized. They remain a pre-migration snapshot; switching back later requires reverse migration to preserve new metadata and ratings.

Multiple clubs and workers

The database shares metadata across processes. Analysis updates atomically compare the expected revision; recipe versions and attempt IDs reject conflicting changes. Feature windows and decisions are separate ordered documents. Large individual documents are rejected rather than truncated. Global review sequencing preserves the latest-rating-per-reviewer rule.

Workers must also share the audio data volume, with the same UUID-relative layout, or use a separately implemented asset distribution layer. Merely connecting to MongoDB cannot make one computer's local WAV files available to another. Docker deployments need a read-only secret mount and a persistent data mount. Existing club emulator containers remain independently managed.

Each session is controlled by the daemon that launched it. Other daemons may discover sessions and read shared artifacts; they cannot pause or cancel another process's job. Distributed job leases, owner crash recovery and remote control routing are future work. A new daemon does not mark another daemon's jobs interrupted.

Migration and operations

Stop source writers, then run tools/export_metadata.py DATA_DIR in aidj-platform. It validates PCM hashes, backs up SQLite with SQLite's backup API, and copies metadata into a timestamped migration bundle. Audio remains in place and requires its own backup. storage-admin migrate BUNDLE imports metadata without replacing conflicting destination values; storage-admin verify BUNDLE checks round trips. An unchanged bundle can be retried safely. Recipe/attempt/review identities and review chronology are retained; built-in recipes are seeded. Only one migration writer should run at a time.

The selected MongoDB server is a standalone instance. This provides centralized durable storage, but no replica-set failover or multi-document transactions. Writes are atomic per document, with journaled write concern in deployment configuration. Partial batches can be retried. An ambiguous write result retains audio rather than deleting potentially committed assets. Unreferenced analysis/asset cleanup is not automated.

Back up MongoDB and filesystem assets as a coordinated dataset. Session JSON exports cannot reconstruct human ratings. Replica-set deployment, backup scheduling, object storage and distributed asset caches remain separate operational work.

Implementation and commands: platform storage guide.

Validation on 2026-09-12

The initial main-library migration preserved six tracks and ten sessions, and initialized four built-in recipes. The source had no transition-rating SQLite database. Separate synthetic test data verified recipe/attempt/review migration and repeat-run idempotence without adding test ratings to the main database. The 58-test Rust suite, real MongoDB competing-writer test, full import/analysis/audition/rating/replay API check and independent-daemon read/artifact check passed.