Music library and background processing

The music library is a dedicated section of AI-DJ Admin, using the same Keycloak login and private workspace as the virtual booth.

Pages

Page	Purpose
/library	Search title, artist and album; filter by BPM range, musical key, tracks/samples and review status
/library/queue	See queued, processing, retrying, completed, failed and cancelled track jobs, with remaining counts
/library/tracks/{id}	Listen, inspect native and musical measurements, review beat timing/key and download the full report
/library/transfer	Upload audio files, export catalog JSON, check club-export readiness and browse registered source files
/library/analysis	Select tracks and choose full analysis, audio measurements, musical analysis or indexing only

Search filters are stored in the URL. Track details have their own route, so opening measurements no longer requires scrolling below the catalog. Queue counts are tracks, not individual analysis stages or RabbitMQ messages. Each track row displays completed stages and its current stage. Registered folder backlog is shown separately on the queue page, so pending source files are not confused with runnable analysis jobs.

Uploads import the original/canonical audio into the user's library and then save a background analysis job. Uploading and initial decoding still take place in the import request; subsequent audio-feature, musical-analysis and indexing stages run independently of the browser. If import succeeds but job creation fails, the import result explains how to queue the track from the analysis page.

Catalog exports contain tags and summary metrics as JSON, not audio or full analysis timelines. Full measurement reports are available on track-detail pages. The club-readiness check identifies review/media issues; downloadable club audio bundles are not yet implemented in the web UI.

Processing path

flowchart LR
  UI[Private library UI] --> API[User-scoped Rust API]
  API --> Jobs[MongoDB processing job and outbox]
  Jobs --> Dispatcher[Background dispatcher]
  Dispatcher --> Rabbit[RabbitMQ]
  Rabbit --> Native[Audio measurements]
  Rabbit --> Neural[Musical analysis]
  Rabbit --> Index[Qdrant index]
  Native --> Jobs
  Neural --> Jobs
  Index --> Jobs
  Jobs --> UI

aidj-library-worker runs analyzerd queue-worker in prochain-aidj and connects to RabbitMQ in prochain-rabbitmq. The worker has one consumer per stage, each with prefetch one. A full track job proceeds through native → neural → index; choosing indexing alone does not decode or rerun the model. The index stage needs a compatible current musical report and fails visibly if none exists.

The native stage reuses current PCM-matched feature measurements. Musical analysis retains its immutable, content/version-addressed report cache. The result includes analyzer/model/runtime provenance. Pipeline identity is library-pipeline-20260912-v3; bump this identity when the processing contract, algorithms or configured model set changes. The ledger records input/current revision, stages, results, attempts and timestamps. It never marks a prediction as human-reviewed. Vocal scores and musical structure candidates retain their existing confidence and validation limitations.

Delivery, retries and recovery

MongoDB processing_jobs is the authoritative ledger and durable outbox. An API request writes the job there even if RabbitMQ is unavailable. The dispatcher publishes persistent ID-only messages with mandatory routing and publisher confirms. Unclaimed jobs are republished after 60 seconds, so a crash between a Mongo write and broker confirmation does not strand the job.

A partial unique index allows one active job per workspace/track. Equivalent requests reuse the stored job; concurrent duplicate deliveries cannot acquire its running lease. Workers acknowledge after the stage outcome is committed. Advancing to the next stage and making that stage eligible for dispatch is a single MongoDB update. Terminal application failures are also dead-lettered to RabbitMQ's failed queue; MongoDB remains the source used by the UI. An exhausted lease-recovery attempt may be marked failed directly in MongoDB.

Stage leases last two minutes, renew every 20 seconds and use an ownership token to reject stale writes. An individual stage has a 30-minute deadline and at most three attempts. Transient failures use the 30-second RabbitMQ retry queue. Worker termination/expired leases recover from the last incomplete stage. Successful earlier stages remain recorded. A native job marker is saved atomically with its track revision: if that write succeeds before a worker stops, replay recovers it without incrementing the revision again. A human review clears the marker, so stale work cannot be mistaken for the reviewed version. Two outstanding jobs per workspace limit how much one large library can place ahead of other users; this is a bounded-backlog fairness policy, not a strict weighted scheduling guarantee.

Users can cancel jobs that are queued or waiting to retry, and retry failed or cancelled jobs. Running jobs cannot currently be cancelled in the UI. If a track was reviewed or changed during analysis, create a fresh job for the new revision instead of repeatedly retrying stale input. A retry does not alter its input revision. Broker failed messages are diagnostic references; retrying a Mongo job does not automatically delete its old dead-letter reference.

Tenant boundary

Each API derives the job's scope from its configured Mongo collection prefix; the browser cannot submit an owner, data path or Mongo prefix. List, enqueue and control endpoints enforce that scope. Track IDs are checked against the caller's library before enqueueing. Users do not get broker credentials or global broker queue lengths.

The shared worker is privileged backend infrastructure. It verifies the private workspace exists, derives the data directory from the scope, and launches a new analyzer process for each stage with that workspace's Mongo prefix and Qdrant collection. This avoids reusing process-global storage configuration across users. Each stage validates the stored PCM identity and expected analysis revision. Shared-booth membership does not grant access to another user's queue. Messages for a deleted workspace are acknowledged without processing and its outstanding job is marked failed, preventing an orphaned task from blocking a shared queue.

Operations

kubectl -n prochain-aidj logs deployment/aidj-library-worker
kubectl -n prochain-aidj rollout status deployment/aidj-library-worker
kubectl -n prochain-rabbitmq exec rabbitmq-0 -- rabbitmqctl list_queues -p prochain_aidj name messages consumers

Enable the API with BOOTH_QUEUE_ENABLED=true. The worker mounts only the amqp-url field from aidj-rabbitmq, plus existing scoped database/vector credentials, shared audio storage and model assets. Broker credentials remain backend-only. Worker readiness checks its recent dispatch heartbeat; the UI also reports recent worker presence from MongoDB. MongoDB and RabbitMQ must both be available for new stages to start. Existing private runtime pods need the updated platform image/configuration; newly provisioned workspaces use the GitOps template.

The existing suspended analyzer CronJob and one-track CLI helper remain available for operator diagnostics. The Windows source folder has not been mounted in Kubernetes, and full-folder ingestion remains paused. Registered source inventory is shown separately on Import/export; its pending files are not represented as already imported analysis jobs. This release does not silently start a whole source scan or copy the Windows music collection.

See RabbitMQ operations for credentials, topology, persistence, network policy and single-node availability limitations.