Audiolla
Self-hosted MCP server: audio stems, mastering, MIR analysis, DSP, MIDI, speech tools.
Open source Open in the app JSON README (API)
About
Self-hosted MCP server: audio stems, mastering, MIR analysis, DSP, MIDI, speech tools.
Details
- Kind
- MCP servers
- Topic
- No topic detected
- Publisher
- psyb0t
- Origin
- official
- Category
- ferramentas
- Transport
- local
- Version
- 1.1.7
- Stars
- 3
- Forks
- 2
- Last push
- 2026-08-01T19:53:53Z
- Repository state
- ativo
- Language
- Python
- License
- WTFPL
- Added
- 2026-08-29 04:01:13
- Updated
- 2026-08-29 04:01:13
- Origin id
io.github.psyb0t/audiolla
README
# audiolla
[](https://github.com/psyb0t/docker-audiolla/actions/workflows/pipeline.yml)
[](https://github.com/psyb0t/docker-audiolla/releases)
[](LICENSE)
[](https://hub.docker.com/r/psyb0t/audiolla)
**Thirty audio engines. One port. Zero cloud. Fire-and-forget async jobs. Webhooks.**
You needed Demucs for stems. Then librosa for BPM and key. Then basic-pitch for MIDI transcription. Then pyannote for speaker diarization. Then DeepFilterNet for speech enhancement. Then you spent three days debugging Python version conflicts and now you hate everything.
audiolla is what happens when you stop doing that.
Every audio processing tool worth using — wrapped in one HTTP API, running in one Docker container. POST a file. Get audio, JSON, or MIDI back. Drive it from curl, shell scripts, Python notebooks, Makefiles, or point an LLM agent at the MCP endpoint and let it rip.
No account. No subscription. No per-minute billing. No vendor lock-in. `docker run` and you're done.
---
## What's in the box
| | |
|--|--|
| 🎛️ **Stem separation** | Demucs — htdemucs, fine-tuned, 6-stem, MDX variants |
| 🎚️ **Mastering** | Reference mastering (matchering) + custom pedalboard chains |
| 📊 **Analysis** | BPM · key · LUFS · beats · onsets · melody · structural segments |
| 🎹 **Chords + key** | Chord detection + Krumhansl-Schmuckler key estimation |
| 🎵 **Audio → MIDI** | Polyphonic transcription via Spotify's basic-pitch (ONNX, no TF) |
| 🧹 **Restoration** | De-reverb · de-echo · de-noise via UVR BS-Roformer + MelBand Roformer |
| 🗣️ **Speech** | Enhancement (DeepFilterNet) · VAD (silero-vad) · diarization (pyannote) |
| 🖼️ **Visuals** | Spectrogram + waveform PNGs + 8-mode animated MP4/WebM |
| 🔍 **Fingerprint** | Chromaprint acoustic fingerprinting (AcoustID-compatible) |
| ✂️ **Silence** | Detect gaps · trim edges · strip all silence |
| 🎼 **MIDI pipeline** | Compose from JSON · inspect · transform · render via fluidsynth |
| 🎸 **Effects** | 23-effect pedalboard chain — Compressor, Reverb, PitchShift, filters… |
| 🔧 **Transforms** | Sox DSP — pitch, tempo, EQ, reverb, gain |
| 📢 **Loudness** | Measure LUFS · normalize to target |
| 🥁 **HPSS** | Harmonic/percussive source separation via librosa median filter |
| 🔇 **Noise reduction** | Spectral noise reduction via noisereduce — stationary + adaptive modes |
| ⏩ **Time-stretch** | Independent tempo factor + pitch shift via librosa phase vocoder |
| 🏷️ **Audio tagging** | Top-K AudioSet class labels via Audio Spectrogram Transformer |
| 🔗 **Audio embeddings** | 512-dim semantic embeddings via LAION CLAP + optional text similarity |
| 🏷️ **Zero-shot classify** | CLAP cosine similarity against any free-form text labels — genres, moods, instruments |
| 📋 **Audio info** | ffprobe metadata — duration, sample rate, channels, codec, bit depth |
| ✂️ **Trim** | Cut a clip by start/end seconds — any format in, any format out |
| 🎚️ **Mix** | Combine N staged tracks with per-track gain_db — pure ffmpeg, no model |
| 🔗 **Concat** | Stitch N audio files end-to-end in order |
| ⏩ **Speed** | Change playback speed without pitch shift (0.1× – 10×) via ffmpeg atempo |
| 🔄 **Convert** | Re-encode: format, sample rate, channel count in one call |
| 🔍 **Similar** | Cosine similarity between two audio files via CLAP embeddings |
| 🎹 **MIDI quantize** | Snap MIDI note timings to a rhythmic grid (16th, 8th, quarter…) |
| 🌅 **Fade** | Fade-in and/or fade-out with 13 curve shapes |
| ⏪ **Reverse** | Flip audio backwards |
| 🔁 **Loop** | Repeat audio N times |
| 🎯 **BPM match** | Auto-detect BPM then stretch to a target — no manual math |
| 📈 **Loudness curve** | RMS envelope over time — time-stamped dB values for gain automation |
| 🎤 **Pitch correct** | Auto-tune toward nearest chromatic semitone — configurable strength |
| 🔧 **Repair** | Declip + dehum — fix clipped peaks and remove power-line hum |
| 🔁 **Loop point** | Find best seamless loop boundary — score, bar count, candidates list |
| 🥁 **Drum machine** | Step-sequencer spec → GM drum MIDI — 16-step pattern, swing, tempo |
| 🎼 **Chords to MIDI** | Chord progression → MIDI file — root+3rd+5th voicings per segment |
| ↔️ **Stereo width** | Widen or collapse the stereo image via M/S processing |
| ✂️ **Split** | Split into N equal parts or on silence — returns ZIP of segments |
| 🔊 **Pan** | Position audio in the stereo field (-1 left → 0 center → 1 right) |
| 🎚️ **EQ** | Parametric EQ — JSON array of freq/gain_db/width_hz bands |
| 🎵 **Key match** | Detect source key then pitch-shift to a target key |
| 🎙️ **Sidechain duck** | Duck music when a trigger track (voice) is loud |
| 🏷️ **Metadata** | Read and write ID3/Vorbis/FLAC/WAV audio tags via mutagen |
| 🔴 **Clip detect** | Detect digital clipping — count, ratio, peak dBFS |
| ↔️ **Mid/Side** | Encode L/R → Mid+Side or decode Mid+Side → L/R |
| ✂️ **Beat slice** | Slice audio at detected beat positions — returns ZIP of segments |
| 🏟️ **Conv reverb** | Convolution reverb via impulse response — wet_mix control |
| 🥁 **Transient shaper** | Attack/sustain dual-compressor — punch up drums, cut room tail |
| 🎚️ **Multiband compress** | N-band compressor with zero-phase LR4 crossovers — mastering-grade dynamics |
| 🎛️ **DJ prep** | One call: BPM + key + Camelot wheel position + integrated LUFS |
| 📦 **Batch** | Run trim/convert/fade/reverse/speed/eq on staged files in sequence |
| 🧩 **Presets + pipeline** | Curated YAML workflows (`master-for-spotify`, `podcast-cleanup`, …) + ad-hoc op chaining server-side |
| 🗂️ **Catalog** | `GET /v1/catalog` — machine-readable endpoint list grouped by category for discovery |
| ⚡ **Async jobs** | Every endpoint supports `async_job=true` — fire-and-forget + webhook callbacks |
---
## Table of Contents
- [Run it](#run-it)
- [Quick start](#quick-start)
- [What it can do](#what-it-can-do)
- [Split stems](#split-stems)
- [Master](#master)
- [Analyze](#analyze)
- [Beats, onsets, melody, segments](#beats-onsets-melody-segments)
- [Silence detection and trimming](#silence-detection-and-trimming)
- [Visualize (spectrogram, waveform, video)](#visualize-spectrogram-waveform-video)
- [Acoustic fingerprint](#acoustic-fingerprint)
- [De-reverb, de-echo, de-noise](#de-reverb-de-echo-de-noise)
- [Audio-to-MIDI transcription](#audio-to-midi-transcription)
- [Neural speech and vocal enhancement](#neural-speech-and-vocal-enhancement)
- [Chord and key detection](#chord-and-key-detection)
- [Voice activity detection](#voice-activity-detection)
- [Speaker diarization](#speaker-diarization)
- [Transform](#transform)
- [Loudness measurement](#loudness-measurement)
- [Loudness curve](#loudness-curve)
- [Loudness normalization](#loudness-normalization)
- [HPSS (harmonic/percussive split)](#hpss-harmonicpercussive-split)
- [Spectral noise reduction](#spectral-noise-reduction)
- [Time-stretch and pitch-shift](#time-stretch-and-pitch-shift)
- [Pitch correct](#pitch-correct)
- [Repair](#repair)
- [Audio tagging](#audio-tagging)
- [Audio embeddings](#audio-embeddings)
- [Zero-shot classification](#zero-shot-classification)
- [Audio info](#audio-info)
- [Trim](#trim)
- [Mix](#mix)
- [Concat](#concat)
- [Speed](#speed)
- [Convert](#convert)
- [Similar](#similar)
- [MIDI quantize](#midi-quantize)
- [Fade](#fade)
- [Reverse](#reverse)
- [Loop](#loop)
- [BPM match](#bpm-match)
- [Stereo width](#stereo-width)
- [Split](#split)
- [Pan](#pan)
- [EQ](#eq)
- [Key match](#key-match)
- [Sidechain duck](#sidechain-duck)
- [Effects chain](#effects-chain)
- [Loop point](#loop-point)
- [Compose MIDI](#compose-midi)
- [Inspect MIDI](#inspect-midi)
- [Transform MIDI](#transform-midi)
- [Render MIDI to audio](#render-midi-to-audio)
- [Generate music from a spec](#generate-music-from-a-spec)
- [Drum pattern](#drum-pattern)
- [Chords to MIDI](#chords-to-midi)
- [Audio metadata tags](#audio-metadata-tags)
- [Clip detection](#clip-detection)
- [Mid/Side encode and decode](#midside-encode-and-decode)
- [Beat slice](#beat-slice)
- [Convolution reverb](#convolution-reverb)
- [Transient shaper](#transient-shaper)
- [Multiband compression](#multiband-compression)
- [DJ prep](#dj-prep)
- [De-ess](#de-ess)
- [Stereo field analysis](#stereo-field-analysis)
- [Audio thumbnail](#audio-thumbnail)
- [MIDI humanize](#midi-humanize)
- [Batch operations](#batch-operations)
- [Async jobs and webhooks](#async-jobs-and-webhooks)
- [Stage files](#stage-files)
- [Remote URLs](#remote-urls)
- [Engines](#engines)
- [Workflows — presets + pipeline](#workflows--presets--pipeline)
- [API catalog](#api-catalog)
- [Endpoints](#endpoints)
- [MCP](#mcp)
- [Agent integrations](#agent-integrations)
- [Configuration](#configuration)
- [What's not in here](#whats-not-in-here)
- [Build & dev](#build--dev)
- [Supply chain](#supply-chain)
- [License](#license)
---
## Run it
```bash
# no GPU
docker run --rm -it \
-v $HOME/.audiolla-data:/data \
-p 8000:8000 \
psyb0t/audiolla:latest
# GPU
docker run --rm -it --gpus all \
-v $HOME/.audiolla-data:/data \
-e AUDIOLLA_DEVICE=cuda \
-p 8000:8000 \
psyb0t/audiolla:latest-cuda
```
Demucs weights prefetch at container startup (for whichever variants are enabled) and cache in `/data/torch_cache/`. First boot downloads them; same `-v` mount next time and they're already there. Other engines (matchering, pedalboard, librosa, sox, fx, midi) have no weights — they're ready as soon as `/healthz` is green.
---
## Migration from v0.23.x → v1.0.0
**v1.0.0 is a breaking API release.** Every existing client breaks. The new shape:
- Every audio endpoint takes a **JSON body** (no more `multipart/form-data` except at `/v1/files`)
- Input is **`file_path`** (FILES_DIR-relative) xor **`file_url`** (server-side fetch). Pre-stage the file via `PUT /v1/files/{path}` first.
- Output requires **`output_path`** xor **`output_url`**. No more raw audio bytes in responses.
- Async path: `async_job=true` auto-stages to `jobs/{id}.{ext}` if neither output is given.
- MCP audio-producing tools dropped `audio_base64` (and `midi_base64` / `image_base64` / `video_base64`). Same `output_path` xor `output_url` requirement.
- `openapi.yaml` is now the contract — Pydantic models regenerate from it via `make generate`. Never hand-edit `src/audiolla/schema/_generated.py`.
```diff
- curl -X POST http://localhost:8000/v1/audio/normalize \
- -F "file=@track.wav" -F "target_lufs=-14" -o normalized.wav
+ # 1) stage the file (multipart only lives here now)
+ curl -X PUT --data-binary @track.wav \
+ -H 'Content-Type: application/octet-stream' \
+ http://localhost:8000/v1/files/uploads/track.wav
+ # 2) process via JSON body — response is JSON, not bytes
+ curl -X POST http://localhost:8000/v1/audio/normalize \
+ -H 'Content-Type: application/json' \
+ -d '{"file_path":"uploads/track.wav","target_lufs":-14,"output_path":"out/normalized.wav"}'
+ # 3) retrieve the result
+ curl -o normalized.wav http://localhost:8000/v1/files/out/normalized.wav
```
Why? See the [v1.0.0 CHANGELOG entry](CHANGELOG.md) for the full rationale.
## Quick start
Once the container is up, this is a complete audio pipeline in six commands (every audio endpoint is JSON-body now; stage your input file at `/v1/files/...` first):
```bash
# stage your input file
curl -X PUT --data-binary @song.wav \
-H 'Content-Type: application/octet-stream' \
http://localhost:8000/v1/files/uploads/song.wav
# rip the vocals out of a track
curl -X POST http://localhost:8000/v1/audio/separate \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/song.wav","engine":"htdemucs","stems":["vocals"],"output_path":"out/vocals.wav"}'
# → {"path":"out/vocals.wav","size":...,"output_format":"wav"}
curl -o vocals.wav http://localhost:8000/v1/files/out/vocals.wav
# what key is it in? what are the chords?
curl -X POST http://localhost:8000/v1/audio/chords \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/song.wav"}'
# → {"key":"F# minor","key_confidence":0.91,"chords":[{"chord":"F#m","start_sec":0.0,...},...]}
# transcribe that vocal melody to MIDI
curl -X PUT --data-binary @out/vocals.wav -H 'Content-Type: application/octet-stream' \
http://localhost:8000/v1/files/uploads/vocals.wav # only if not already staged
curl -X POST http://localhost:8000/v1/audio/to_midi/basic-pitch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocals.wav","output_path":"out/melody.mid"}'
# render the MIDI back to audio through a SoundFont
curl -X POST http://localhost:8000/v1/midi/render \
-H 'Content-Type: application/json' \
-d '{"file_path":"out/melody.mid","output_path":"out/rendered.wav"}'
# strip background noise from a voice recording
curl -X POST http://localhost:8000/v1/audio/noise-reduce/uvr-denoise \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/interview.wav","output_path":"out/clean.wav"}'
# who's speaking and when?
curl -X POST http://localhost:8000/v1/audio/diarize/pyannote \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/interview.wav"}'
# → {"num_speakers":2,"segments":[{"speaker":"SPEAKER_00","start_sec":0.5,"end_sec":8.2},...]}
```
Audio in. MIDI out. Chords detected. Speakers identified. De-noised. Re-synthesized. No Python environment to set up. No API keys. No account. Just HTTP.
---
## What it can do
Output defaults to `wav`. Add `"output_format":"mp3"` to the JSON body to get mp3 instead (`flac`, `opus`, `aac`, `pcm` also work).
Every audio endpoint takes an `application/json` body. The only place multipart still lives is `PUT /v1/files/{path}` (raw bytes for staging an input file).
**Input** — every audio endpoint requires exactly one of:
- `file_path` — path inside the `/v1/files` staging area (stage with `PUT /v1/files/{path}` first)
- `file_url` — remote URL the server fetches (disabled by default — see [Remote URLs](#remote-urls))
**Output** — audio-producing endpoints require exactly one of:
- `output_path` — server writes to `/v1/files/<path>`, returns JSON `{"path":..., "size":..., ...}`
- `output_url` — server PUTs to a presigned URL, returns JSON `{"url":..., "size":..., ...}`
Analysis-only endpoints (those that return JSON data, e.g. `/v1/audio/analyze`, `/v1/audio/loudness`, `/v1/audio/info`) don't need `output_path` / `output_url` — the response is the result.
### Split stems
```bash
# stage input
curl -X PUT --data-binary @track.wav \
-H 'Content-Type: application/octet-stream' \
http://localhost:8000/v1/files/uploads/track.wav
# vocals only
curl -X POST http://localhost:8000/v1/audio/separate \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","engine":"htdemucs","stems":["vocals"],"output_path":"out/vocals.wav"}'
curl -o vocals.wav http://localhost:8000/v1/files/out/vocals.wav
# all 4 stems as a ZIP
curl -X POST http://localhost:8000/v1/audio/separate \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","engine":"htdemucs","output_path":"out/stems.zip"}'
curl -o stems.zip http://localhost:8000/v1/files/out/stems.zip
```
### Master
```bash
# stage track + reference
curl -X PUT --data-binary @track.wav -H 'Content-Type: application/octet-stream' \
http://localhost:8000/v1/files/uploads/track.wav
curl -X PUT --data-binary @ref.wav -H 'Content-Type: application/octet-stream' \
http://localhost:8000/v1/files/uploads/ref.wav
# match EQ + loudness to a reference track
curl -X POST http://localhost:8000/v1/audio/master \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","mode":"reference","reference_path":"uploads/ref.wav","output_path":"out/mastered.wav"}'
curl -o mastered.wav http://localhost:8000/v1/files/out/mastered.wav
# run a built-in pedalboard chain (presets: transparent, loud)
curl -X POST http://localhost:8000/v1/audio/master \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","mode":"chain","preset":"loud","output_path":"out/mastered.wav"}'
curl -o mastered.wav http://localhost:8000/v1/files/out/mastered.wav
```
### Analyze
```bash
# returns JSON. features: bpm, key, loudness, duration,
# spectral_centroid, rms, zcr. Omit features to get them all.
curl -X POST http://localhost:8000/v1/audio/analyze \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","features":["bpm","key","loudness"]}'
```
### Beats, onsets, melody, segments
```bash
# beat grid — returns bpm + beat timestamps
curl -X POST http://localhost:8000/v1/audio/beats \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# onset timestamps — note attacks, transients
curl -X POST http://localhost:8000/v1/audio/onsets \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# dominant melody contour — pitch in Hz per frame
curl -X POST http://localhost:8000/v1/audio/melody \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# structural segmentation — labels recurring sections A, B, C...
curl -X POST http://localhost:8000/v1/audio/segments \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","num_segments":6}'
```
Beat detection also generates a click-track file when `click_track=true` (set `output_path` to receive it) — handy for aligning a mix to a grid. Pass `start_bpm=140` to seed the tracker when you already know the rough tempo (faster, more accurate). Melody can be exported as a single-track MIDI file via `as_midi=true` + `output_path`.
### Silence detection and trimming
```bash
# find silent gaps in a recording (no trim_mode → JSON only)
curl -X POST http://localhost:8000/v1/audio/silence \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","threshold_db":-30,"min_duration_sec":1.0}'
# trim all silence and stage the result
curl -X POST http://localhost:8000/v1/audio/silence \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","threshold_db":-30,"min_duration_sec":0.5,"trim_mode":"all","output_path":"out/trimmed.wav"}'
curl -o trimmed.wav http://localhost:8000/v1/files/out/trimmed.wav
# trim only leading/trailing silence
curl -X POST http://localhost:8000/v1/audio/silence \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","threshold_db":-40,"min_duration_sec":0.3,"trim_mode":"edges","output_path":"processed/trimmed.wav"}'
```
`trim_mode=edges` — chop leading + trailing silence only. `trim_mode=all` — remove every detected gap (compress a talk recording, tighten a loop). Without `trim_mode`, the response is JSON only: `silent_ranges`, `non_silent_ranges`, `duration` — and `output_path` / `output_url` is not required.
### Visualize (spectrogram, waveform, video)
Visual output splits into two sub-namespaces by output type:
```bash
# Static PNG spectrogram (color + scale params)
curl -X POST http://localhost:8000/v1/audio/visualize/image/spectrogram \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","width":1280,"height":720,"output_path":"out/spec.png"}'
curl -o spec.png http://localhost:8000/v1/files/out/spec.png
# Static PNG waveform (color param)
curl -X POST http://localhost:8000/v1/audio/visualize/image/waveform \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","width":1280,"height":240,"output_path":"out/wave.png"}'
curl -o wave.png http://localhost:8000/v1/files/out/wave.png
# Animated MP4 spectrum analyser (fps + container params)
curl -X POST http://localhost:8000/v1/audio/visualize/video/spectrum \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","width":1280,"height":720,"fps":30,"container":"mp4","output_path":"out/viz.mp4"}'
curl -o viz.mp4 http://localhost:8000/v1/files/out/viz.mp4
```
**`/image/spectrogram`**: produces a PNG (staged via `output_path` or PUT to `output_url`). Params: `width`, `height`, `color` (default `intensity`), `scale` (`log`/`lin`).
**`/image/waveform`**: produces a PNG. Params: `width`, `height`, `color` (default `lime`).
**`/video/{mode}`**: `spectrum` (scrolling FFT), `waves` (oscilloscope), `cqt` (constant-Q transform), `freqs` (bar-graph analyzer), `volume` (VU meter), `vectorscope` (stereo X/Y scope), `phasemeter`, `histogram`. Params: `width`, `height`, `fps`, `container` (`mp4` default, `webm`).
### Acoustic fingerprint
```bash
# Chromaprint fingerprint — identifies a recording regardless of encoding
curl -X POST http://localhost:8000/v1/audio/fingerprint \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# → {"duration": 215.34, "fingerprint": "AQADtEqRRIuQ..."}
# include the raw integer array (for custom similarity scoring)
curl -X POST http://localhost:8000/v1/audio/fingerprint \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","return_raw":true}'
```
The base64 fingerprint string is compatible with the [AcoustID](https://acoustid.org) lookup service.
### De-reverb, de-echo, de-noise
AI audio restoration via UVR ecosystem models — BS-Roformer and MelBand Roformer. All three are unified under `POST /v1/audio/restore/{engine}`.
```bash
# Remove room reverb (BS-Roformer, SDR 19+)
curl -X POST http://localhost:8000/v1/audio/restore/uvr-dereverb \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","output_path":"out/dry.wav"}'
# Remove echo — normal mode
curl -X POST http://localhost:8000/v1/audio/restore/uvr-deecho \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","output_path":"out/noecho.wav"}'
# Remove echo — aggressive mode (same engine, harder suppression)
curl -X POST http://localhost:8000/v1/audio/restore/uvr-deecho \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","aggressive":true,"output_path":"out/noecho.wav"}'
# Remove broadband background noise — ML (MelBand Roformer, SDR 28)
curl -X POST http://localhost:8000/v1/audio/restore/uvr-denoise \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","output_path":"out/clean.wav"}'
```
All support `output_format`, `output_path`, `output_url`. For DSP-based noise reduction (no GPU) use `noise-reduce/noise-reduce`.
UVR engines also work through `/v1/audio/separate` — `uvr-vocal-bsr` (BS-Roformer, SDR 13) and `uvr-karaoke` return vocal + instrumental stems like Demucs but often with higher quality.
### Audio-to-MIDI transcription
Polyphonic audio-to-MIDI via Spotify's basic-pitch (ONNX backend, no TensorFlow). Play guitar, hum a melody, record a piano riff — get a MIDI file back with all the notes.
```bash
# Any audio → MIDI file (staged)
curl -X POST http://localhost:8000/v1/audio/to_midi/basic-pitch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/guitar_riff.wav","output_path":"out/riff.mid"}'
curl -o riff.mid http://localhost:8000/v1/files/out/riff.mid
# Tune the detection thresholds
curl -X POST http://localhost:8000/v1/audio/to_midi/basic-pitch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/piano.wav","onset_threshold":0.6,"frame_threshold":0.3,"minimum_note_length_ms":80,"output_path":"out/piano.mid"}'
# Write directly to a different staging path
curl -X POST http://localhost:8000/v1/audio/to_midi/basic-pitch \
-H 'Content-Type: application/json' \
-d '{"file_path":"recordings/bass.wav","output_path":"midi/bass_notes.mid"}'
# → {"path":"midi/bass_notes.mid","size":...,"engine":"basic-pitch","output_format":"mid"}
```
Optional params: `onset_threshold` (0–1, default 0.5), `frame_threshold` (0–1, default 0.3), `minimum_note_length_ms` (default 58), `minimum_frequency` / `maximum_frequency` (Hz, default unconstrained), `multiple_pitch_bends` (bool, default false), `melodia_trick` (bool, default true — helps with melodic content). Default engine: `basic-pitch`.
The MIDI file is piped straight into `/v1/midi/inspect` or `/v1/midi/render` — audio → MIDI → audio is a complete round-trip.
### Neural speech and vocal enhancement
DeepFilterNet DF3 — deep learning noise suppression trained on speech. Better than broadband de-noise for voice recordings; more surgical than UVR's de-noise on vocals specifically.
```bash
# Enhance a vocal recording
curl -X POST http://localhost:8000/v1/audio/enhance/deepfilter \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal_recording.wav","output_path":"out/enhanced.wav"}'
curl -o enhanced.wav http://localhost:8000/v1/files/out/enhanced.wav
# Stage the output as mp3
curl -X POST http://localhost:8000/v1/audio/enhance/deepfilter \
-H 'Content-Type: application/json' \
-d '{"file_path":"vocals/raw.wav","output_format":"mp3","output_path":"vocals/enhanced.mp3"}'
```
Supports `output_format`, `output_path`, `output_url`.
### Generate music + SFX
Text-to-audio generation under `POST /v1/audio/generate/{engine}`. v1.0.0 ships **five engines** spanning music + sound effects, with different licence / VRAM / sound profiles — all CUDA-only.
```bash
# Stable Audio Open 1.0 — 47s cap, no vocals, great for loops + SFX
curl -X POST http://localhost:8000/v1/audio/generate/stable-audio-open \
-H 'Content-Type: application/json' \
-d '{"prompt":"130 bpm tech house drum loop, punchy kick, crisp hats, no vocals","duration_sec":10,"seed":42,"output_path":"out/loop.wav"}'
curl -o loop.wav http://localhost:8000/v1/files/out/loop.wav
# MusicGen 300M — 30s cap, instrumental, CC-BY-NC (opt-in required)
curl -X POST http://localhost:8000/v1/audio/generate/musicgen-small \
-H 'Content-Type: application/json' \
-d '{"prompt":"lo-fi hip-hop beat with vinyl crackle, 90 bpm","duration_sec":15,"output_path":"out/beat.wav"}'
# Riffusion — spectrogram-to-audio via Griffin-Lim, ~5s, lo-fi character
curl -X POST http://localhost:8000/v1/audio/generate/riffusion \
-H 'Content-Type: application/json' \
-d '{"prompt":"ambient drone with metallic resonance","output_path":"out/drone.wav"}'
# AudioLDM 2 — general SFX (no opt-in gate, CC-BY 4.0 commercial-OK)
curl -X POST http://localhost:8000/v1/audio/generate/audioldm2 \
-H 'Content-Type: application/json' \
-d '{"prompt":"heavy rain on a metal roof with distant thunder","duration_sec":10,"num_inference_steps":50,"output_path":"out/rain.wav"}'
```
**Engine details:**
| Engine | Licence | Max length | VRAM (fp16) | Output |
|---|---|---|---|---|
| `stable-audio-open` | **Stability Community Licence** (commercial OK below the revenue threshold) | 47 s hard cap | ~12 GB | 44.1 kHz stereo. Loops, SFX, ambient textures — instrumental only |
| `musicgen-small` | **CC-BY-NC 4.0** (non-commercial only — opt-in via `AUDIOLLA_ENABLE_NONCOMMERCIAL=1`) | 30 s hard cap | ~3 GB | 32 kHz mono. Meta MusicGen 300M, instrumental |
| `musicgen-medium` | **CC-BY-NC 4.0** (same opt-in) | 30 s hard cap | ~6-8 GB | 32 kHz mono. Higher quality than -small |
| `riffusion` | **CreativeML OpenRAIL-M** (commercial OK with the licence's usage restrictions) | ~5 s per pass | ~3 GB | 22.05 kHz mono. SD-style spectrogram, Griffin-Lim reconstruction — lo-fi / loop-y character |
| `audioldm2` | **CC-BY 4.0** (commercial use OK — no opt-in gate) | 30 s hard cap | ~8-10 GB (CPU offload) | 16 kHz mono. General SFX: ambience, foley, animal, mechanical, impact sounds. Slow (200-step DDIM default; pass `num_inference_steps=50` for ~4x speedup) |
All engines support `async_job=true`, `webhook_url`, `output_path`, `output_url`, and `seed` for reproducibility. `stable-audio-open` and `audioldm2` additionally accept `num_inference_steps` (trade quality for speed). Model weights download on first call to `HF_HOME` (default `/data/hf` inside the container — ~7 GB across all five). Subsequent calls are inference-only. All five are flagged `cuda_only` — non-CUDA hosts get HTTP 400.
**Licence opt-in for MusicGen.** MusicGen weights are CC-BY-NC 4.0. The engine code ships with the image but refuses to load the model unless the operator explicitly sets `AUDIOLLA_ENABLE_NONCOMMERCIAL=1` in the server's environment. Same pattern matchering (GPL v3) follows — licence-encumbered code in the image, conscious opt-in to actually use it. Read [the MusicGen weights licence](https://github.com/facebookresearch/audiocraft/blob/main/LICENSE_weights) before opting in. **AudioLDM 2 is CC-BY 4.0** (commercial use allowed, no opt-in gate) — it's the only generator in this set that's commercial-safe without flipping any flags.
**Deferred to a future release** (researched but not shipped in v1.0.0):
- **ACE-Step v1** (3.5B, Apache 2.0, full songs with vocals up to 4 min) — requires `AceStepPipeline` from `diffusers>=0.38`, which itself requires a pre-release `safetensors`. Doesn't pass the project's hash-locked supply-chain gate. Revisit when `safetensors 0.8.x` ships stable, or vendor ACE-Step's pipeline directly.
- **DiffRhythm full v1.2** (Apache 2.0) — unpackaged research repo (no `setup.py` / PyPI release). Revisit when upstream ships a package or we vendor under `thirdparty/`.
- **Stable Audio Open Small** (Stability Community Licence, 11 s SFX-specialist) — requires `stable-audio-tools` which pins `python >=3.10, <3.11`; audiolla is on Python 3.12, hard incompatibility. Revisit when `stable-audio-tools` widens the Python constraint or diffusers grows a pipeline for it.
- **TangoFlux** (ICLR 2026, 44.1 kHz, 30 s, fast) — git-only install (no PyPI package). Could be SHA-pinned in the hash-locked supply chain; deferred for now to keep the heavy-deps stack PyPI-only.
- **AudioGen** (Meta, CC-BY-NC) — `audiocraft==1.3.0` pins `transformers<=4.31.0`, hard conflict with audiolla's 4.51.3. Would require an isolated subprocess / sidecar container.
- **YuE 7B** (Apache 2.0, full songs with vocals) — needs 16-24 GB VRAM at fp16, doesn't fit 12 GB GPUs without int4 quant tooling.
### Chord and key detection
Krumhansl-Schmuckler key estimation + chroma-template chord segmentation via librosa. No extra deps beyond the librosa stack.
```bash
curl -X POST http://localhost:8000/v1/audio/chords \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# → {
# "key": "C major",
# "key_confidence": 0.87,
# "duration": 183.4,
# "chords": [
# {"chord": "C", "start_sec": 0.0, "end_sec": 2.3, "confidence": 0.91},
# {"chord": "Am", "start_sec": 2.3, "end_sec": 4.6, "confidence": 0.85},
# ...
# ]
# }
# Tune the hop length (lower = finer time resolution)
curl -X POST http://localhost:8000/v1/audio/chords \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","hop_length":256}'
```
Optional params: `hop_length` (default 512), `segment_min_duration_sec` (default 0.5 — merge very short chord segments).
### Voice activity detection
silero-vad — ONNX-based VAD, fast and accurate on both speech and music. Returns timestamped speech and non-speech segments.
```bash
curl -X POST http://localhost:8000/v1/audio/vad \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/interview.wav"}'
# → {
# "speech_ratio": 0.73,
# "duration": 120.0,
# "threshold": 0.5,
# "speech_segments": [
# {"start_sec": 1.2, "end_sec": 8.4},
# ...
# ],
# "non_speech_segments": [
# {"start_sec": 0.0, "end_sec": 1.2},
# ...
# ]
# }
# Tighter detection
curl -X POST http://localhost:8000/v1/audio/vad \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/podcast.wav","threshold":0.7,"min_speech_duration_ms":300,"min_silence_duration_ms":200}'
```
Optional params: `threshold` (0–1, default 0.5), `min_speech_duration_ms` (default 250), `min_silence_duration_ms` (default 100).
### Speaker diarization
pyannote/speaker-diarization-3.1 — state-of-the-art speaker diarization from HuggingFace Hub. Returns per-speaker timestamped segments and speaker count.
> **Note:** This engine requires a HuggingFace account. You must accept the model terms at
> [https://huggingface.co/pyannote/speaker-diarization-3.1](https://huggingface.co/pyannote/speaker-diarization-3.1)
> and then set `HF_TOKEN` (or the older alias `HUGGINGFACE_TOKEN` — the entrypoint mirrors them both ways) when starting the container. A read-only token with model access is enough. The same token also unlocks the gated text-to-audio engines (`stable-audio-open`, `musicgen-small`, `musicgen-medium`) provided you've accepted their licences on huggingface.co.
```bash
docker run ... \
-e HF_TOKEN=hf_your_token_here \
psyb0t/audiolla:latest
```
```bash
curl -X POST http://localhost:8000/v1/audio/diarize/pyannote \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/interview.wav"}'
# → {
# "num_speakers": 2,
# "speakers": ["SPEAKER_00", "SPEAKER_01"],
# "duration": 120.0,
# "segments": [
# {"speaker": "SPEAKER_00", "start_sec": 0.5, "end_sec": 8.2, "duration_sec": 7.7},
# {"speaker": "SPEAKER_01", "start_sec": 8.5, "end_sec": 14.1, "duration_sec": 5.6},
# ...
# ]
# }
# Hint the expected speaker count
curl -X POST http://localhost:8000/v1/audio/diarize/pyannote \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/roundtable.wav","num_speakers":4}'
# Or constrain the range
curl -X POST http://localhost:8000/v1/audio/diarize/pyannote \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/panel.wav","min_speakers":2,"max_speakers":6}'
```
Optional params: `num_speakers` (exact count hint), `min_speakers`, `max_speakers`.
### Transform
```bash
# pitch shift up 2 semitones + add reverb, export mp3.
# operations is a JSON array — ops: gain, equalizer, compand, reverb,
# pitch, tempo, rate, channels, trim, pad.
curl -X POST http://localhost:8000/v1/audio/transform \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","operations":[{"op":"pitch","params":{"n_semitones":2}},{"op":"reverb","params":{"reverberance":50}}],"output_format":"mp3","output_path":"out/out.mp3"}'
curl -o out.mp3 http://localhost:8000/v1/files/out/out.mp3
```
### Loudness measurement
```bash
# Measure integrated LUFS — returns JSON, no audio output
curl -X POST http://localhost:8000/v1/audio/loudness \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# → {"loudness_lufs": -18.4}
```
### Loudness curve
RMS envelope over time — returns a list of `{time_sec, rms_db}` points. Useful for generating gain automation curves, finding loud and quiet sections, or visualising dynamic range before mastering.
```bash
# Default hop (512 samples) — fine-grained envelope
curl -X POST http://localhost:8000/v1/audio/loudness/curve \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}' | jq '.curve[:5]'
# → [
# {"time_sec": 0.0, "rms_db": -18.4},
# {"time_sec": 0.012, "rms_db": -17.9},
# ...
# ]
# Coarser envelope (2048-sample hop)
curl -X POST http://localhost:8000/v1/audio/loudness/curve \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","hop_length":2048}' | jq '{duration, sample_rate, points}'
```
Response fields: `curve` (array of `{time_sec, rms_db}`), `duration` (seconds), `sample_rate`, `points` (total curve length). Optional param: `hop_length` (default 512).
### Loudness normalization
```bash
# Normalize to -14 LUFS (streaming platform standard)
curl -X POST http://localhost:8000/v1/audio/normalize \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","target_lufs":-14,"output_path":"out/normalized.wav"}'
curl -o normalized.wav http://localhost:8000/v1/files/out/normalized.wav
# Write to a different staging path
curl -X POST http://localhost:8000/v1/audio/normalize \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","target_lufs":-23,"output_path":"mastered/norm.wav"}'
```
`target_lufs` is required. The response JSON carries `loudness_lufs` with the measured pre-normalization level alongside `path` / `url` / `size`.
### HPSS (harmonic/percussive split)
Median-filter harmonic/percussive source separation via librosa. Harmonic = tonal content (pitched instruments, pads); percussive = transients (drums, percussion). No ML — pure DSP, fast, no GPU needed.
```bash
# Get both stems in a ZIP
curl -X POST http://localhost:8000/v1/audio/separate/hpss \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","output_path":"out/stems.zip"}'
curl -o stems.zip http://localhost:8000/v1/files/out/stems.zip
# → stems.zip contains harmonic.wav + percussive.wav
# Wider margin = harder separation (more aggressive)
curl -X POST http://localhost:8000/v1/audio/separate/hpss \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","margin":3.0,"output_path":"out/stems.zip"}'
# Output to a different staging path
curl -X POST http://localhost:8000/v1/audio/separate/hpss \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","output_path":"hpss/stems.zip"}'
```
Params: `margin` (default 1.0 — ≥1.0, higher = more aggressive), `kernel_size` (default 31 — odd int, median filter width), `output_format` (default `wav`).
### Spectral noise reduction
Noise reduction with two engine options under the same endpoint — pick DSP for no-GPU fast cleanup or ML for higher-quality removal.
```bash
# DSP (noisereduce) — no GPU, pure spectral subtraction + Wiener filtering
curl -X POST http://localhost:8000/v1/audio/noise-reduce/noise-reduce \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","output_path":"out/clean.wav"}'
# Stationary mode — constant hum, hiss, fan noise
curl -X POST http://localhost:8000/v1/audio/noise-reduce/noise-reduce \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","stationary":true,"output_path":"out/clean.wav"}'
# Partial reduction — subtle noise floor cleanup
curl -X POST http://localhost:8000/v1/audio/noise-reduce/noise-reduce \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","prop_decrease":0.5,"output_path":"out/clean.wav"}'
# ML (UVR MelBand Roformer, SDR 28) — higher quality, GPU-accelerated
curl -X POST http://localhost:8000/v1/audio/noise-reduce/uvr-denoise \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","output_path":"out/clean.wav"}'
```
DSP params (only apply to `noise-reduce` engine): `stationary` (bool, default `false`), `prop_decrease` (0–1, default 1.0). Both engines accept `output_format`, `output_path`, `output_url`.
### Time-stretch and pitch-shift
Independent tempo factor and semitone offset via librosa phase vocoder. Slow a track down to learn it; shift a vocal up 3 semitones for a different key; transpose a MIDI melody to a different register first, then render.
```bash
# Slow down to 80% speed, no pitch change
curl -X POST http://localhost:8000/v1/audio/stretch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","tempo_factor":0.8,"output_path":"out/slow.wav"}'
# Shift up 3 semitones, no tempo change
curl -X POST http://localhost:8000/v1/audio/stretch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal.wav","pitch_semitones":3,"output_path":"out/pitched.wav"}'
# Both — pitch-corrected time stretch (traditional chipmunk effect)
curl -X POST http://localhost:8000/v1/audio/stretch \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","tempo_factor":0.5,"pitch_semitones":6,"output_format":"mp3","output_path":"out/stretched.mp3"}'
```
Params: `tempo_factor` (default 1.0 — 0.5 = half speed), `pitch_semitones` (default 0.0 — ±semitones), `output_format`, `output_path`.
### Pitch correct
Auto-tune audio toward the nearest chromatic semitone using librosa's phase vocoder. Full `strength=1.0` snaps hard to pitch; lower values blend the corrected and original signal.
```bash
# Hard auto-tune — snap every note to the nearest semitone
curl -X POST http://localhost:8000/v1/audio/pitch-correct \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal.wav","output_path":"out/tuned.wav"}'
# Subtle correction — 50% blend
curl -X POST http://localhost:8000/v1/audio/pitch-correct \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal.wav","strength":0.5,"output_format":"mp3","output_path":"out/tuned.mp3"}'
# Async for long files, staged output
curl -X POST http://localhost:8000/v1/audio/pitch-correct \
-H 'Content-Type: application/json' \
-d '{"file_path":"sessions/take1.wav","strength":1.0,"async_job":true,"output_path":"sessions/take1_tuned.wav"}'
```
Params: `strength` (0.0–1.0, default 1.0), `output_format`, `output_path`, `async_job`, `webhook_url`. Requires `librosa-analyze` engine.
### Repair
Declip clipped peaks and/or remove power-line hum. Declipping uses cubic interpolation to reconstruct flattened waveform tops and bottoms. Dehumming applies a notch filter at `hum_freq` (and harmonics).
```bash
# Declip only (default)
curl -X POST http://localhost:8000/v1/audio/repair \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/overdriven.wav","output_path":"out/repaired.wav"}'
# Remove 60 Hz hum (North American power grid)
curl -X POST http://localhost:8000/v1/audio/repair \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","declip":false,"dehum":true,"hum_freq":60.0,"output_path":"out/clean.wav"}'
# Both — declip a 50 Hz humming mic recording
curl -X POST http://localhost:8000/v1/audio/repair \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/problem_track.wav","declip":true,"dehum":true,"hum_freq":50.0,"output_format":"flac","output_path":"out/repaired.flac"}'
```
Params: `declip` (bool, default `true`), `dehum` (bool, default `false`), `hum_freq` (Hz, default 50.0), `output_format`, `output_path`, `async_job`, `webhook_url`.
### Audio tagging
Top-K AudioSet class label classification via Audio Spectrogram Transformer (MIT/ast-finetuned-audioset-10-10-0.4593). Identifies what's in a recording — music, speech, specific instruments, environmental sounds, etc.
```bash
curl -X POST http://localhost:8000/v1/audio/tag \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav"}'
# → {
# "tags": [
# {"label": "Music", "score": 0.94},
# {"label": "Drum", "score": 0.87},
# {"label": "Guitar", "score": 0.71},
# ...
# ],
# "duration": 5.2
# }
# Get top 20 results instead of the default 10
curl -X POST http://localhost:8000/v1/audio/tag \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/soundscape.wav","top_k":20}'
```
Requires the HF model cache. First run downloads the weights to `/data/hf/`. Optional: `top_k` (default 10).
> The image defaults to `HF_HUB_OFFLINE=0` so first call lazy-downloads the weights into `/data/hf/`. For locked-down deployments (no egress), prefetch the model with `huggingface-cli download <model>` into a mounted `/data/hf` volume, then start the container with `-e HF_HUB_OFFLINE=1`.
### Audio embeddings
512-dimensional L2-normalized audio embeddings via LAION CLAP (laion/larger_clap_music_and_speech). Useful for semantic audio search, similarity scoring, and clustering.
```bash
# Get the embedding vector
curl -X POST http://localhost:8000/v1/audio/embed \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# → {"embedding": [0.032, -0.11, ...], "dim": 512, "norm": 1.0}
# Semantic similarity — how well does the audio match a text description?
curl -X POST http://localhost:8000/v1/audio/embed \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","query_text":"energetic rock guitar riff"}'
# → {"embedding": [...], "dim": 512, "norm": 1.0,
# "query_text": "energetic rock guitar riff", "similarity": 0.73}
```
`similarity` is cosine similarity in [-1, 1]. Requires HF model cache — same first-run download caveat as audio tagging.
### Zero-shot classification
Given audio and a list of free-form text labels, return cosine similarity scores for each using the existing CLAP model. No extra model download — uses the same `clap-embed` engine. Works for genres, moods, instruments, sonic descriptors — anything CLAP understands.
```bash
# Genre detection
curl -X POST http://localhost:8000/v1/audio/classify \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","labels":["jazz","hip-hop","classical","electronic","rock"]}'
# → {"results": [
# {"label": "hip-hop", "score": 0.42},
# {"label": "electronic", "score": 0.38},
# ...
# ]}
# Mood / energy
curl -X POST http://localhost:8000/v1/audio/classify \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","labels":["energetic","calm","melancholic","aggressive","uplifting"]}'
# Speaker gender
curl -X POST http://localhost:8000/v1/audio/classify \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/interview.wav","labels":["male voice","female voice","child voice","multiple speakers"]}'
```
Results are sorted by descending score. Scores are cosine similarities in [-1, 1] — higher = more similar. Requires `clap-embed` model cache.
### Audio info
Probe any audio file for metadata without loading it into memory for processing. Uses ffprobe — handles any format.
```bash
curl -X POST http://localhost:8000/v1/audio/info \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav"}'
# → {
# "size_bytes": 52428800,
# "duration_sec": 297.241,
# "sample_rate": 44100,
# "channels": 2,
# "codec": "pcm_s16le",
# "sample_fmt": "s16",
# "format": "wav",
# "bit_depth": 16,
# "bit_rate": 1411200
# }
# Works on any staged file
curl -X POST http://localhost:8000/v1/audio/info \
-H 'Content-Type: application/json' \
-d '{"file_path":"recordings/interview.mp3"}'
# → {"codec": "mp3", "bit_rate": 192000, ...}
```
### Trim
Cut a precise time range out of any audio file. Common use: extract a chorus, clip a sample, chop a stem at bar boundaries.
```bash
# Extract seconds 30–90 from a track
curl -X POST http://localhost:8000/v1/audio/trim \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","start_sec":30.0,"end_sec":90.0,"output_path":"out/chorus.wav"}'
# Clip a specific beat range, export as mp3
curl -X POST http://localhost:8000/v1/audio/trim \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/stem.wav","start_sec":0.0,"end_sec":8.0,"output_format":"mp3","output_path":"out/loop.mp3"}'
# From staged file, write to a different staging path
curl -X POST http://localhost:8000/v1/audio/trim \
-H 'Content-Type: application/json' \
-d '{"file_path":"sessions/full.wav","start_sec":120.5,"end_sec":180.0,"output_path":"clips/verse.wav"}'
```
`start_sec` defaults to 0. `end_sec` is required and must be greater than `start_sec`. Supports all standard `output_format` values.
### Mix
Combine multiple staged or URL-accessible tracks into one. Per-track `gain_db` lets you balance levels before mixing. Useful for bouncing separated stems back together at custom levels, layering synth parts, or combining click-track + music.
```bash
# Mix drums and bass at equal levels
curl -X POST http://localhost:8000/v1/audio/mix \
-H 'Content-Type: application/json' \
-d '{"tracks":[{"file_path":"stems/drums.wav"},{"file_path":"stems/bass.wav"}],"output_path":"out/rhythm.wav"}'
# Stems at custom levels (drums -3 dB, bass 0 dB, vocals +2 dB)
curl -X POST http://localhost:8000/v1/audio/mix \
-H 'Content-Type: application/json' \
-d '{"tracks":[
{"file_path":"stems/drums.wav","gain_db":-3},
{"file_path":"stems/bass.wav","gain_db":0},
{"file_path":"stems/vocals.wav","gain_db":2}
],"output_format":"wav","output_path":"out/custom_mix.wav"}'
# Write to a different staging path
curl -X POST http://localhost:8000/v1/audio/mix \
-H 'Content-Type: application/json' \
-d '{"tracks":[{"file_path":"stems/harmonic.wav"},{"file_path":"stems/percussive.wav","gain_db":-6}],"output_path":"mixed/recombined.wav"}'
```
`tracks` is a required JSON array. Each entry needs `file_path` or `file_url` and an optional `gain_db` (default 0.0). Requires at least 2 tracks. Shorter tracks are padded with silence to match the longest.
### Concat
Stitch N audio files together in order. Handles different sample rates and channel counts automatically (ffmpeg resamples on the fly).
```bash
curl -X POST http://localhost:8000/v1/audio/concat \
-H 'Content-Type: application/json' \
-d '{"files":[{"file_path":"intro.wav"},{"file_path":"verse.wav"},{"file_path":"outro.wav"}],"output_path":"out/full_track.wav"}'
# output_format change + different staging path
curl -X POST http://localhost:8000/v1/audio/concat \
-H 'Content-Type: application/json' \
-d '{"files":[{"file_path":"a.wav"},{"file_path":"b.wav"}],"output_format":"mp3","output_path":"concat/result.mp3"}'
```
`files` is a required JSON array of `{file_path?, file_url?}` objects. Requires at least 2 entries.
### Speed
Change playback speed without pitch shifting — useful for auditioning at half/double speed, or creating slow-motion effects. Uses ffmpeg `atempo` filter chained for extreme multipliers.
```bash
# Half speed
curl -X POST http://localhost:8000/v1/audio/speed \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","speed":0.5,"output_path":"out/slow.wav"}'
# Double speed
curl -X POST http://localhost:8000/v1/audio/speed \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","speed":2.0,"output_path":"out/fast.wav"}'
# 4× speed (chains two atempo=2.0 filters internally)
curl -X POST http://localhost:8000/v1/audio/speed \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","speed":4.0,"output_format":"mp3","output_path":"out/fast.mp3"}'
```
`speed` is required. Range: 0.1–10.0. Note: this changes duration but not pitch. For pitch-preserving tempo changes use `/v1/audio/stretch`.
### Convert
Re-encode audio to a different format, sample rate, or channel count in a single call.
```bash
# WAV → 16 kHz mono FLAC (for speech models)
curl -X POST http://localhost:8000/v1/audio/convert \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","output_format":"flac","sample_rate":16000,"channels":1,"output_path":"out/prepared.flac"}'
# Stereo → mono WAV
curl -X POST http://localhost:8000/v1/audio/convert \
-H 'Content-Type: application/json' \
-d '{"file_path":"stereo.wav","channels":1,"output_path":"out/mono.wav"}'
# Any format → Opus at 48 kHz
curl -X POST http://localhost:8000/v1/audio/convert \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/audio.mp3","output_format":"opus","sample_rate":48000,"output_path":"out/out.opus"}'
```
`output_format` defaults to `wav`. `sample_rate` and `channels` are optional; if omitted, the source values are preserved.
### Similar
Compute cosine similarity between two audio files using CLAP embeddings. Returns a score in [-1, 1] — 1 = identical sound, 0 = unrelated, negative = acoustically opposite. Useful for duplicate detection, cover matching, or finding the closest sample in a library.
```bash
curl -X POST http://localhost:8000/v1/audio/similar \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/original.wav","reference_file_path":"uploads/remix.wav"}'
# → {"similarity": 0.847, "dim": 512}
# Different staged paths
curl -X POST http://localhost:8000/v1/audio/similar \
-H 'Content-Type: application/json' \
-d '{"file_path":"stems/vocals.wav","reference_file_path":"stems/vocals_ref.wav"}'
```
Primary file: `file_path` / `file_url`. Reference file: `reference_file_path` / `reference_file_url`. Requires `clap-embed` engine.
### MIDI quantize
Snap all note timings in a MIDI file to the nearest rhythmic grid. Cleaner dedicated endpoint than `/v1/midi/transform`'s `quantize_grid_beats` param.
```bash
# Quantize to 16th notes (0.25 beats)
curl -X POST http://localhost:8000/v1/midi/quantize \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/sloppy.mid","grid_beats":0.25,"output_path":"out/tight.mid"}'
# 8th note grid
curl -X POST http://localhost:8000/v1/midi/quantize \
-H 'Content-Type: application/json' \
-d '{"file_path":"recorded.mid","grid_beats":0.5,"output_path":"midi/quantized.mid"}'
```
`grid_beats`: grid size in beats — `0.25` = 16th note, `0.5` = 8th, `1.0` = quarter note. Default: `0.25`.
### Fade
Apply fade-in, fade-out, or both. 13 curve shapes: `tri`, `qsin`, `esin`, `hsin`, `log`, `ipar`, `qua`, `cub`, `squ`, `cbr`, `par`, `exp`, `lin`.
```bash
# 2s fade-in
curl -X POST http://localhost:8000/v1/audio/fade \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","fade_in":2.0,"output_path":"out/faded.wav"}'
# 3s fade-out with exponential curve
curl -X POST http://localhost:8000/v1/audio/fade \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","fade_out":3.0,"curve":"exp","output_path":"out/faded.wav"}'
# Both — 1s in, 2s out
curl -X POST http://localhost:8000/v1/audio/fade \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","fade_in":1.0,"fade_out":2.0,"output_path":"out/faded.wav"}'
```
At least one of `fade_in` / `fade_out` must be > 0.
### Reverse
Flip audio backwards via ffmpeg `areverse`.
```bash
curl -X POST http://localhost:8000/v1/audio/reverse \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/sample.wav","output_path":"out/reversed.wav"}'
curl -X POST http://localhost:8000/v1/audio/reverse \
-H 'Content-Type: application/json' \
-d '{"file_path":"stems/vocals.wav","output_format":"mp3","output_path":"out/reversed.mp3"}'
```
### Loop
Repeat audio N times. Uses ffmpeg `aloop` filter — no re-encoding overhead per iteration.
```bash
# Play 4 times total
curl -X POST http://localhost:8000/v1/audio/loop \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/beat.wav","count":4,"output_path":"out/looped.wav"}'
# 8-bar loop → 32 bars
curl -X POST http://localhost:8000/v1/audio/loop \
-H 'Content-Type: application/json' \
-d '{"file_path":"stems/drums.wav","count":4,"output_path":"loops/drums32.wav"}'
```
`count` must be ≥ 2 (total plays, not extra loops).
### BPM match
Detect the source BPM via librosa, then time-stretch to the target — no manual math.
```bash
# Stretch anything to 128 BPM
curl -X POST http://localhost:8000/v1/audio/bpm-match \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/loop.wav","target_bpm":128,"output_path":"out/matched.wav"}'
# Match tempo and also shift pitch
curl -X POST http://localhost:8000/v1/audio/bpm-match \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/loop.wav","target_bpm":140,"pitch_semitones":2,"output_path":"out/matched.wav"}'
```
Response JSON includes `source_bpm`, `target_bpm`, and `tempo_factor` alongside the staged `path` / `url`. Requires both `librosa-analyze` and `stretch` engines.
### Stereo width
Widen or collapse the stereo image via M/S processing. `width=0.0` → mono, `1.0` → original, `>1.0` → wider. Works on mono input too (upmixes first).
```bash
# Widen to 1.5×
curl -X POST http://localhost:8000/v1/audio/stereo-width \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/mix.wav","width":1.5,"output_path":"out/wide.wav"}'
# Collapse to mono
curl -X POST http://localhost:8000/v1/audio/stereo-width \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/mix.wav","width":0.0,"output_path":"out/mono.wav"}'
# Subtle narrowing for mix bus
curl -X POST http://localhost:8000/v1/audio/stereo-width \
-H 'Content-Type: application/json' \
-d '{"file_path":"master/mix.wav","width":0.8,"output_path":"master/narrow.wav"}'
```
Range: `[0.0, 3.0]`.
### Split
Split a file into segments. Two modes: `equal` (N equal time parts) or `silence` (split on quiet gaps). Returns a ZIP of numbered files.
```bash
# Split into 4 equal parts
curl -X POST http://localhost:8000/v1/audio/split \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/track.wav","mode":"equal","count":4,"output_path":"out/segments.zip"}'
# Split a DJ mix on silence
curl -X POST http://localhost:8000/v1/audio/split \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/djmix.wav","mode":"silence","threshold_db":-40,"min_duration_sec":1.0,"output_path":"out/tracks.zip"}'
# Split to mp3
curl -X POST http://localhost:8000/v1/audio/split \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/album.flac","mode":"equal","count":10,"output_format":"mp3","output_path":"out/parts.zip"}'
```
`mode=equal` requires `count >= 2`. `mode=silence` uses `threshold_db` (default -30) and `min_duration_sec` (default 0.5); requires the `silence-detect` engine.
### Pan
Position audio in the stereo field. Works on mono and stereo input.
```bash
# Hard left
curl -X POST http://localhost:8000/v1/audio/pan \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal.wav","position":-1.0,"output_path":"out/left.wav"}'
# Slight right (e.g. guitar in mix)
curl -X POST http://localhost:8000/v1/audio/pan \
-H 'Content-Type: application/json' \
-d '{"file_path":"stems/guitar.wav","position":0.4,"output_path":"out/guitar_panned.wav"}'
# Center (no-op but valid)
curl -X POST http://localhost:8000/v1/audio/pan \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/mono.wav","position":0.0,"output_path":"out/stereo.wav"}'
```
`position`: -1.0 = hard left, 0.0 = center, 1.0 = hard right.
### EQ
Parametric EQ via ffmpeg `equalizer` filter. Pass any number of bands — each with a center frequency, gain, and optional bandwidth.
```bash
# Low-cut + presence boost
curl -X POST http://localhost:8000/v1/audio/eq \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/vocal.wav","bands":[{"freq":100,"gain_db":-6,"width_hz":80},{"freq":3000,"gain_db":3,"width_hz":500}],"output_path":"out/eq.wav"}'
# Single band: cut 60 Hz hum
curl -X POST http://localhost:8000/v1/audio/eq \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/recording.wav","bands":[{"freq":60,"gain_db":-20,"width_hz":30}],"output_path":"out/clean.wav"}'
```
Each band: `freq` (Hz, required), `gain_db` (dB, required, range ±30), `width_hz` (optional, default 100).
### Key match
Detect the source key via CLAP chord analysis, then pitch-shift to a target key — one call instead of two.
```bash
# Shift everything to C major
curl -X POST http://localhost:8000/v1/audio/key-match \
-H 'Content-Type: application/json' \
-d '{"file_path":"uploads/loop.wav","target_key":"C","output_path":"out/matched.wav"}'
# Match to F# (response includes source_key + semitones shifted)
curl -X POST http://localhost:8000/v1/audio/key-match \
-H 'Content-Type: application/json' \
-d '{"file_path":"stems/melody.wav","target_key":"F#","output_path":"matched/melody_fsharp.wav"}'
```
`target_key`: root note, e.g. `C`, `F#`, `Bb`, `D#`. Mode suffix (`major`/`minor`/`m`) is ignored — only the root matters for pitch. Requires `chord-detect` and `stretch` engines.
### Sidechain duck
Duck a primar