{
  "markdown": "# 🎵 Audio Sonic MCP\n\n[![Tests](https://github.com/ripunjay-kashyap/audio-sonic-mcp/actions/workflows/test.yml/badge.svg)](https://github.com/ripunjay-kashyap/audio-sonic-mcp/actions/workflows/test.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/)\n[![MCP](https://img.shields.io/badge/MCP-server-8A2BE2.svg)](https://modelcontextprotocol.io/)\n\n**Turn any song into a structured \"sonic signature\" — extracting tempo, musical key, a 512-dimension CLAP vibe embedding, human-readable vibe tags, and a production profile — from a single local call.**\n\nAudio Sonic MCP runs entirely on your local machine (requiring no API keys, external servers, or cloud dependencies) and exposes two premium access points to the same underlying high-fidelity audio analysis engine:\n\n| | Tailored For | Core Interface & Mechanics |\n|---|---|---|\n| 🤖 **MCP Server** | LLMs, AI agents, & IDEs (Claude, Cursor, Windsurf, Cline) | Asynchronous, fire-and-forget analysis of YouTube URLs. Avoids blocking client LLMs during heavy audio processing. |\n| 🎚️ **Local CLI** | Musicians, sound producers, & audio engineers | Deep command-line tool targeting local files for full-song multi-window analysis and high-fidelity output. |\n\n---\n\n## 🎹 Quick Taste: What You Get\n\n### 1. Musician-Friendly CLI Summary (`--summary` mode)\n```text\n🎵 SONIC SIGNATURE — my_demo.mp3  (3:24)\n\n  TEMPO    153.8 BPM  (steady)\n  KEY      G Major  ·  shifts to G Phrygian @0:30   (confidence 74%)\n  VIBE     aggressive · dark · driving · hip-hop · gritty\n\n  PRODUCTION\n     Vocals     forward\n     Punch      0.62  (moderate)\n     Stereo     wide\n     Low end    ~55 Hz dominant\n\n  Overall confidence: 88%   ·   analyzed in 0:28 (GPU-accelerated)\n```\n\n### 2. Comprehensive JSON (Returned by MCP and CLI by default)\n```json\n{\n  \"header\": {\n    \"job_id\": \"sig_a3f9b2c1\",\n    \"status\": \"success\",\n    \"confidence_score\": 0.88,\n    \"source_metadata\": {\n      \"title\": \"Acoustic Vibe Demo\",\n      \"duration_sec\": 204,\n      \"source_type\": \"file\"\n    }\n  },\n  \"sonic_signature\": {\n    \"bpm\": 153.8,\n    \"bpm_engine\": \"madmom\",\n    \"bpm_variable\": false,\n    \"key\": \"G Major\",\n    \"key_variable\": true,\n    \"key_map\": [\n      { \"start_sec\": 0.0,  \"end_sec\": 30.0, \"key\": \"G Major\" },\n      { \"start_sec\": 30.0, \"end_sec\": 90.0, \"key\": \"G Phrygian\" }\n    ],\n    \"mode_confidence\": 0.74,\n    \"vibe_vector\": [0.012, -0.034, \"... 512 float dimensions ...\"],\n    \"vibe_tags\": [\"aggressive\", \"dark\", \"driving\", \"hip-hop\", \"gritty\"],\n    \"production_profile\": {\n      \"vocal_presence\": \"forward\",\n      \"transient_punch\": 0.62,\n      \"stereo_width\": \"wide\",\n      \"dominant_freq_peaks_hz\": {\n        \"harmonic\": [55.0, 110.2],\n        \"percussive\": [125.0, 250.1]\n      }\n    }\n  },\n  \"telemetry\": {\n    \"inference_time_sec\": 28.0\n  }\n}\n```\n\n---\n\n## ⚡ Key Features\n\n* 🥁 **Tempo & Beat Tracking** — Full BPM computation with variable-tempo drift detection and transient windowing.\n* 🎹 **Key & Harmonic Mapping** — Computes structural musical key + mode, generating a detailed `key_map` tracking section-by-section modulations.\n* 🌈 **Vibe & Style Embeddings** — Compiles a 512-dimensional CLAP embedding and human-readable style tags (covering energy, texture, mood, and genre) using zero-shot music vocab classification.\n* 🎚️ **Production Analytics** — Measures vocal spatial presence, transient punch coefficients, stereo width, and dominant frequency peaks.\n* 🤖 **MCP-Native System** — Fully exposes 4 standardized Model Context Protocol tools for instant integration into AI tools.\n* 🪶 **Robust Graceful Degradation** — Automatically utilizes a CUDA GPU if present and falls back to CPU; gracefully degrades to HPSS and standard librosa feature arrays if heavy deep learning packages (`[clap]`) are omitted.\n* 🔒 **100% Offline & Private** — All conversion, separation, and inference occur locally.\n\n---\n\n## 📦 Installation & Setup\n\n### System Prerequisites\nEnsure you have **Python 3.10+** and **FFmpeg** installed and accessible on your system `PATH`.\n\n#### Installing FFmpeg:\n* **macOS**: `brew install ffmpeg`\n* **Linux (Debian/Ubuntu)**: `sudo apt update && sudo apt install -y ffmpeg`\n* **Windows**: Run `winget install Gyan.FFmpeg` via PowerShell (Administrator), or download manually from [ffmpeg.org](https://ffmpeg.org/download.html) and add the `bin` directory to your system environment variables.\n\n---\n\n### Step-by-Step Installation\n\n1. **Clone the Repository**\n   ```bash\n   git clone https://github.com/ripunjay-kashyap/audio-sonic-mcp.git\n   cd audio-sonic-mcp\n   ```\n\n2. **Initialize Virtual Environment**\n   ```bash\n   python -m venv .venv\n   # Activate on macOS/Linux:\n   source .venv/bin/activate\n   # Activate on Windows (PowerShell):\n   .venv\\Scripts\\activate\n   ```\n\n3. **Install Dependencies**\n   Choose between the lightweight core engine or the full high-fidelity ML suite:\n   \n   * **Option A: Full High-Fidelity ML Suite (Recommended)**\n     Includes demixing stems (Demucs) and zero-shot vibe vectors (CLAP). Requires ~4 GB disk space.\n     ```bash\n     pip install -e \".[clap]\"\n     ```\n   * **Option B: Core Lightweight Pipeline**\n     Uses standard digital signal processing (HPSS/librosa). Rapid install and minimal footprint.\n     ```bash\n     pip install -e .\n     ```\n\n> [!NOTE]\n> The optional `[clap]` stack installs `torch`, `torchaudio`, `transformers`, and `demucs`. Without these, the server automatically switches to light fallbacks (HPSS instead of Demucs, standard feature matrices instead of CLAP vectors, and leaves out `vibe_tags`).\n\n---\n\n## 🤖 MCP Client Configuration Guide\n\nAudio Sonic MCP registers itself as a standard package script. This enables you to run it using the global executable name (`audio-sonic-mcp`) directly from your virtual environment's bin folder, or run the script file manually.\n\n### 1. Claude Desktop Setup\nOpen your Claude configuration file:\n* **Windows**: `%APPDATA%\\Claude\\claude_desktop_config.json`\n* **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`\n* **Linux**: `~/.config/Claude/claude_desktop_config.json`\n\nAdd the server to your `mcpServers` object:\n\n```json\n{\n  \"mcpServers\": {\n    \"audio-sonic-mcp\": {\n      \"command\": \"C:\\\\path\\\\to\\\\audio-sonic-mcp\\\\.venv\\\\Scripts\\\\audio-sonic-mcp.exe\",\n      \"args\": [],\n      \"env\": {\n        \"JOBS_ROOT\": \"C:\\\\path\\\\to\\\\audio-sonic-mcp\\\\jobs\"\n      }\n    }\n  }\n}\n```\n\n> [!IMPORTANT]\n> **Windows Users**: Always use **double backslashes** (`\\\\`) in JSON configuration paths. Point the executable directly to the `.exe` inside your `.venv\\Scripts\\` directory.\n\n---\n\n### 2. Cursor IDE Integration\nTo integrate Audio Sonic MCP into Cursor's AI pane:\n1. Navigate to **Settings** ➔ **Features** ➔ **MCP**.\n2. Click **+ Add New MCP Server**.\n3. Fill in the parameters:\n   * **Name**: `audio-sonic-mcp`\n   * **Type**: `command`\n   * **Command**: `/path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp` (use `.exe` extension on Windows)\n\n---\n\n### 3. Windsurf Integration\nOpen your Windsurf MCP configurations file (typically found at `~/.codeium/windsurf/mcp_config.json`) and append the configuration:\n\n```json\n{\n  \"mcpServers\": {\n    \"audio-sonic-mcp\": {\n      \"command\": \"/path/to/audio-sonic-mcp/.venv/bin/python\",\n      \"args\": [\"/path/to/audio-sonic-mcp/server.py\"],\n      \"env\": {\n        \"JOBS_ROOT\": \"/path/to/audio-sonic-mcp/jobs\"\n      }\n    }\n  }\n}\n```\n\n---\n\n### 4. Cline (VS Code Extension) Setup\nOpen Cline's MCP setting file (usually located at `%APPDATA%\\Code\\User\\globalStorage\\saoudrizwan.claude-dev\\settings\\cline_mcp_settings.json` or equivalent platform storage) and add:\n\n```json\n{\n  \"mcpServers\": {\n    \"audio-sonic-mcp\": {\n      \"command\": \"/path/to/audio-sonic-mcp/.venv/bin/audio-sonic-mcp\",\n      \"args\": [],\n      \"env\": {\n        \"JOBS_ROOT\": \"/path/to/audio-sonic-mcp/jobs\"\n      }\n    }\n  }\n}\n```\n\n---\n\n## 🤖 Interaction Flow for AI Agents & LLMs\n\nLLMs automatically learn how to use this server by reading its exposed tool definitions. Because audio stem separation and CLAP embeddings are computationally demanding, Audio Sonic MCP uses an **Asynchronous Fire-and-Forget Job Pattern**.\n\n### Automated LLM Workflow\n```\n  [User Prompts LLM]\n          │\n          ▼\n1. Submit URL ──────────────► [Tool: get_sonic_signature]\n                                      │ (Returns Job ID instantly)\n                                      ▼\n2. Notify User ◄───────────── [LLM acknowledges job is queued]\n          │\n          ├───► 3. Wait 10-15s (Or proceed with other tasks)\n          │\n          ▼\n4. Check Progress ──────────► [Tool: get_job_status]\n                                      │ (Checks status: running/success/error)\n                                      ▼\n5. Present Signature ◄─────── [LLM formats rich output for user]\n```\n\n### Natural Prompts to Try\n* *\"Check the health of my audio-sonic-mcp server to make sure all ML components are ready.\"*\n* *\"Submit this YouTube track for sonic analysis: `https://www.youtube.com/watch?v=XXXXXX`.\"*\n* *\"Check the progress of my sonic signature job `sig_a1b2c3d4` and summarize the BPM, production width, and vibe once complete.\"*\n\n---\n\n## 🎚️ CLI Usage (Local Files)\n\nFor musicians, engineers, and producers working directly in the terminal, you can analyze a full-length local file directly without running any background servers:\n\n```bash\n# Get a visual, musician-friendly sonic signature digest (recommended)\npython analyze_file.py \"my_demo.wav\" --summary\n\n# Print full raw JSON directly to the stdout stream\npython analyze_file.py \"my_demo.wav\"\n\n# Dump JSON payload to a file while keeping the stdout clean\npython analyze_file.py \"my_demo.wav\" > signature.json\n```\n\n### CLI Command Options Reference\n\n| Option | Shorthand | Description |\n|---|---|---|\n| `path` | *None* | Absolute or relative path to the local audio file (Required). |\n| `--summary` | `-s` | Print a clean, formatted terminal summary instead of standard JSON. |\n| `--no-vector` | *None* | Generate JSON signature but omit the heavy 512-dimension vibe float array. |\n| `--out FILE` | `-o` | Output the final JSON signature directly to the specified file. |\n| `--keep` | `-k` | Do not delete intermediate WAV files or separated stem files in `jobs/`. |\n| `--job-id ID` | `-j` | Explicitly define the internal identifier (useful for batch scripts). |\n\n**Supported File Formats**: `wav`, `mp3`, `flac`, `ogg`, `m4a`, `aac`.\n\n---\n\n## 🔧 Environment Variables Reference\n\nConfigure environment options by declaring these variables in your active terminal session, container environment, or the `env` block of your MCP configuration file:\n\n| Variable | Default Value | Description / Practical Use |\n|---|---|---|\n| `JOBS_ROOT` | `./jobs` | Workspace directory where audio files, temporary converted WAVs, and stems are processed. |\n| `KEEP_JOB_FILES` | *Unset* | Set to `1` or `true` to keep separated stem WAVs on disk (adds ~75MB per job, useful for troubleshooting). |\n| `FILE_MAX_DURATION_SEC`| `600` | Safety ceiling for local file processing duration (YouTube downloads are capped at 60 minutes). |\n| `FFMPEG_BIN` | *Unset* | Path to folder containing the `ffmpeg` binary if it is not present in your system `PATH`. |\n| `YTDLP_PROXY` | *Unset* | HTTP/SOCKS proxy string passed directly to `yt-dlp` to bypass rate limits or network blocks. |\n| `TRANSPORT_MODE` | `stdio` | Transport the server listens on: `stdio` (default, for local MCP clients), `sse` (remote MCP over HTTP), or `hybrid` (MCP SSE **and** the REST API from `app_cloud.py`). `sse`/`hybrid` need `pip install \".[cloud]\"`. |\n| `PORT` | `8000` | Listening port when `TRANSPORT_MODE` is `sse` or `hybrid`. Ignored for `stdio`. |\n\n---\n\n## 🐳 Docker / Podman Execution\n\nIf you prefer to avoid setting up local Python libraries, running via containers encapsulates FFmpeg, yt-dlp, and the core Python dependencies (CPU-based pipeline):\n\n```bash\n# Build the container image\ndocker build -t audio-sonic-mcp .\n\n# Run the MCP server over stdio, mounting local folders for job persistence\ndocker run -i --rm \\\n  -v \"$(pwd)/jobs:/app/jobs\" \\\n  -v \"$(pwd)/models:/app/models\" \\\n  audio-sonic-mcp\n```\n\nTo connect Claude Desktop to your Docker container, configure `claude_desktop_config.json`:\n```json\n{\n  \"mcpServers\": {\n    \"audio-sonic-mcp-docker\": {\n      \"command\": \"docker\",\n      \"args\": [\n        \"run\", \"-i\", \"--rm\",\n        \"-v\", \"/absolute/path/to/jobs:/app/jobs\",\n        \"-v\", \"/absolute/path/to/models:/app/models\",\n        \"audio-sonic-mcp\"\n      ]\n    }\n  }\n}\n```\n\n---\n\n## ⚙️ How it Works under the Hood\n\nAudio Sonic MCP pipelines are constructed modularly, using transactional checkpoints to ensure reliability. \n\n```text\n  LLM Agent / Claude Desktop                 Musician (Terminal)\n            │                                          │\n            │  MCP (stdio JSON-RPC)                    │  analyze_file.py\n            ▼                                          ▼\n┌──────────────────────────────────────────────────────────────────────────┐\n│  Modular 6-Stage Analysis Pipeline                                       │\n│                                                                          │\n│  Stage 1: Ingestion   │ Pre-checks format, scans duration metadata       │\n│  Stage 2: Download    │ Fetches audio tracks via yt-dlp (URLs only)      │\n│  Stage 3: Conversion  │ normalizes sample formats to 44.1kHz WAV (FFmpeg)│\n│  Stage 4: Separation  │ Splits stems: Vocals, Drums, Bass, Other (Demucs)│\n│  Stage 5: Analysis    │ Computes BPM, modulations, key, punch (librosa)  │\n│  Stage 6: Embeddings  │ Generates 512-dim zero-shot music vibe tags (CLAP)│\n└─────────────────────────────────────┬────────────────────────────────────┘\n                                      ▼\n             Result Payload: (header · sonic_signature · telemetry)\n```\n\n1. **Stem Demixing**: Meta AI's **Demucs (`mdx_extra`)** separates the track into isolation stems (`vocals`, `drums`, `bass`, `other`). If missing, it gracefully drops back to **Harmonic-Percussive Source Separation (HPSS)**.\n2. **Analysis engine**: **librosa** extracts rhythmic and tonal structures, matching chord patterns and sub-bass movements against Krumhansl-Schmuckler and Phrygian template engines.\n3. **Semantic Vibe Tagging**: **LAION CLAP** (`laion/larger_clap_music_and_speech`) runs zero-shot inference against high-coverage aesthetic descriptors (moods, textures, genres), choosing top candidates across stylistic poles.\n\n---\n\n## 🩺 Resiliency & Troubleshooting\n\n### 1. One-Time Setup Download Delays\nUpon the **very first analysis job** utilizing the full ML pipeline, `demucs` and `transformers` will download their pre-trained model weights (approximately **400 MB** for Demucs, and **200 MB** for CLAP). \n* The server redirects download progress indicators to `stderr` so they **do not corrupt** the JSON-RPC standard stream.\n* During this download, `get_job_status` will remain in `running`. Allow 1–3 minutes depending on your network speed. Subsequent startups take under **10 seconds**.\n\n### 2. FastMCP Concurrency Controls\nModel inference on multi-staged architectures is highly CPU/VRAM intensive. To protect consumer hardware and virtual environments from crashing (OutOfMemory exceptions), Audio Sonic MCP enforces a strict global serialization lock (`CONCURRENCY_LOCK`). \n* If you submit multiple URLs simultaneously, they will be processed **sequentially**. \n* Polling `get_job_status` for subsequent jobs will report `queued` or `running` while they wait in the pipeline queue.\n\n### 3. Windows Librosa Deadlock Fix\nFastMCP thread dispatching under Windows can cause Numba compilation deadlocks inside background worker threads. To prevent this, Audio Sonic MCP incorporates a **Pre-warming Routine** (`_prewarm_librosa()` and `_prewarm_demucs()`) on launch. It forces JIT compile of resampling, HPSS, and mono-mixing functions in the main thread before starting the RPC listener.\n\n### 4. BPM Accuracy and the `bpm_engine` Field\nTempo is estimated by **madmom**'s RNN beat tracker. madmom is an *optional* dependency: it is unmaintained (latest release 0.16.1, classifiers stop at Python 3.7) and requires a Cython build, so it cannot be installed reliably everywhere and is **not** part of the default install.\n\nWhen madmom is unavailable the pipeline falls back to librosa. That fallback is good on steady four-on-the-floor material but can lock onto a **2:3 or octave multiple** of the true tempo — on one of our regression fixtures it reports 99.4 BPM against a ground truth of 148.\n\nSo the tempo is never reported unqualified. **Every payload carries a `bpm_engine` field** naming the engine that actually produced the number:\n\n| `bpm_engine` | Meaning |\n|---|---|\n| `madmom` | RNN beat tracker — full accuracy. |\n| `librosa-fallback` | madmom unavailable; treat BPM as approximate and expect occasional octave/triplet errors. |\n\n`check_health` reports madmom's status explicitly. To enable the accurate path:\n\n```bash\npip install \".[beats]\"\n```\n\nIf the build fails on a recent Python, use 3.10 for the analysis environment — madmom has no wheels for newer interpreters.\n\n### 5. Diagnosing with `check_health`\nIf the server reports as `degraded` or tools are missing, call the `check_health` tool or check CLI warnings. It queries:\n* Availability of `ffmpeg` on the execution path.\n* Installation status of Python packages (`librosa`, `soundfile`, `mcp`, etc.).\n* Presence of the optional `madmom` beat tracker, and which `bpm_engine` will be used as a result.\n* Access permissions to the `JOBS_ROOT` directory.\n\n---\n\n## 🛠️ Development & Testing\n\nRun unit tests inside your virtual environment to verify the mathematical pipelines using synthesized audio waveforms:\n\n```bash\n# Install development test framework\npip install -e \".[dev]\"\n\n# Execute full suite (requires no network or model downloads)\npytest\n\n# Test specifically CLI execution code paths\npytest tests/test_cli.py\n```\n\n---\n\n## 📄 License\n\nDistributed under the **MIT License**. See `LICENSE` for details.\n\n© 2026 Ripunjay Kashyap. All rights reserved.\n",
  "bytes": 18143,
  "sha": "4f00bd435445af7adabbf43547c4f524a80a41c2ba6aff606851b38f53b8fc63",
  "repo_slug": "ripunjay-kashyap/audio-sonic-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_ripunjay_kashyap_audio_sonic_m_4cabe161/readme"
}