{
  "markdown": "# Jarvis Voice Plugin\n\nA voice-driven interaction plugin for Claude Code. Local speech-to-text, text-to-speech, and speaker verification — fully offline, fully hands-free.\n\n## What It Does\n\nJarvis turns Claude Code into a voice assistant. You speak, it listens. It speaks back. No cloud APIs, no latency penalties — everything runs on-device.\n\n### Pipeline\n\n```\nMic (SoX rec) → 16kHz Downsample → VAD (Silero) → Denoiser (GTCRN)\n    → Speaker Verification (WeSpeaker) → STT (Whisper tiny.en)\n    → Transcription Queue → Claude Code\n\nClaude Code → TTS (Kokoro 24kHz) → Temp File → SoX play → Speakers\n```\n\n### Features\n\n- **Kokoro TTS** — High-quality neural text-to-speech at 24kHz with 11 voice options\n- **Whisper STT** — Local speech-to-text using Whisper tiny.en\n- **Speaker Verification** — WeSpeaker embeddings ensure only the enrolled user's voice is processed\n- **Speech Denoiser** — GTCRN model provides ~34dB noise reduction before processing\n- **Voice Activity Detection** — Silero VAD detects speech segments in real-time\n- **Speech Accumulation** — Consecutive speech segments are merged with 3-second silence gap detection, delivering complete sentences instead of fragments\n- **Pause/Resume** — Say \"Jarvis pause\" to mute listening, \"Jarvis resume\" to restart\n- **Wake Word** (optional) — Queue filtering mode that only passes messages starting with \"Jarvis\"\n- **Listening Indicator** — Short tone plays when the assistant is ready for input\n- **Concurrent Capture** — Mic stays active during TTS playback so user speech queues up\n- **Acoustic Self-Test** — Loopback test that speaks a phrase and verifies transcription through the full pipeline\n\n### Voice Enrollment\n\nOn first run, Jarvis guides you through 5 phrases to build a speaker profile. This profile is used for speaker verification — filtering out background voices and ambient noise.\n\n## Architecture\n\n```\nsrc/\n  main.ts                  — Entry point, MCP server + orchestrator\n  mcp/\n    entry.ts               — JSON-RPC stdio server with lifecycle handlers\n    tools.ts               — MCP tool definitions and handlers\n  pipeline/\n    orchestrator.ts        — Coordinates all pipeline components\n    vad.ts                 — Voice Activity Detection (Silero)\n    stt.ts                 — Speech-to-Text (Whisper)\n    tts.ts                 — Text-to-Speech (Kokoro / Piper VITS fallback)\n    queue.ts               — Transcription queue with wake word filtering,\n                             pause/resume, and speech accumulation\n    embedding-extractor.ts — Speaker embedding extraction\n    speaker-verify.ts      — Cosine similarity speaker verification\n  audio/\n    capture.ts             — Mic capture via SoX rec with rate downsampling\n    playback.ts            — Audio playback via SoX play with temp files\n  profile/\n    enrollment.ts          — Voice enrollment session management\n    storage.ts             — Profile persistence\n    passive-refine.ts      — Background profile refinement\n  models/\n    registry.ts            — Model registry with optional voice downloads\n    downloader.ts          — Model download and extraction\n  config.ts                — Runtime configuration\n  logging/\n    logger.ts              — Ring buffer logger with scoped contexts\n```\n\n## Models\n\n| Model | Purpose | Size |\n|-------|---------|------|\n| Silero VAD | Voice activity detection | 2 MB |\n| Whisper tiny.en | Speech-to-text | 150 MB |\n| WeSpeaker ResNet34 | Speaker verification | 20 MB |\n| Kokoro v0.19 | Text-to-speech (24kHz) | 330 MB |\n| Kokoro voices | Voice embeddings | 5.5 MB |\n| GTCRN | Speech denoising | 0.5 MB |\n\n## MCP Tools\n\n| Tool | Description |\n|------|-------------|\n| `GetVoiceStatus` | Pipeline state, VAD activity, listening mode, pause state |\n| `ListenForResponse` | Block until speech detected (with accumulation) |\n| `SpeakText` | TTS synthesis + playback, supports `expect_response` for Q&A flow |\n| `StartEnrollment` | Begin/advance voice enrollment session |\n| `TestEnrollment` | Verify enrollment quality |\n| `SaveProfile` | Persist voice profile |\n| `ResetProfile` | Delete voice profile |\n| `SetMode` | Change capture mode |\n| `SetThreshold` | Adjust VAD sensitivity or speaker confidence at runtime |\n| `DownloadModels` | Download missing ML models |\n| `GetDebugLog` | Ring buffer log entries |\n| `GetSessionStats` | Utterance counts, verification rates, latency stats |\n\n## Installation\n\n### Prerequisites\n\n- **macOS** (CoreAudio required for mic capture)\n- **Node.js 18+**\n- **SoX** — audio capture and playback\n  ```bash\n  brew install sox\n  ```\n\n### Install as Claude Code Plugin\n\n```bash\n# Add the Jarvis marketplace\n/plugin marketplace add civitas-cerebrum/jarvis-plugin\n\n# Install the plugin\n/plugin install jarvis-voice@jarvis-marketplace\n```\n\nDependencies install automatically on first session start. Voice models (~300MB) download on first use.\n\n### Activate\n\nStart a Claude Code session and say:\n```\n/jarvis-voice:jarvis-voice\n```\n\nOn first run, Jarvis will guide you through voice enrollment (~30 seconds) to learn your voice for speaker verification.\n\n## Development\n\n```bash\nnpm install\nnpm run build\nnpm test                           # 67 unit tests\nnpx vitest run tests/acoustic/     # Acoustic loopback self-test (needs mic + speakers)\n```\n\n## Stability Fixes\n\nThe plugin includes several reliability improvements discovered during development:\n\n- **File-based playback** — SoX `play` doesn't handle Node.js socketpair stdin correctly; writing PCM to temp files avoids truncated audio\n- **SoX rate effect** — macOS audio hardware doesn't support 16kHz capture natively; `rec` with the `rate` SoX effect downsamples from 48kHz\n- **Readline lifecycle** — MCP stdio server registers close/error handlers to prevent orphan processes\n- **EPIPE tolerance** — Process-level `uncaughtException` handler treats EPIPE as non-fatal\n- **Deferred speech processing** — `setImmediate` prevents native addon calls from blocking the event loop during MCP I/O\n",
  "bytes": 5978,
  "sha": "82837dbd31b06c5cdb443f30652ad6d67ca46532af6bed417f0ee0dbba005ee4",
  "repo_slug": "civitas-cerebrum/jarvis-plugin",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_civitas_cerebrum_jarvis_plugin_jarvis_18c5ba87/readme"
}