{
  "markdown": "# speech-to-cli\n\n**Live docs:** [jphein.github.io/speech-to-cli](https://jphein.github.io/speech-to-cli/)\n\nVoice interface for AI coding assistants — talk to your CLI agent and hear it respond, powered by Azure Speech Services. Works with **GitHub Copilot CLI**, **Claude Code**, and **Gemini CLI**.\n\n![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue)\n![Platform](https://img.shields.io/badge/platform-Linux-lightgrey)\n![License](https://img.shields.io/badge/license-GPLv3-green)\n\n## What it does\n\nThis project adds voice input and output to your terminal AI workflow via the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/):\n\n| Tool | Description |\n|------|-------------|\n| **MCP Server** (`mcp_speech.py`) | Integrates with Copilot CLI, Claude Code, and Gemini CLI via MCP — gives your AI `listen`, `speak`, and `converse` tools |\n| **Voice Chat** (`voice_chat.py`) | Standalone voice chat companion — runs in a second terminal alongside Copilot CLI |\n| **Speech-to-Text** (`speech.py`) | Simple mic → text → clipboard tool |\n| **Text-to-Speech** (`tts.py`) | Simple text → speech tool (reads from args, stdin, or clipboard) |\n| **Listen** (`listen.py`) | One-shot scriptable listen — record until silence, transcript on stdout (for agent voice loops) |\n| **Speak** (`speak.py`) | Agent-friendly TTS — per-call `--voice` override, uses `tts_region` (HD voices) |\n\n## Features\n\n- **Azure HD Voices**: Uses high-quality DragonHD voices for natural-sounding speech.\n- **Thinking Hum**: A subtle 150Hz tone that loops while the AI is processing.\n- **Visual Status**: Colorful progress bars with live VU meters and real-time subtitles (🎤/🧠/🔊).\n- **Audio Feedback**: Configurable chimes for ready, processing, speak, and done states.\n- **Low Latency**: Streaming playback and persistent connections for fast responses.\n- **VAD**: Energy-gated voice activity detection that auto-calibrates to your environment.\n\n## About GitHub Copilot CLI\n\n[GitHub Copilot CLI](https://docs.github.com/copilot/concepts/agents/about-copilot-cli) brings agentic AI coding assistance directly to your terminal. It can edit files, run commands, search code, and interact with GitHub — all through natural language.\n\n### Available models\n\nCopilot CLI gives you access to multiple frontier models via the `/model` command:\n\n| Model | Tier |\n|-------|------|\n| Claude Sonnet 4.5 | Standard (default) |\n| Claude Sonnet 4 | Standard |\n| Claude Opus 4.5 | Premium |\n| Claude Opus 4.6 | Premium |\n| Claude Haiku 4.5 | Fast |\n| GPT-5.1 / 5.2 / 5.4 | Standard |\n| GPT-5.1-Codex / 5.2-Codex / 5.3-Codex | Standard |\n| GPT-5.1-Codex-Max | Standard |\n| GPT-5 mini | Fast |\n| Gemini 3 Pro (Preview) | Standard |\n\n### Free for students\n\n**GitHub Education** members get **Copilot Pro free for 1 year**, which includes Copilot CLI access. Sign up at [education.github.com](https://education.github.com/) with your school email — no credit card required. Each prompt uses one premium request from your monthly quota.\n\n### Install Copilot CLI\n\n```bash\ncurl -fsSL https://gh.io/copilot-install | bash\ncopilot          # launch and authenticate\n```\n\n## Quick start\n\n### Prerequisites\n\n- Linux with ALSA audio (`arecord`/`aplay`)\n- Python 3.8+\n- An [Azure Speech Services](https://azure.microsoft.com/en-us/products/ai-services/speech-services) API key (free tier: 5 hours STT + 500K characters TTS per month)\n\n### Azure Speech HD voices\n\nThis project defaults to **Azure HD (DragonHD) voices** — specifically `en-US-Ava:DragonHDLatestNeural`. These are Azure's highest-quality neural voices with natural intonation, breathing, and expressiveness that sounds remarkably human. You can change the voice via the `AZURE_SPEECH_VOICE` env var or config file.\n\nBrowse all available voices in the [Azure Voice Gallery](https://speech.microsoft.com/portal/voicegallery). Look for voices tagged **HD** or **DragonHD** for the best quality.\n\n### Azure for nonprofits and education\n\n- **Nonprofits**: Microsoft offers up to **$3,500/year in free Azure credits** through [Azure for Nonprofits](https://nonprofit.microsoft.com/en-us/getting-started). This more than covers Speech Services usage for voice-enabled Copilot workflows.\n- **Students**: The [Azure for Students](https://azure.microsoft.com/en-us/free/students/) program provides **$100 in free credits** with no credit card required — just verify with your school email.\n\n### Install\n\n```bash\ngit clone https://github.com/techempower-org/speech-to-cli.git\ncd speech-to-cli\n./install.sh\n```\n\n### Configure\n\nSet your Azure credentials via environment variables:\n\n```bash\nexport AZURE_SPEECH_KEY=\"your-key-here\"\nexport AZURE_SPEECH_REGION=\"westus2\"           # optional, default: westus2\nexport AZURE_SPEECH_VOICE=\"en-US-Ava:DragonHDLatestNeural\"  # optional\n```\nOr create a JSON config file at `~/.config/speech-to-cli/config.json`. You can create the directory if it doesn't exist:\n\n```bash\nmkdir -p ~/.config/speech-to-cli\n```\n\nExample `config.json`:\n\n```json\n{\n  \"key\": \"your-azure-speech-key\",\n  \"region\": \"westus2\",\n  \"voice\": \"en-US-Ava:DragonHDLatestNeural\",\n  \"fast_voice\": \"en-US-AvaNeural\",\n  \"chime_hum\": true,\n  \"visual_indicator\": true\n}\n```\n\n### Configuration Settings\n\n| Key | Default | Description |\n|-----|---------|-------------|\n| `key` | None | Your Azure Speech Services API key. |\n| `region` | `westus2` | The Azure region for your speech resource. |\n| `voice` | `en-US-Ava:DragonHDLatestNeural` | Primary voice for high-quality (HD) synthesis. |\n| `fast_voice` | `en-US-AvaNeural` | Low-latency voice for fast responses. |\n| `chime_ready` | `true` | Play an ascending tone when the microphone opens. |\n| `chime_processing`| `false` | Play a short \"blip\" when speech is recognized. |\n| `chime_hum` | `false` | Start a looping 150Hz tone while thinking. |\n| `chime_speak` | `false` | Play a descending tone before starting to speak. |\n| `chime_done` | `false` | Play a double-tap tone when the AI is done talking. |\n| `visual_indicator`| `true` | Show status icons (🎤/🧠) in the terminal. |\n| `live_subtitles` | `true` | Show real-time partial transcription in progress bar. |\n| `vu_meter` | `true` | Show live volume meter in progress bar. |\n| `silence_timeout` | `3.0` | Seconds of silence after speech before auto-stop. |\n| `loop_silence_timeout` | `1.2` | Silence timeout in continuous loop mode (shorter = faster turnaround). |\n| `no_speech_timeout` | `7.0` | Max seconds to wait for any speech before giving up. |\n| `talk_silence_timeout` | `4.0` | Silence timeout for talk/converse mode. |\n| `energy_multiplier` | `2.5` | Noise gate threshold multiplier. Lower = more sensitive. |\n| `half_duplex` | `\"auto\"` | `\"auto\"` (detect speakers/headphones), `\"true\"`, or `\"false\"`. |\n| `continuous_dictation` | `false` | Auto-restart listening after each utterance (used by gnome-speaks). |\n| `dictation_mode` | `true` | Type transcribed text at cursor (vs clipboard only). |\n| `terminal_mode` | `false` | All lowercase, no auto-capitalization. |\n| `end_word` | `\"over\"` | Say this word to immediately stop recording. |\n| `max_record_seconds` | `120` | Absolute maximum recording duration. |\n| `debug` | `false` | Write detailed logs to `/tmp/speech-debug.log`. |\n\n> ⚠️ **Never commit your API key.** Use environment variables or the config file (which is in your home directory, outside the repo). See [Security](#security) below.\n\n### Offline fallback (Wyoming)\n\nIf you run a [Wyoming](https://github.com/rhasspy/wyoming) server on your LAN\n(e.g. Piper for TTS, an ONNX-ASR/whisper server for STT — the same services\nHome Assistant's Assist uses), STT and TTS automatically fall back to it on\nnetwork-class Azure failures (timeouts, connection errors, HTTP 5xx — never on\ncontent errors or recognized silence):\n\n| Key | Default | Meaning |\n|-----|---------|---------|\n| `wyoming_host` | `\"\"` | LAN hostname/IP of the Wyoming server; empty disables the feature. |\n| `wyoming_tts_port` | `10200` | Wyoming TTS (e.g. wyoming-piper). |\n| `wyoming_stt_port` | `10300` | Wyoming STT (e.g. wyoming-onnx-asr / wyoming-faster-whisper). |\n| `wyoming_tts_voice` | `\"\"` | Piper voice name; empty uses the server default. |\n| `wyoming_wake_port` | `10400` | Wyoming wake-word server (e.g. wyoming-openwakeword). |\n| `wake_word` | `false` | Arm the wake-word watcher (consumers like gnome-speaks stream mic audio for detection). |\n| `wake_word_model` | `\"\"` | openwakeword model name — effectively your wake phrase; keep it out of public repos. |\n\nA 60-second circuit breaker skips Azure entirely while it's marked down, so\noffline calls don't wait out the timeout. Set `SPEECH_FORCE_OFFLINE=1` to force\nthe fallback (manual offline mode; also the test lever). Degradation is\ngraceful: everything speaks in the single Piper voice, and streaming dictation\nloses live partials (the final transcript still arrives).\n\n## Usage\n\n### MCP Server (recommended)\n\nThe MCP server works with any AI CLI that supports the Model Context Protocol. **When paired with a modern terminal AI like Gemini CLI, Claude Code, or Copilot CLI, it creates an incredibly seamless voice loop.** The AI automatically invokes the `listen` tool when it needs input, processes your request, and calls the `speak` tool to respond—all while providing rich, real-time terminal UI feedback (like live subtitles and VU meters) without you ever needing to type a command.\n\n#### GitHub Copilot CLI\n\nAdd to your `~/.copilot/mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"azure-speech\": {\n      \"command\": \"python3\",\n      \"args\": [\"/path/to/speech-to-cli/mcp_speech.py\"]\n    }\n  }\n}\n```\n\n#### Claude Code\n\nOption A — Use the included `.mcp.json` (auto-detected when working in the project directory).\n\nOption B — Add globally via the CLI:\n\n```bash\nclaude mcp add --transport stdio azure-speech -- python3 /path/to/speech-to-cli/mcp_speech.py\n```\n\n#### Gemini CLI\n\nInstall as a Gemini extension (uses the included `gemini-extension.json`):\n\n```bash\ngemini extensions install /path/to/speech-to-cli\n```\n\nOr add manually to your `~/.gemini/settings.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"azure-speech\": {\n      \"command\": \"python3\",\n      \"args\": [\"/path/to/speech-to-cli/mcp_speech.py\"]\n    }\n  }\n}\n```\n\n---\n\nRestart your CLI — it will now have `listen`, `speak`, `multi_speak`, and `converse` tools available. Just say \"listen\" and your AI will record your voice, transcribe it, and respond. Ask it to \"speak\" and it'll read its response aloud. Use \"converse\" for a continuous voice chat loop. Use `multi_speak` for multi-agent conversations where each agent speaks with a different voice.\n\n**MCP Tools:**\n\n| Tool | Parameters | Description |\n|------|-----------|-------------|\n| `listen` | `seconds` (1-30), `mode` (streaming/vad/whisper/fixed) | Records from mic, returns transcribed text |\n| `speak` | `text` (required), `quality` (fast/hd) | Speaks text aloud via Azure TTS |\n| `talk` | `text` (required), `quality` (fast/hd) | Speaks text then listens for a reply — full-duplex TTS+STT in one call |\n| `converse` | `seconds`, `mode` | Like `listen`, but signals conversational intent — Copilot will speak its reply then listen again |\n| `multi_speak` | `segments` (array of {text, voice}), `quality` (fast/hd) | Speak multiple text+voice segments in one call — TTS requests fire in parallel, audio plays back-to-back |\n| `multi_speak_stream` | `segments`, `quality` | Like `multi_speak` with streaming progress events |\n| `configure` | `key`, `value` | View or change runtime settings |\n| `get_voices` | (none) | List available Azure TTS voices |\n| `pause` | (none) | Pause current TTS playback |\n| `resume` | (none) | Resume paused TTS playback |\n\n**STT Modes** (auto-selected by default):\n\n| Mode | Description |\n|------|-------------|\n| `streaming` | Real-time Azure WebSocket + energy-gated VAD (fastest, default) |\n| `vad` | Record with VAD, upload on silence |\n| `whisper` | Local transcription via faster-whisper (offline, no network) |\n| `fixed` | Record for full duration, then upload (fallback) |\n\n**TTS Quality:**\n\n| Quality | Voice | Azure latency | Best for |\n|---------|-------|--------------|----------|\n| `fast` (default) | AvaNeural | ~120ms | Conversation, quick responses |\n| `hd` | DragonHD | ~1200ms | High-quality narration |\n\n### Half-Duplex vs Full-Duplex\n\nThe `half_duplex` setting controls whether TTS and STT can overlap:\n\n- **`\"auto\"`** (default): Auto-detects speakers vs headphones. Headphones get full duplex; speakers get half duplex.\n- **`\"true\"`** (half duplex): TTS must finish before the mic opens. Prevents the mic from hearing speaker output. A 0.5s drain buffer is added after TTS ends.\n- **`\"false\"`** (full duplex): TTS and STT can run simultaneously. The recorder is prewarmed during TTS so listening starts immediately. Only works well with headphones.\n\n### Talk Tool\n\nThe `talk` MCP tool combines TTS and STT in a single call — it speaks the provided text, then immediately listens for the user's reply:\n\n```\ntalk(text=\"What would you like me to do?\")\n→ speaks the text, records user's reply, returns transcribed text\n```\n\nThis is the same mechanism that gnome-speaks exposes over D-Bus as `org.gnome.Speaks.Talk(text)`. When called via the MCP server, it uses `speech_tts.talk_fullduplex()` which handles the TTS→STT handoff, respecting the duplex setting. The `talk_silence_timeout` (default 4.0s) controls how long it waits for a reply.\n\n### Voice Chat (standalone companion)\n\nRun in a separate terminal alongside Copilot CLI:\n\n```bash\npython3 voice_chat.py\n```\n\nFlow:\n1. Press **Enter** → records your voice → transcribes → copies to clipboard\n2. Paste into Copilot CLI with **Ctrl+Shift+V**\n3. Copy Copilot's response with **Ctrl+Shift+C**\n4. Press **Enter** → speaks the response aloud → starts recording again\n\n### Standalone tools\n\n```bash\n# Speech-to-text: record and transcribe to clipboard\npython3 speech.py\n\n# Text-to-speech: speak text aloud\npython3 tts.py \"Hello world\"\necho \"Hello\" | python3 tts.py\npython3 tts.py  # speaks clipboard contents\n\n# One-shot listen: record until silence, print transcript to stdout\npython3 listen.py                        # exit 0 = speech, 1 = silence\npython3 listen.py --markers --silence-timeout 2\n\n# Speak with a per-call voice (no env juggling; HD voices via tts_region)\npython3 speak.py --voice en-US-AndrewNeural \"A different character\"\n```\n\n### Agent voice loop (no MCP needed)\n\nWhen the MCP server isn't connected in a session, an AI CLI agent can still\nhold a real voice conversation from Bash — speak a turn, listen for the reply,\nrespond, repeat:\n\n```bash\npython3 speak.py --voice en-US-AndrewNeural \"What do you think?\" \\\n  && python3 listen.py --markers 2>/dev/null\n```\n\nProgress/VU noise goes to stderr; only the transcript reaches stdout. Born in a\nlive smol brainstorm session where Claude pitched roadmap ideas by voice using\nexactly this loop.\n\n## How it works\n\n- **Recording**: Uses `arecord` (ALSA) to capture 16kHz mono audio from the default input device\n- **Voice Activity Detection**: Energy-gated VAD auto-calibrates to ambient noise, stops recording on silence (~400ms after speech ends)\n- **Speech-to-Text**: Streams audio to Azure via persistent WebSocket for real-time recognition, with local Whisper fallback for offline use\n- **Text-to-Speech**: Sends SSML to Azure TTS REST API with HTTP connection pooling; streams MP3 audio through mpv for immediate playback\n- **Ready chime**: A short ascending tone plays before each recording so you know when to speak\n- **Performance**: Connections are pre-warmed on startup; noise floor is cached between calls; response latency is ~275ms from end of speech to first audio byte\n- **MCP Protocol**: Implements [MCP](https://modelcontextprotocol.io/) (v2024-11-05) over stdio JSON-RPC for direct Copilot CLI integration\n\nNo Azure SDK required — just plain REST/WebSocket API calls.\n\n## Used as a library\n\nOther projects import the core modules from this repository directly rather than going through the MCP server:\n\n- **[gnome-speaks](https://github.com/techempower-org/gnome-speaks)** — a GNOME Shell extension that adds voice control to the desktop. It imports `state.py`, `audio.py`, `stt.py`, and `speech_tts.py` via `sys.path`.\n- **[the-oracle](https://github.com/techempower-org/the-oracle)** — proxies this MCP server through a FastMCP gateway so multiple clients can share a single speech backend.\n\nThe env var `SPEECH_ENGINE_PATH` can be set to the path of this directory so that downstream projects can locate and import the modules at runtime.\n\n**File roles:**\n\n| File | Role |\n|------|------|\n| `state.py` | Shared state, constants, config, helpers |\n| `audio.py` | Audio I/O, device detection, chimes, UI, VAD |\n| `stt.py` | Speech-to-text backends |\n| `speech_tts.py` | Text-to-speech (Azure TTS, multi_speak) |\n| `mcp_speech.py` | MCP protocol adapter (tool schemas, request routing, stdio) |\n| `speech.py` | Standalone CLI — mic to clipboard |\n| `tts.py` | Standalone CLI — text to speech |\n| `voice_chat.py` | Standalone CLI — interactive voice chat loop |\n\nIf you are building on top of these modules, import the four core library files (`state`, `audio`, `stt`, `speech_tts`) and leave `mcp_speech.py` and the standalone CLIs out of your import graph.\n\n## Security\n\nThis project handles audio data and API credentials. Please review:\n\n### API key management\n- **Never hardcode your Azure key in source code or commit it to git.** The `.gitignore` includes `.env` to help prevent this.\n- Store your key via environment variable (`AZURE_SPEECH_KEY`) or in the user-level config file (`~/.config/speech-to-cli/config.json`).\n- Azure keys can be rotated at any time in the [Azure Portal](https://portal.azure.com/) → your Speech resource → Keys and Endpoint.\n- Consider using a restricted key with only Speech Services access (not a broad subscription key).\n\n### Audio and data privacy\n- **Audio is recorded from your local microphone** and sent to Azure Speech Services for processing. No audio is stored locally after transcription (temp files are deleted immediately).\n- **Azure processes your audio** to produce transcriptions. Review the [Azure AI Services data privacy policy](https://learn.microsoft.com/en-us/legal/cognitive-services/speech-service/speech-to-text/data-privacy-security) to understand how Microsoft handles your audio data.\n- **Text sent to TTS** is transmitted to Azure for synthesis. The same privacy policies apply.\n- **No data is sent anywhere other than Azure Speech Services** — there are no analytics, telemetry, or third-party services.\n\n### Network security\n- All Azure API calls use **HTTPS** (TLS encrypted in transit).\n- Audio data and API keys are sent over encrypted connections only.\n\n### MCP server scope\n- The MCP server exposes tools (`listen`, `speak`, `multi_speak`, `converse`, `configure`, `get_voices`, `pause`, `resume`). It cannot read files, execute commands, or access anything beyond the microphone and Azure API.\n- The server communicates with Copilot CLI over local stdio only — no network listeners are opened.\n\n### Recommendations\n- Rotate your Azure key periodically.\n- Use Azure's free tier to limit potential cost exposure from a leaked key.\n- If running on a shared machine, be aware that other users with access to your environment variables or config file can read your API key.\n\n## Legal\n\n### Azure Speech Services\nUse of Azure Speech Services is subject to the [Microsoft Azure terms of service](https://azure.microsoft.com/en-us/support/legal/) and the [Azure AI Services terms](https://www.microsoft.com/licensing/terms/productoffering/MicrosoftAzure). You are responsible for your own Azure usage and billing.\n\n### GitHub Copilot\nUse of GitHub Copilot CLI requires an active Copilot subscription and is subject to the [GitHub Copilot terms](https://docs.github.com/en/site-policy/github-terms/github-terms-for-additional-products-and-features#github-copilot). Copilot is free for verified students, teachers, and maintainers of popular open-source projects.\n\n### This project\nThis project is independently developed and is **not affiliated with, endorsed by, or sponsored by Microsoft or GitHub**. It is a third-party integration that connects to their respective APIs.\n\nLicensed under the GNU General Public License v3.0 (GPLv3) — see [LICENSE](LICENSE).\n\n## License\n\nGNU General Public License v3.0 (GPLv3) — see [LICENSE](LICENSE) for details.\n",
  "bytes": 20395,
  "sha": "3c74c6c21893c2f614b04b3b81619c07baa1c8266219d23690d3c69f34565baa",
  "repo_slug": "techempower-org/speech-to-cli",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_techempower_org_speech_to_cli_4eae2cd8/readme"
}