{
"markdown": "# talkthrough-mcp\n\n\n\n[](https://github.com/korovin-aa97/talkthrough-mcp/actions/workflows/ci.yml)\n[](LICENSE)\n[](pyproject.toml)\n[](https://pypi.org/project/talkthrough-mcp/)\n\n[Quickstart](#quickstart) · [Tools](#tools) · [Benchmarks](benchmarks/) ·\n[FAQ](#faq) · [Troubleshooting](docs/TROUBLESHOOTING.md) ·\n[Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md)\n\n# Don't write a bug report. Record it.\n\nGive Claude Code or Codex a narrated `.mov`/`.mp4` — or a public video link —\nand talkthrough turns it into searchable transcript, exact frames, OCR and\nwall-clock timestamps, locally — so your agent writes an evidence-backed\nissue draft or investigates the fix.\n\nAlso works for meetings, workshops, product demos, and production incidents.\n\n\n\n*Illustration — an animation of the `/talkthrough:bug` storyline, not a screen\ncapture: the recording is indexed locally (transcript · keyframes · OCR ·\nwall-clock), the evidence checkpoint is assembled, and the draft is ready to\nfile with your own tracker tooling (`gh`, Jira, a GitHub MCP server —\ntalkthrough itself never leaves your machine and never posts anything).*\n\n**Real runs, no animation:**\n\n- **[▶ Watch the demo with sound (1:18)](https://youtu.be/nHfGfEiVdE8)** — an\n unedited session: a narrated recording goes in, a ready-to-file\n `bug-report.md` comes out.\n- **[Silent recording → issue draft](examples/bug-from-silent-recording/)** —\n the whole thing as files you can re-run: a Playwright-recorded, audio-free\n `.mp4` and the unedited agent output. Every number is reproducible —\n processing that file gives `job_id 8703a66bbe77a7d0` (the job id *is* the\n sha256 prefix, so `shasum -a 256` predicts it), 17 keyframes / 4 unique, and\n 0 transcript segments because there is no audio track.\n\n## Quickstart\n\nOne command, no system dependencies: ffmpeg falls back to a bundled build,\nOCR is pip-only, and whisper models download themselves on first use. The\nonly prerequisite is [uv](https://docs.astral.sh/uv/) (`brew install uv` or\n`curl -LsSf https://astral.sh/uv/install.sh | sh`).\n\nCold setup has two separate stages: `uvx` first resolves a compatible Python\nand the pinned server environment, then the first `process_media` downloads\nany missing media/model assets. A plugin update can create a new environment\nand, without system ffmpeg, fetch the ~80 MB bundled ffmpeg again; shared\nWhisper/OCR/diarization caches and warm, network-free jobs remain reusable.\nSee [Troubleshooting](docs/TROUBLESHOOTING.md#first-run-is-slow--downloads-a-lot).\n\n\n\n[](https://cursor.com/en/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyItLXB5dGhvbiIsIj49My4xMSw8My4xNCIsInRhbGt0aHJvdWdoLW1jcFtkaWFyaXphdGlvbix1cmxdIl19)\n[](https://vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22--python%22%2C%22%3E%3D3.11%2C%3C3.14%22%2C%22talkthrough-mcp%5Bdiarization%2Curl%5D%22%5D%2C%22type%22%3A%22stdio%22%7D)\n[](https://insiders.vscode.dev/redirect/mcp/install?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22--python%22%2C%22%3E%3D3.11%2C%3C3.14%22%2C%22talkthrough-mcp%5Bdiarization%2Curl%5D%22%5D%2C%22type%22%3A%22stdio%22%7D&quality=insiders)\n[](https://lmstudio.ai/install-mcp?name=talkthrough&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyItLXB5dGhvbiIsIj49My4xMSw8My4xNCIsInRhbGt0aHJvdWdoLW1jcFtkaWFyaXphdGlvbix1cmxdIl19)\n[](https://kiro.dev/launch/mcp/add?name=talkthrough&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22--python%22%2C%22%3E%3D3.11%2C%3C3.14%22%2C%22talkthrough-mcp%5Bdiarization%2Curl%5D%22%5D%7D)\n\n### Claude Code\n\nTwo install paths — **pick one**, not both (the plugin already includes\nthe server; installing both would register it twice):\n\n**Server only** — the 9 tools + 6 prompts, and nothing else on your\nsystem. Choose this for a minimal setup, or when you manage MCP servers\nyourself across several clients:\n\n```bash\nclaude mcp add -s user talkthrough -- uvx --python \">=3.11,<3.14\" \"talkthrough-mcp[diarization,url]\"\n```\n\n**Full plugin** — the same server, plus native slash commands\n(`/talkthrough:bug`, `/talkthrough:triage-recording`, …) that handle the\nceremony for you, a ready-made triage subagent, and an agent skill that\nteaches Claude the workflow. Choose this for the best out-of-the-box\nexperience:\n\n```\n/plugin marketplace add korovin-aa97/talkthrough-mcp\n/plugin install talkthrough@talkthrough\n```\n\n### Every other MCP client\n\n\nClaude Desktop
\n\n`claude_desktop_config.json`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/claude-desktop/`](integrations/claude-desktop/)\n\n \n\n\nCursor
\n\n`~/.cursor/mcp.json (or project .cursor/mcp.json)`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/cursor/`](integrations/cursor/)\n\n \n\n\nOpenAI Codex CLI
\n\n`~/.codex/config.toml (or project-scoped .codex/config.toml in trusted projects)`:\n\n```toml\n[mcp_servers.talkthrough]\ncommand = \"uvx\"\nargs = [\"--python\", \">=3.11,<3.14\", \"talkthrough-mcp[diarization,url]\"]\n```\n\nMore: [`integrations/codex/`](integrations/codex/)\n\n \n\n\nGemini CLI
\n\n`~/.gemini/settings.json`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/gemini-cli/`](integrations/gemini-cli/)\n\n \n\n\nCline / Roo Code
\n\n`cline_mcp_settings.json (via MCP Servers UI)`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/cline/`](integrations/cline/)\n\n \n\n\nOpenClaw
\n\n`~/.openclaw/openclaw.json`:\n\n```json\n{\n \"mcp\": {\n \"servers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n }\n}\n```\n\nMore: [`integrations/openclaw/`](integrations/openclaw/)\n\n \n\n\nOpenCode
\n\n`opencode.json (project) or ~/.config/opencode/opencode.json`:\n\n```json\n{\n \"mcp\": {\n \"talkthrough\": {\n \"type\": \"local\",\n \"command\": [\n \"uvx\",\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ],\n \"enabled\": true\n }\n }\n}\n```\n\nMore: [`integrations/opencode/`](integrations/opencode/)\n\n \n\n\nGoose
\n\n`~/.config/goose/config.yaml`:\n\n```yaml\nextensions:\n talkthrough:\n enabled: true\n type: stdio\n cmd: uvx\n args: [\"--python\", \">=3.11,<3.14\", \"talkthrough-mcp[diarization,url]\"]\n```\n\nMore: [`integrations/goose/`](integrations/goose/)\n\n \n\n\nGitHub Copilot CLI
\n\n`~/.copilot/mcp-config.json`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/copilot-cli/`](integrations/copilot-cli/)\n\n \n\n\nWindsurf
\n\n`~/.codeium/windsurf/mcp_config.json`:\n\n```json\n{\n \"mcpServers\": {\n \"talkthrough\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n}\n```\n\nMore: [`integrations/windsurf/`](integrations/windsurf/)\n\n \n\n\nZed
\n\n`settings.json (Zed)`:\n\n```json\n{\n \"context_servers\": {\n \"talkthrough\": {\n \"source\": \"custom\",\n \"command\": {\n \"path\": \"uvx\",\n \"args\": [\n \"--python\",\n \">=3.11,<3.14\",\n \"talkthrough-mcp[diarization,url]\"\n ]\n }\n }\n }\n}\n```\n\nMore: [`integrations/zed/`](integrations/zed/)\n\n \n\nAny other MCP stdio client uses the same server command: `uvx --python \">=3.11,<3.14\" \"talkthrough-mcp[diarization,url]\"`.\nPer-engine folders with exactly these snippets plus verification steps live\nin [`integrations/`](integrations/); agents can self-install via\n[`llms-install.md`](llms-install.md).\n\n\n\n### Who said what (speaker diarization) — included in the configs above\n\nMulti-person recordings (meetings, interviews, panels) can carry `S1`/`S2`/…\nspeaker labels. Every install button, snippet, and the plugin above already\nship the `[diarization]` engine, so asking your agent \"who said what\" just\nworks — diarization itself still runs only when requested per call\n(`process_media(path=..., diarize=true, num_speakers=)`),\nand its models download once on first use.\n\nPrefer the minimal server without the diarization engine? Use\n`uvx --python \">=3.11,<3.14\" talkthrough-mcp` as the command instead (the MCP registry entry also\nresolves to this lean form) — an explicit `diarize=true` will then answer\nwith the one-line install fix. Details in\n[Speakers](#speakers-optional-diarization).\n\n### Upgrading from 0.3.x\n\nRegenerated configs and the plugin carry `[diarization,url]`. A config you\nwrote by hand for 0.3.x — `uvx --python \">=3.11,<3.14\" talkthrough-mcp`, or a\npin without the `url` extra — upgrades the server in place and lists the new\n`process_url` tool,\nbut only direct `https://` media links work until the extra is there:\nYouTube and video pages answer with the one-line install fix. Add `url` to\nyour command (`uvx --python \">=3.11,<3.14\" \"talkthrough-mcp[diarization,url]\"`),\nrestart the client, and check with `talkthrough-mcp --version` (0.4.1+),\nwhich names the extras the environment has; the server logs the same line\nto stderr at every start, so your client's MCP log shows it too.\n\n### Local checkout (development)\n\n```bash\ngit clone https://github.com/korovin-aa97/talkthrough-mcp\nclaude mcp add talkthrough -- uv run --directory /path/to/talkthrough-mcp talkthrough-mcp\n```\n\nThen, in your agent:\n\n> Process `~/Desktop/recording.mov` and triage it — or just invoke the\n> `triage-recording` server prompt.\n\n## Tools\n\n| Tool | What it does |\n|---|---|\n| `process_media(path, recorded_at?, vocabulary?, language?, model?, diarize?, num_speakers?, force?)` | Ingest a video/audio file: local STT, keyframes, OCR, wall-clock, opt-in speaker labels. Returns a compact summary. Idempotent by content hash — re-calls are instant; `diarize=true` on a processed job adds speakers without re-transcribing. |\n| `process_url(url, recorded_at?, vocabulary?, language?, model?, diarize?, num_speakers?, refresh?, force?)` | The one network tool: download one public video/audio URL once (a direct `https://` media link, one YouTube video, or any public video page yt-dlp can read — with the `[url]` extra), keep the source inside the job, then run the same local pipeline. Same-URL re-calls serve the stored job without network unless `refresh=true`; `force=true` rebuilds from the kept source; the raw URL is never stored. |\n| `get_transcript(job_id, start_ms?, end_ms?, format?)` | Paginated transcript as `segments`, `text`, or `srt` (speaker-prefixed when diarized, plus a roster header); truncation returns `next_start_ms`. |\n| `get_frames(job_id, at_ms? \\| start_ms?+end_ms?, max_frames?, include_duplicates?)` | Keyframe images nearest a timestamp or evenly thinned across a range (unique frames by default, max 6/call); each frame names its absolute `path`. |\n| `get_moment(job_id, start_ms, end_ms)` | The \"one remark\" bundle: transcript slice + up to 3 frames + their OCR text + wall-clock range (+ `speakers_in_range` when diarized). |\n| `search(job_id, query, speaker?, match_mode?)` | Substring search over the transcript AND on-screen OCR text. `all_words` remains the default; `any_word` broadens lexical recall. Hits carry `t_ms`/`t_wall`, frame refs, and the speaker when diarized. The optional filter accepts a raw label or saved name. |\n| `label_speakers(job_id, labels, evidence?)` | Atomically persist verified names for anonymous speaker labels. Raw `S1`/`S2` labels remain canonical; blank/null removes a name. |\n| `extract_frame(job_id, at_ms, crop?)` | Exact-timestamp full-resolution re-extract from the source video (optional crop) when keyframes miss the instant; returns the file's absolute `path`. URL jobs decode their kept source — no network. |\n| `list_jobs()` | Recent processed recordings with source paths, durations, wall-clock starts, counts, speaker counts when diarized, and the provider/id for URL jobs. |\n\nEvery tool description ships 10+ usage examples, so agents pick the right tool\nwithout extra prompting.\n\n## Server prompts (slash commands in MCP clients)\n\n| Prompt | Workflow |\n|---|---|\n| `bug` | One recording → evidence-backed GitHub issue draft (silent, narration-free recordings work too) |\n| `triage-recording` | Narrated screencast → precise findings JSON (bug/feature/question routing, frame evidence) |\n| `spec-from-workshop` | Recorded workshop → structured spec with quoted decisions and open questions |\n| `backlog-from-demo` | Product demo → prioritized backlog with timestamped evidence |\n| `meeting-actions` | Meeting audio → action items, decisions, open questions |\n| `correlate-with-logs` | Recording remarks ↔ system logs via wall-clock windows |\n\nThe same prompts live as plain files in [`examples/prompts/`](examples/prompts/)\nif your client doesn't surface MCP prompts. The findings contract used by\n`triage-recording` is [`examples/output-contract.schema.json`](examples/output-contract.schema.json).\n\n## Works as a skill too (no MCP required)\n\nThe same workflow ships as a cross-engine [Agent Skill](https://agentskills.io)\nat [`.agents/skills/talkthrough/`](.agents/skills/talkthrough/) — Claude Code,\nCodex CLI (`$talkthrough`), Cursor, Copilot, Gemini CLI, Goose and other\nSKILL.md-compatible tools read it. Agents without MCP wiring can drive the\n[CLI](#cli) directly: `talkthrough-mcp process recording.mov --json` prints the\nsame summary the MCP tool returns, and the job store is shared either way.\n\n## Wall-clock anchoring\n\nEvery timestamped result carries both `t_ms` (video-relative) and `t_wall`\n(ISO 8601 real time) once the recording start is known. Resolution ladder:\n\n1. `recorded_at` parameter (agent/user override) → confidence `exact`\n2. QuickTime `com.apple.quicktime.creationdate` tag, carries the local\n timezone (QuickTime Player recordings; ⌘⇧5 wrote it before macOS 26) → `high`\n3. Container `creation_time` tag (UTC) → `medium` — macOS 26+ ⌘⇧5/ReplayKit\n screen recordings land here (no `creationdate` tag anymore); pass\n `recorded_at=` when local-tz `t_wall` matters\n4. File mtime minus duration (recorders finalize files at recording END) → `low`\n5. Nothing → tools still work with relative `t_ms` only\n\nWhy it matters: \"the upload spinner froze *here*\" becomes a ±30 s grep window\nin your server logs.\n\n## Speakers (optional diarization)\n\nWith the `[diarization]` extra installed (included in every generated config —\nsee [Quickstart](#who-said-what-speaker-diarization--included-in-the-configs-above)),\n`process_media(diarize=true)` labels who said what — locally, like everything\nelse here ([sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) runtime, no\ntorch, no accounts, no GPU):\n\n- Speakers become `S1`, `S2`, … **in order of first appearance**; new\n diarized jobs split speaker changes at word boundaries, while old jobs\n continue to report honest segment-level precision. Every\n transcript segment gets a `speaker`, and the tools surface it everywhere —\n roster with talk time in `get_transcript`, `speakers_in_range` in\n `get_moment`, `speaker` on `search` hits, `S1:` prefixes in the text/SRT\n formats, a speaker count in `list_jobs`.\n- **Know the headcount? Pass `num_speakers`.** Clustering toward an exact k\n removes the main failure mode of unknown-count mode (similar voices merging\n or one voice splitting). It is a target, not a guarantee: the clusterer can\n converge on fewer clusters than k, and a re-run that changed nothing says\n so in the payload (`labels_changed: false`). Agents are instructed to pass\n the headcount via the tool guidance; do the same in your own calls.\n- **Already processed a recording?** Calling `process_media(diarize=true)` on\n it re-runs *only* diarization — whisper is not re-run, and labels land in\n the existing job. Same for changing `num_speakers`. The diarization stage\n itself still re-scans the full audio: minutes on long recordings.\n- **Full rebuilds keep named jobs safe.** If a job has active or pending names,\n `force=true` also requires `diarize=true`; otherwise the call refuses before\n changing stored data. A successful force rebuilds in staging and moves every\n previous identity to pending review against the fresh roster. Any processing\n or commit failure leaves the prior manifest and frames intact.\n- Labels start anonymous. After checking self-introductions, vocatives, or\n video evidence, call `label_speakers` to preserve a verified mapping such\n as `S1` → \"Alice\" across sessions. The roster can expose bounded OCR\n `name_candidates`, but those are raw hints and are never saved\n automatically. The raw label remains present beside `speaker_name`. If a\n later diarization amend changes the labels, verified names stop being\n active and move to bounded `speaker_names_pending_review` evidence instead\n of being silently lost. Re-check the current roster and explicitly confirm\n or remove each affected label with `label_speakers`.\n- Video jobs produced before 0.3.1 keep their original flat OCR and can return\n `name_candidates_note` to explain why hints are absent. They remain fully\n readable without migration; `force=true, diarize=true` regenerates line-aware\n OCR while preserving saved identities for review.\n\nModels download once (~47 MB total) from pinned, checksum-verified URLs into\n`~/.talkthrough/models/`; warm runs are zero-network like the rest of the\npipeline. Speed on an M-series CPU (4 threads): a 26-minute meeting diarizes\nin about 2 minutes (RTF ≈ 0.08), on top of the transcription time. Memory:\nexpect on the order of 1–1.5 GB peak RSS while an hour-plus meeting is being\ndiarized (measured on a real 73-minute recording); it is released when the\nstage completes.\n\n| Role | Model | Download | Weights license |\n|---|---|---|---|\n| Segmentation | [pyannote segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0) (ONNX export by k2-fsa) | 7 MB | MIT |\n| Embedding (default) | [NeMo](https://github.com/NVIDIA/NeMo) `en_titanet_small` | 40 MB | Apache-2.0 (per its NGC model card, the NeMo Toolkit license) |\n| Embedding (alt) | [WeSpeaker](https://github.com/wenet-e2e/wespeaker) `en_voxceleb_resnet34_LM` | 27 MB | CC-BY-4.0 |\n| Embedding (alt) | [3D-Speaker](https://github.com/modelscope/3D-Speaker) `campplus_sv_en_voxceleb_16k` | 30 MB | Apache-2.0 |\n\nThe default won a real-meeting accept-eval (RU/EN/ES + a 3-speaker 26-minute\nmeeting): it was the only candidate to isolate all three real voices at\n`num_speakers=3`, at 2× the speed of the runner-up.\n\nPick an alternate embedding model (or point at your own `.onnx` file for\noffline machines) via `TALKTHROUGH_DIARIZATION_EMB_MODEL`; tune the\nunknown-count sensitivity via `TALKTHROUGH_DIARIZATION_THRESHOLD` (see\n[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)). Honest quality notes\nlive in [Limitations](#limitations).\n\n## Privacy\n\nEverything runs locally: your recordings never leave your machine, speech is\ntranscribed by a local whisper model, OCR and speaker diarization are local\nONNX inference, and there is no telemetry. For local files the only network\naccess is one-time tool/model downloads (ffmpeg build, whisper model, OCR\nmodels, diarization models — the latter pinned by URL + sha256). The one\ndeliberate exception is `process_url`: it downloads the public source you\nname from its provider or CDN, once, and nothing else — no media ever goes\n*up*, no cloud STT or LLM is called, and after that download every tool on\nthe job is network-free again. The raw URL (which may carry signed tokens)\nis not stored: the job keeps a hash, the public provider id or host and a\nbounded title. Diarization keeps no voiceprint database: voice embeddings\nlive only in process memory, and only anonymous turn labels (`S1`/`S2`) land\non disk. Your agent sees only the payloads the MCP tools return (text and\nselected frames) in your existing session; talkthrough itself makes no LLM\ncalls.\n\n## Languages\n\nNarration in any of Whisper's ~99 languages works: the language is\nauto-detected per recording, and the summary reports both `language` and\n`language_probability` so agents can tell a confident detection from a shaky\none (silence or music at the start can fool the detector — pin it with\n`language=\"ru\"` and `force=true` when that happens). Speaker diarization is\nacoustic — it fingerprints voices, not words — so it is language-independent\nand works across all of those languages unchanged.\n\nPick the model for your languages — per call (`model=` parameter, agents do\nthis themselves when a transcript comes back garbled) or as the server\ndefault (`TALKTHROUGH_WHISPER_MODEL`):\n\n| Model | Size | Best for |\n|---|---|---|\n| `small` (default) | 464 MB | English and major-language narration on CPU |\n| `large-v3-turbo` | ~1.5 GB | **recommended for non-English** — near-large quality at near-small speed |\n| `medium` | ~1.5 GB | conservative alternative to turbo |\n| `tiny` / `base` | 75–145 MB | quick drafts, CI |\n| `*.en` variants | — | English-only, slightly faster/better for EN |\n\nTips that work in every language: pass product names via\n`vocabulary=\"Term1, Term2\"` (biases the decoder so jargon survives), and note\nthat the workflow prompts instruct agents to write digests in the\n**narrator's language** while keeping quotes verbatim — the server never\ntranslates (exact quotes are evidence; translation is the agent's job).\n\nOn-screen text (OCR) defaults to RapidOCR's Latin + Chinese models. For other\nscripts set `TALKTHROUGH_OCR_LANG` to your language — `ru`/`uk` (→ the\n`eslav` pack), `ja`, `ko`, `ar`, `hi`, `el`, `th`, or any RapidOCR pack name\nlike `cyrillic` — and reprocess with `force=true`; the matching recognition\nmodel downloads once. Spoken-language support is unaffected either way.\n\n## Configuration\n\n| Env var | Default | Meaning |\n|---|---|---|\n| `TALKTHROUGH_WHISPER_MODEL` | `small` | default whisper model (`tiny`/`base`/`small`/`medium`/`large-v3`/`large-v3-turbo`); the `model` tool param overrides per call |\n| `TALKTHROUGH_OCR` | `on` | set `off` to skip OCR |\n| `TALKTHROUGH_OCR_LANG` | Latin+Chinese | recognition script for on-screen text: a language code (`ru`, `ja`, `ko`, `ar`, `hi`, …) or a RapidOCR pack name (`eslav`, `cyrillic`, `latin`, …); the model downloads once |\n| `TALKTHROUGH_OCR_PARAMS` | — | advanced: JSON object of raw RapidOCR params merged over the derived ones, e.g. `{\"Rec.lang_type\": \"cyrillic\"}` |\n| `TALKTHROUGH_DIARIZE` | `off` | set `on` to diarize by default (needs the `[diarization]` extra; degrades with a warning without it); an explicit `diarize` tool param always wins |\n| `TALKTHROUGH_DIARIZATION_THRESHOLD` | `0.5` | clustering sensitivity when `num_speakers` is unknown: fewer speakers than expected → lower it; more → raise it |\n| `TALKTHROUGH_DIARIZATION_SEG_MODEL` | `pyannote-segmentation-3-0` | segmentation model: allowlist name or a path to a local `.onnx` (offline preseed) |\n| `TALKTHROUGH_DIARIZATION_EMB_MODEL` | `nemo_en_titanet_small` | embedding model: allowlist name (see [Speakers](#speakers-optional-diarization)) or a local `.onnx` path |\n| `TALKTHROUGH_DIARIZATION_THREADS` | `min(4, cpus)` | ONNX threads for both diarization models |\n| `TALKTHROUGH_MAX_SECONDS` | `7200` | max media duration (also checked against provider metadata before a `process_url` download) |\n| `TALKTHROUGH_MAX_FRAMES` | `600` | keyframe budget per job, spread across the whole duration (the 1 s selection floor auto-grows to `duration/budget` on long recordings) |\n| `TALKTHROUGH_MAX_DOWNLOAD_BYTES` | `2147483648` (2 GiB) | hard cap for one `process_url` download, enforced before and during the transfer |\n| `TALKTHROUGH_HOME` | `~/.talkthrough` | job store root (URL jobs keep their downloaded source under `jobs//source/`) |\n\n## CLI\n\nThe pipeline is also a CLI — useful for pre-processing long recordings outside\nan agent session (the store is content-addressed, so the agent then queries the\nsame job instantly):\n\n```bash\ntalkthrough-mcp process ~/Videos/long-session.mov # prints the summary\ntalkthrough-mcp process demo.mov --json # machine-readable\ntalkthrough-mcp process sync.m4a --diarize --num-speakers 3 # who said what\ntalkthrough-mcp process-url \"https://youtu.be/nHfGfEiVdE8\" # one public URL, downloaded once\ntalkthrough-mcp gc --keep-days 30 # clean the job store (sources go with their jobs)\ntalkthrough-mcp serve # stdio MCP server (default)\ntalkthrough-mcp --version # package version + which extras this environment has\n```\n\n`--json` keeps stdout machine-readable on failure too (0.4.1), including a\nmissing argument or unknown option: the process exits with code 2, stderr\ncarries the human `error: …` line, and stdout\ncarries one JSON document, `{\"error\": {\"type\": \"UnsupportedUrlError\",\n\"message\": \"…\"}}`. `--version` also says which optional extras the\nenvironment has — the quickest check when a hand-written config launches\nthe minimal server (a launcher without the `url` extra) that advertises\n`process_url` but can only read direct media links; the server logs the\nsame line to stderr at every start.\n\nFirst run notes: missing system ffmpeg triggers a one-time `static-ffmpeg`\ndownload; the first transcription downloads the whisper model (~460 MB for\n`small`); both are cached. After that, expect roughly 3× faster than real time\non an Apple-Silicon CPU with the default model, OCR included (a 2-minute clip\nprocesses in ~40 s) — and instant re-runs on the same file. Progress streams\nas MCP progress notifications, and the CLI prints stage lines. More:\n[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md).\n\n## Windows\n\nCI runs lint, the unit suite, a full CLI smoke, and a diarize smoke on\n`windows-latest` (static-ffmpeg Windows build, whisper `tiny` transcription,\nOCR, the instant idempotent re-run, and a speaker-roster assert through the\nnative sherpa-onnx stack). Notes: the per-job lock always serializes threads;\nPOSIX also uses `fcntl` for cross-process locking. Quote paths with spaces\n(`uv run talkthrough-mcp process \"C:\\Videos\\Screen Recording.mp4\"`).\nIf something breaks, please open an issue.\n\n## Supported inputs\n\nVideo: `.mov` `.mp4` `.webm` `.mkv` `.ogv` — audio-only: `.m4a` `.mp3` `.wav` `.ogg`\n`.flac` (transcript tools only; frame tools explain why they're unavailable).\n\nURLs (via `process_url`, since 0.4.0): a direct `https://` link to one of\nthose media files; one public YouTube video (`watch`, `youtu.be`, `shorts`,\na completed live); or any public video **page** yt-dlp can read — public\nInstagram reels, TikTok, Wikimedia Commons file pages (each verified on\nrelease day), the rest of yt-dlp's ~1800 site extractors and pages with a\nplain HTML5/HLS player (the `[url]` extra, which the generated configs above\nalready carry, brings yt-dlp). Always anonymous: a site that demands a\nsign-in from anonymous clients (Vimeo does, with this yt-dlp) is refused\nwith the reason. The source is\ndownloaded once, kept inside the job, and never re-fetched for later\nquestions. Not supported: playlists, channels, active live streams,\nprivate, members-only, age-restricted or DRM-protected videos, cookies or\nlogins — a site that hides a video behind a login or a bot wall answers\nwith a clear refusal, not a workaround (Instagram in particular rate-limits\nanonymous access). You are responsible for having the right to download\nand process what you point it at; talkthrough does not bypass any\nrestriction.\n\n## Limitations\n\nHonest edges, so you can decide fast:\n\n- **Speaker labels are word-level and opt-in on new jobs.** Each Whisper word\n is assigned by maximum overlap with the diarization turns, so fast exchanges\n split without losing the raw label. Old jobs remain readable and honestly\n report `attribution_precision=\"segment\"`; reprocess with `force=true` to add\n word timings. Sub-second interjections (\"yeah\", \"mhm\") can still be absorbed\n when the diarization engine does not detect a separate turn, and heavy\n crosstalk degrades clustering (the segmentation model tracks at most 2\n simultaneous voices). Quality is\n pyannote-3.x-generation. The comfort zone without hints is roughly 2–8\n speakers; **pass `num_speakers` whenever the headcount is known** — it\n removes the worst failure mode at any size, and it is the way to go for\n large meetings (10+).\n- **URL ingestion covers one public video at a time, without logins.**\n Direct HTTPS media links, single public YouTube videos and public video\n pages yt-dlp can read; playlists, channels, live streams, gated or DRM\n content and anything behind a login or a bot wall are refused with a\n reason. Sites change; a page that worked yesterday can need a newer yt-dlp\n tomorrow. A provider's upload date is not a recording time, so URL jobs\n have `wall_clock: null` unless you pass `recorded_at`.\n- **Memory: budget about 2 GB for a cold run.** Whisper, the OCR models and\n the frame pass live in one process. A 78-second video on `tiny` with OCR\n peaked at 1.6 GB RSS during the 0.4.0 release QA (download included); the\n default `small` model needs more, larger models proportionally so. An\n 8 GB laptop copes; on anything tighter keep the model small or run the\n CLI ahead of the agent session.\n- **Keyframes + transcript, not motion analysis.** A glitch *between* scene\n changes can be invisible in the frame set; `extract_frame` re-checks any\n instant, but frame-by-frame motion reasoning is your multimodal model's job.\n- **STT quality tracks the model you pick.** The default `small` favors speed;\n non-English narration wants `model=\"large-v3-turbo\"` (see\n [Languages](#languages)).\n- **OCR reads crisp UI text well;** tiny or low-contrast print is best-effort.\n- **Wall-clock confidence depends on recorder metadata** — worst case pass\n `recorded_at=` (see the ladder above).\n- **Windows caveats** — POSIX lock degrades to a no-op; see the Windows\n section above.\n\n## How it compares\n\n| | talkthrough | cloud recorder SaaS | meeting notetakers | typical video-analyzer MCPs |\n|---|---|---|---|---|\n| Runs fully locally | ✅ | ❌ | ❌ | varies |\n| Any local video/audio file | ✅ | browser/app captures | meetings only | ✅ |\n| Public video URL: downloaded once, kept with the job, analyzed locally | ✅ | n/a | n/a | temp download, often cloud analysis |\n| Wall-clock anchoring (log correlation) | ✅ | ❌ | ❌ | ❌ |\n| Who-said-what speaker labels | ✅ local, opt-in | some | ✅ cloud | ❌ |\n| Ships agent workflows (prompts, skill, findings contract) | ✅ | ❌ | ❌ | ❌ |\n| OCR of on-screen text, searchable | ✅ | some | ❌ | rare |\n\n## FAQ\n\n**Why not just upload the video to a multimodal model (e.g. Gemini)?**\nFor a short, non-sensitive clip — do that. The trade-offs appear with length\nand sensitivity: an hour of screen recording costs on the order of a million\ntokens *per question*, the file leaves your machine, and you still can't map a\nremark to `14:32:07 UTC` to grep your server logs. talkthrough indexes once,\nlocally, then answers any number of follow-ups from the index.\n\n**Why not screenpipe?**\nDifferent job. screenpipe is an always-on recorder of *your* machine going\nforward (commercial license). It can't open the `.mov` a teammate or customer\njust sent you. talkthrough analyzes any file it's handed — the two compose\nfine.\n\n**There are agent skills that \"watch\" videos. Why a server with an index?**\nWatch-style skills push a budgeted frame dump into the context window (and go\nsparse on long videos), often call cloud STT for the audio, and keep nothing.\ntalkthrough builds a persistent local index — transcript + OCR, full-text\nsearchable — retrieves exact frames lazily, anchors everything to wall-clock\ntime, and answers the next question without reprocessing.\n\n**I use Jam for bug reports — do I need this?**\nKeep Jam for browser bugs: console+network captured at record time is great\nevidence. talkthrough covers what a browser extension can't — desktop apps,\nmobile screencasts, ops incidents, meetings, any file — with no account, and\ncorrelates with *server-side* logs via wall-clock time.\n\n**Which agent model do I need to drive this?**\nFor v0.4.0 the six model configs below ran 61 isolated behaviour cells on\nURL ingestion (YouTube, a TikTok page, a speechless Instagram reel,\nplaylist refusal, the missing wall clock) and the 0.3.2 integrity fixes;\nevery cell passes on every runner after two product fixes the first\nattempts exposed (Codex needs a per-tool approval for the open-world\n`process_url`, and the server now says itself when a URL job has no wall\nclock). For v0.3.0 we ran 210 isolated agent cells across 6 model configs (Claude\nhaiku/sonnet/opus, Codex gpt-5.5 at two reasoning efforts, and gpt-5.4-mini)\nand 35 logical scenarios on 5 real recordings plus safety and speaker-label\nfixtures. All 102 LLM-judged full-grid results and every mechanical zero were\nmanually audited; the 30 new speaker behavior runs passed, and old-server\ncontrol left **0 release-caused regressions**. This is a model-drift snapshot,\nnot a leaderboard: see the current matrix in\n[docs/MODEL-NOTES.md](docs/MODEL-NOTES.md). The chart and narrative in\n[benchmarks/](benchmarks/) remain the historical v0.2.0 snapshot.\n\n**Can't I just script ffmpeg + whisper myself?**\nYes — that's exactly this pipeline. What you'd be rebuilding: scene-change\ndetection with perceptual dedup, OCR, transcript+OCR search, the wall-clock\nladder, MCP tools with embedded usage examples, six workflow prompts, and a\nfindings contract. One `uvx` command instead of an afternoon of glue.\n\n**Is it really local? What leaves my machine?**\nNothing goes up, ever. For local files the network is used only for one-time\ndownloads (ffmpeg build, whisper/OCR/diarization models). `process_url` is\nthe single tool that talks to the network at runtime, and only *down*: it\nfetches the public source you named, once. No telemetry. See\n[Privacy](#privacy) — and [SECURITY.md](SECURITY.md) treats a violation of\nthis promise as a vulnerability.\n\n## For agents & tooling\n\nMachine-readable entry points, so AI agents can install and use this server\nwithout a human reading docs:\n\n- [`llms-install.md`](llms-install.md) — step-by-step install instructions for agents\n- [`llms.txt`](llms.txt) — index of the documentation\n- [`.agents/skills/talkthrough/SKILL.md`](.agents/skills/talkthrough/SKILL.md) — an [Agent Skill](https://agentskills.io) teaching the tool workflow; discovered automatically inside a checkout by Codex CLI (`$talkthrough`) and readable by Claude Code, Cursor, Copilot, Gemini CLI and other SKILL.md-compatible tools\n- [`AGENTS.md`](AGENTS.md) — instructions for coding agents contributing to this repo\n- [`server.json`](server.json) — MCP registry manifest\n- [`integrations/`](integrations/) — per-engine adapters, all generated from one source of truth and drift-tested (incl. the Claude Code plugin under [`integrations/claude-code/`](integrations/claude-code/))\n- [`docs/URL_ACCEPTANCE_CORPUS.md`](docs/URL_ACCEPTANCE_CORPUS.md) — the live URL corpus behind the release QA of `process_url` (manual, needs the network; CI stays offline)\n\n## Roadmap\n\ncloud STT · embeddings/semantic search · hosted/remote mode · `.mcpb`\nbundle · whisper.cpp backend\n\n## License\n\nMIT\n",
"bytes": 37359,
"sha": "e311668ba0c1ac72e9d87a1efcd0937a17d5ad43f966ee81c465d7dbaa6107ec",
"repo_slug": "korovin-aa97/talkthrough-mcp",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_korovin_aa97_talkthrough_mcp_3a3b494c/readme"
}