{ "markdown": "# talkthru\n\n**[talkthru.dev](https://talkthru.dev)** · [npm](https://www.npmjs.com/package/talkthru) · [@dr_edon](https://x.com/dr_edon) · MIT\n\n**An automated feedback loop for any app.** Screen-record your app on your phone and talk\nwhile you use it. Send it to your machine and your coding agent gets everything you\nsaid, attached to the screen you were looking at when you said it.\n\n```\nyou: *tapping through checkout* \"this button is too small, I keep missing it\"\n \"and this error doesn't say what went wrong\"\n\nagent: sees both screens, finds both components, fixes them.\n```\n\nRuns entirely on your machine. No accounts, no uploads, no API keys.\n\nWorks with a mac screen recording (⌘⇧5), an iPhone one over AirDrop, or Windows Game Bar. Any video with sound works — point the watcher at a folder.\n\n[![Watch the 70-second demo](https://img.youtube.com/vi/3BimSScnGOg/maxresdefault.jpg)](https://youtu.be/3BimSScnGOg)\n\n*70-second demo — install, record, and the agent picking it up.*\n\n\n---\n\n## Install\n\n```bash\nnpx talkthru doctor --fix\n```\n\nPulls `ffmpeg`, `whisper.cpp` and the speech model. Nothing else to set up.\n\n## Start the watcher\n\n```bash\nnpx talkthru watch\n```\n\nLeave it running. It only touches files named like a screen recording — the rest of your\nDownloads folder is invisible to it.\n\n## Connect your agent\n\ntalkthru is an MCP server, so any MCP client works — Cursor, Windsurf, Cline, Zed,\nClaude Desktop. Add it to your client's config:\n\n```json\n{\n \"mcpServers\": {\n \"talkthru\": {\n \"command\": \"npx\",\n \"args\": [\"talkthru-mcp\"]\n }\n }\n}\n```\n\nClaude Code has a one-liner for the same thing:\n\n```bash\nclaude mcp add --scope user talkthru -- npx talkthru-mcp\n```\n\nRestart your client afterwards so it picks up the server.\n\nBuilt and tested against Claude Code. Other MCP clients should work — tell me if yours doesn't.\n\n---\n\n## Set up your mac\n\nOnce. Press **⌘⇧5**, pick a **record** mode (the icons with the ◉ dot), then open\n**Options** and set **Save to → Downloads** and your **Microphone**. Both stick.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\"Screenshot
1 · 2. press ⌘⇧5, pick a record mode, open Options
\"Options\"Options
3. Save to → Downloads — the folder talkthru watch looks at4. pick your mic under Microphone
\"Screenshot
5. hit Record and talk as you use your app
\n\n**No mic selected means a silent video** and nothing to transcribe. Worth checking before\nyour first recording.\n\n---\n\n## Set up your phone\n\nOptional — only if you want to record a phone app rather than your desktop.\n\nOnce. Control Centre → **+** → **Add a Control** → **Screen Recording**. Then press and\nhold it and turn the **microphone on**.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
\"Tap\"Tap\"Pick
1. tap +2. Add a Control3. pick Screen Recording
\"Press\"Microphone\"Recording
4. press and hold ◉5. Microphone On6. tap ◉ and talk
\n\n---\n\n## Recording a web app\n\nOptional — a drop-in script that hands the agent the element you were pointing at, not\njust the picture of it.\n\n```html\n\n```\n\nRecord, then `talkthru.stop()` in the console downloads `hierarchy.json`. Process the two\ntogether:\n\n```bash\ntalkthru process ~/Downloads/screen-recording.mov --hierarchy ~/Downloads/hierarchy.json\n```\n\nThe script and what it captures: [`clients/web`](clients/web).\n\n---\n\n## Use it\n\n1. Record your app, talk as you go\n2. Get the video onto your machine (AirDrop is easiest)\n3. Ask your agent: **\"check the feedback in my last talkthru session and implement it\"**\n\nReady in about twenty seconds. Your original video is kept in `~/.talkthru/archive/`.\n\n**Mute your app if it talks.** Its narration lands in the transcript next to yours and\nwhisper can't tell you apart.\n\n---\n\n## Output\n\nYour words, attached to the screen that was up when you said them:\n\n```markdown\n## 00:34 · f11 — `frames/f11.jpg`\n- [00:39] \"this button is too small, I keep missing it\"\n\n## 00:52 · f13 — `frames/f13.jpg`\n- [00:55] \"and this error doesn't say what actually went wrong\"\n```\n\nPlus the frames. A two-minute session is about 600 tokens.\n\n---\n\n## Element context (optional)\n\nPoint talkthru at a UI hierarchy file and your agent gets element types, labels\nand **stable test ids** instead of guessing which component you meant from\npixels:\n\n```bash\ntalkthru process recording.mp4 --hierarchy hierarchy.json\n```\n\n```markdown\n## 00:34 · f11 — `frames/f11.jpg`\nui: button#checkout-submit \"Place order\" @24,680 342x48\n- [00:39] \"this button is too small, I keep missing it\"\n```\n\nIt is a timestamped sidecar your app writes — nothing here generates it, and the\nformat is plain JSON with no iOS-specific assumption:\n\n```json\n{ \"snapshots\": [\n { \"tMs\": 1200, \"nodes\": [\n { \"type\": \"button\", \"testId\": \"checkout-submit\", \"label\": \"Place order\",\n \"rect\": [24, 680, 342, 48], \"depth\": 4 }\n ] }\n] }\n```\n\nCommon alternate spellings (`tag`, `id`, `text`, `bounds`, `elements`, …) are\naccepted as-is, so a DOM client can usually emit its natural shape. Malformed\nfiles are ignored with a warning rather than failing the recording.\n\n**Full format: [docs/hierarchy.md](https://github.com/EdonZo/talkthru/blob/main/docs/hierarchy.md)** — fields, aliases, ranking rules, and how to write a client.\n\n---\n\n## Network and console (optional)\n\nDrop an `events.json` next to your recording and failed requests land under the\nframe you were looking at:\n\n```markdown\n## 00:34 · f11 — `frames/f11.jpg`\nui: button#checkout-submit \"Place order\" @24,680 342x48\nnet: POST https://api.example.com/checkout 500 340ms\n- [00:39] \"I tapped place order and nothing happened\"\n```\n\n```json\n{ \"events\": [\n { \"tMs\": 12400, \"kind\": \"network\", \"method\": \"POST\",\n \"url\": \"https://api.example.com/checkout\", \"status\": 500, \"durationMs\": 340 },\n { \"tMs\": 12450, \"kind\": \"console\", \"level\": \"error\", \"text\": \"TypeError: ...\" }\n] }\n```\n\n**No headers, no bodies** — auth tokens live in headers and the point of this\ntool is that your data stays on your machine. Secret-looking query values are\nreplaced with `REDACTED` at parse time. Everything you send is stored; only\nfailures and `error`/`warn` reach the agent by default, because two minutes of\nXHR is more text than the frames and narration combined.\n\n**Full format: [docs/events.md](https://github.com/EdonZo/talkthru/blob/main/docs/events.md)**\n\n---\n\n## Windows\n\n`Win+Alt+R` (Game Bar) records to `Videos\\Captures`, which is where the watcher\nlooks by default — no name filter there, because that folder holds nothing else:\n\n```powershell\nwinget install -e --id Gyan.FFmpeg\nnpx talkthru watch\n```\n\nwhisper.cpp has no winget package. Build it from\n[ggml-org/whisper.cpp](https://github.com/ggml-org/whisper.cpp) and point\n`TALKTHRU_WHISPER` at the binary.\n\nWritten and unit-tested for Windows, but **not yet run on a Windows machine** —\nI do not have one. If you try it, an issue either way is useful.\n\n---\n\n## Commands\n\n```bash\ntalkthru watch # watch ~/Downloads for screen recordings\ntalkthru watch ~/some-folder # watch your own folder, any video\ntalkthru process video.mp4 # process one file\ntalkthru list # recent sessions\ntalkthru show latest # print one\ntalkthru doctor # check the setup\ntalkthru compact # reclaim disk: drops old originals, keeps every frame + transcript\ntalkthru prune # delete old sessions entirely\n```\n\n`tt` also works.\n\n---\n\n## How it works\n\n`ffmpeg` pulls the frames where your screen actually changed. `whisper.cpp` transcribes\nyour voice locally, split on the real silence between sentences, so each thing you said\nlands on the screen you were looking at. An MCP server hands it to your agent.\n\nNeeds Node 20.11+.\n\n---\n\n## Why\n\nI built this because I test my own app on my phone every day and kept losing the feedback\non the way back to my editor — screenshotting, cropping, trying to describe which screen I\nmeant. Now I just record, talk, and Claude Code picks it up. Sharing it in case it saves\nyou the same trip.\n", "bytes": 9513, "sha": "13364ec0658f3883da8432407fa982ae3a4f8ba55a66f772a60d49e6fca16c01", "repo_slug": "edonzo/talkthru", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/mcp_io_github_edonzo_talkthru_ba20bf15/readme" }