{
"markdown": "# claude-watch-video\n\n> **Give Claude eyes and ears for any video** — local files, public URLs (YouTube, Loom, Vimeo, TikTok…), or Jira attachments. Pipeline runs locally on your machine; produces a paste-ready Markdown evidence bundle.\n\n[](https://github.com/MarcinSufa/claude-watch-video/actions/workflows/smoketest.yml)\n[](LICENSE)\n[](https://www.python.org/)\n[](#prerequisites-any-path)\n[](https://docs.claude.com/en/docs/claude-code/plugins)\n\nTurn *\"watch PROJ-1234 and tell me what broke\"* into a single command. The skill downloads the video, extracts keyframes with ffmpeg, transcribes audio with local or hosted Whisper, deduplicates near-identical frames while preserving narrated moments, optionally OCRs on-screen text, and writes a paste-ready `report.md`. Costs **$0 to a few cents** per video. See [what it costs](#what-it-costs) for the breakdown.\n\n---\n\n## What it produces\n\nA 5:30 FOMC press conference distilled to one moment, picked by Claude against the prompt *\"summarize the rate decision, inflation outlook, and rate-path forecast\"*:\n\n> **00:22 — the rate decision**\n>\n> ![\"Powell](\"docs/images/fomc/00-22-rate-decision.jpg\")
\n>\n> *\"Today, the FOMC decided to leave our policy rate unchanged. We see the current stance of monetary policy as appropriate to promote progress toward our maximum employment...\"* Committee judges current policy stance appropriate for the dual mandate.\n\nThat's one card from a real `highlights.md` rendered inline. The pipeline also produces a full `report.md` evidence bundle (transcript + all timestamped frames), `report.html`, `report.docx`, and machine-readable `highlights.json`. **65 seconds end-to-end, $0** with local Whisper. See the [Powell FOMC walkthrough](docs/walkthrough-fomc.md) for the full artifacts, dedup metrics, and per-step timings.\n\n---\n\n## Install — pick one\n\n### Claude Code (recommended)\n\n```bash\n/plugin marketplace add MarcinSufa/claude-watch-video\n/plugin install watch-video@claude-watch-video\n```\n\nThen ask Claude: *\"Watch https://youtu.be/… and tell me what's in it.\"* or *\"Watch PROJ-1234 and identify the bug.\"*\n\nWorks on every Claude Code surface — CLI, IDE extensions, desktop. The plugin auto-installs the same underlying skill (see [`SKILL.md`](SKILL.md) for the manual install path if you prefer to clone into `~/.claude/skills/` directly).\n\n### CLI direct (CI / scripting / power users)\n\n```bash\ngit clone https://github.com/MarcinSufa/claude-watch-video.git\npython claude-watch-video/scripts/watch_video.py \"\" --workdir /tmp/test --dedup --verbose\n```\n\nZero-host, zero-MCP, fastest path on any platform. Produces `transcript.md`, `frames/`, `report.md` / `.html` / `.docx`. Good for CI pipelines, batch processing, or driving the analysis yourself.\n\n### Other MCP hosts (Claude Desktop, Cursor, Cline, Codex CLI, …)\n\nAn MCP server install is available — full instructions in [`mcp-server/README.md`](mcp-server/README.md).\n\n> ⚠️ **Honest warning:** on Windows + Claude Desktop the first run is **slow (~2-3 minutes)** because Windows Defender scans the freshly-spawned Python subprocess. The Claude Code plugin path above bypasses that entirely, and the CLI bypasses MCP entirely. **If you have the choice, use one of those.** The MCP path exists for environments where neither is an option.\n\n> **Codex CLI / Agent Skills note:** OpenAI's [openai/skills](https://github.com/openai/skills) framework presents skills as portable Agent Skills, but the install mechanism is still evolving. For Codex CLI today, **the CLI direct path above works without any agent-side install** (just `python scripts/watch_video.py ...` from your shell). If you specifically need MCP integration, the MCP path is available; if a future Codex skill-installer convention lands, this README will be updated.\n\n### Prerequisites (any path)\n\nRequired: **Python 3.10+** and **`ffmpeg`** (which bundles `ffprobe`).\n\nInstall ffmpeg per platform:\n\n| OS | Command |\n|---|---|\n| Windows | `winget install Gyan.FFmpeg` |\n| macOS | `brew install ffmpeg` |\n| Linux (Debian/Ubuntu) | `sudo apt install ffmpeg` |\n| Linux (Fedora/RHEL) | `sudo dnf install ffmpeg` |\n\nOptional (unlock features as you need them):\n\n| Adds | Install |\n|---|---|\n| URL mode (YouTube, Loom, …) | `pip install --user yt-dlp` |\n| Local transcription (`--whisper local`) | `pip install --user faster-whisper` |\n| Smart dedup (`--dedup`) | `pip install --user Pillow imagehash` |\n| OCR (`--ocr`) | `pip install --user pytesseract Pillow` + Tesseract binary ([SKILL.md → OCR setup](SKILL.md#on-screen-text-search-with---ocr)) |\n| Hosted Whisper (Groq/OpenAI/Deepgram) | API key — env var or `~/.watch-video/credentials.json` |\n| **Local diarization (`--whisper whisperx`)** | `pip install pyannote.audio` (the underlying lib; ~500 MB-1 GB of model weights cached on first run) + HF token (`HF_TOKEN` env or `hf_token` in credentials.json) + accept terms at **three** gated repos: [pyannote/speaker-diarization-3.1](https://huggingface.co/pyannote/speaker-diarization-3.1), [pyannote/segmentation-3.0](https://huggingface.co/pyannote/segmentation-3.0), and [pyannote/speaker-diarization-community-1](https://huggingface.co/pyannote/speaker-diarization-community-1) |\n| LLM highlights (`--highlights-prompt`) | Anthropic/OpenAI/Groq/DeepSeek/Gemini API key |\n| Jira auto-fetch + opt-in posting | Atlassian token at `~/.atlassian-token/credentials.json` |\n\n---\n\n## Pipeline\n\n```\n input\n │\n ▼\n fetch (URL → yt-dlp · path → as-is · Jira key → REST API)\n │\n ▼\n probe (ffprobe metadata + audio volume detection)\n / \\\n / \\\n frames transcribe (captions · local Whisper · Groq · OpenAI · Deepgram+diarization)\n \\ /\n \\ /\n dedup (perceptual hash + transcript-aware protection)\n │\n ▼\n ocr (Tesseract on kept frames; optional)\n │\n ▼\n report (report.md / .html / .docx)\n │\n ╎ (opt-in, never default)\n ▼\n Jira comment\n```\n\nEvery step is independently cached — re-run with a tweaked flag and only the affected tail of the pipeline executes (~0.25 s for a no-op re-run vs ~30 s cold).\n\n---\n\n## What it costs\n\n> ⚠️ The numbers below have caused confusion. The first table is what **this skill costs**. The second is what **alternatives charge** for the same job. They are not the same thing.\n\n### What this skill costs (per ~40-min video)\n\n| Setup | Cost | Time |\n|---|---|---|\n| **Pipeline only** (no agent reads the artifacts) | **$0** | ~15 s |\n| Pipeline + Claude Haiku 4.5 reads the artifacts | **~$0.04** | ~4-5 min |\n| Pipeline + Claude Sonnet 4.6 reads the artifacts | ~$0.16 | ~4-5 min |\n\nThe pipeline runs locally — no API calls for the download, frame extraction, transcription (with `--whisper local`, the default), dedup, OCR, or report generation. Video data never leaves your computer.\n\nThe only token cost is when an *agent* reads the generated artifacts. Hosted transcription (Groq/OpenAI/Deepgram) is optional and costs a fraction of a cent per minute.\n\n### What alternatives charge (for context)\n\nSame 40-min video, same output goal (transcript + visual context + structured analysis):\n\n| Alternative tool | Cost | Where the video goes |\n|---|---|---|\n| Gemini 3 Flash native video upload | ~$0.30 | Google |\n| Gemini 3 Pro native video upload | ~$0.80 | Google |\n| OpenAI Whisper + GPT-4o vision DIY | ~$4.50 | OpenAI |\n| Microsoft Video Indexer (advanced) | ~$8 | Microsoft |\n| Anthropic Claude Haiku raw video upload | ~$17 | Anthropic |\n| Anthropic Claude Sonnet raw video upload | ~$66 | Anthropic |\n| Anthropic Claude Opus raw video upload | ~$331 | Anthropic |\n\nThe Anthropic raw-upload column is the cleanest apples-to-apples on output quality (same Claude model reads the same content) — and the 30 fps frame tokenization makes that path uneconomic by ~400-8,000×. By preprocessing locally and feeding only the transcript + deduped frames into the agent, this skill replaces hundreds of dollars of frame tokens with a $0 pipeline + a few cents of structured text.\n\nFull per-tier breakdown + replication commands: [docs/cost-study-atlassian-video.md](docs/cost-study-atlassian-video.md).\n\n---\n\n## Features\n\n- **Multi-source input** — local file path, public URL (`yt-dlp` supports 1500+ sites), Jira issue key (`PROJ-1234`), or `auto` (newest video in `~/Downloads/`)\n- **Six transcription modes** — `captions` (free from YouTube VTTs), `local` faster-whisper (offline, default), `groq` Whisper-large-v3, `openai` Whisper-1, `deepgram` Nova-3 (hosted diarization, ~$0.0043/min), and `whisperx` (**free + offline** diarization via local Whisper + pyannote.audio directly; ~500 MB-1 GB of model weights cached on first run). Both diarization providers produce `**S0**` / `**S1**` paragraph labels + a `speakers.json` summary; relabel anonymous speakers with real names via `scripts/relabel_speakers.py`.\n- **Smart frame dedup** — pHash + temporal protection + transcript-aware keep rules. ~50% token reduction on screen recordings *without* losing the moment the user typed the wrong value\n- **OCR you can grep** — Tesseract on kept frames; `grep -i \"unload\" ocr.txt` answers \"when did the user enter 90?\" in milliseconds\n- **Per-step cache** — re-run with a tweaked flag and only the affected tail of the pipeline executes\n- **Paste-ready evidence bundle** — `report.md` interleaves transcript paragraphs with frame thumbnails; drop into Jira / PR / design-review doc as-is\n- **Bulk mode** — process a sprint's worth of bug tickets in one command\n- **Opt-in Jira posting** — explicit `--post-to-jira` + confirmation prompt; never default, never silent. See [safety model](#safety-model)\n- **LLM highlights** — six providers (Anthropic / OpenAI / Groq / DeepSeek / Gemini / generic openai-compat) for picking the moments that match your prompt\n\n---\n\n## End-to-end walkthroughs\n\nTwo real runs with all artifacts and timings captured verbatim — separate docs so they don't bloat this page:\n\n- 📊 **[Walkthrough — Powell's FOMC statement](docs/walkthrough-fomc.md)** — 5:30 Federal Reserve press conference distilled to 5 quantitative highlights. **/usr/bin/bash** (local Whisper), **65s** end-to-end. Demonstrates: continuous-narration source, zero-cost transcription, structured macro analysis.\n- 🆕 **[Walkthrough — Claude Code release-notes](docs/walkthrough-claude-code-release.md)** — 54s product release video reduced to 5 actionable workflow changes. **$0**, **29s** end-to-end, **44%** dedup reduction. Demonstrates: fast-cut B-roll, screen-recording context, agent-readable timestamps.\n\n---\n\n## Use cases\n\n- **Bug triage from Jira screen recordings** — `python scripts/watch_video.py PROJ-1234 --dedup --ocr` gives you the transcript + key frames + on-screen text in under a minute. Add `--post-to-jira` to attach the analysis back to the ticket\n- **Sprint retro on bug videos** — `python scripts/watch_batch.py --jira-jql \"project = PROJ AND labels = video-bug AND created >= -7d\"` processes a week's videos at once\n- **Researching public content** — point at any YouTube/Loom URL; captions-first means free + sub-5-second on most YouTube videos\n- **Podcast / multi-speaker transcription** — `--whisper deepgram` gives anonymous speaker labels (`S0`, `S1`) and a `speakers.json` summary you can later relabel with real names\n- **Compliance / privacy-first** — default `--whisper local` runs entirely on your machine; nothing is uploaded\n- **CI integration** — the CLI is pipeline-friendly: shell out from a GitHub Actions / GitLab CI / Jenkins step on test-failure videos, parse `meta.json` for the result. Per-step caching means re-runs after a flag tweak are near-instant.\n\n---\n\n## Examples\n\n```bash\n# Most common: full-auto Jira workflow\npython scripts/watch_video.py PROJ-1234 --dedup --ocr\n\n# YouTube clip with bumped resolution for tiny on-screen text\npython scripts/watch_video.py \"https://youtu.be/abc\" --resolution 1280\n\n# Scope to a 10-second window of a long video\npython scripts/watch_video.py PROJ-1234 --start 0:30 --end 0:40\n\n# Fast cold-start with hosted Whisper\npython scripts/watch_video.py PROJ-1234 --whisper groq\n\n# Multi-speaker podcast with hosted diarization (Deepgram; ~$0.0043/min)\npython scripts/watch_video.py \"https://youtu.be/joe-vs-naval\" --whisper deepgram\n\n# Same, but local + offline (WhisperX recipe; free; ~1 GB first-run model download)\n# Implemented via faster-whisper + pyannote.audio directly (no whisperx package\n# wrapper, which has stale dep pins broken on Python 3.14). Requires HF token\n# + accepting terms on three gated pyannote models. See SKILL.md.\npython scripts/watch_video.py \"https://youtu.be/joe-vs-naval\" --whisper whisperx\n\n# Relabel anonymous speakers with real names (v2.3.1+; works for both providers)\n# Read speakers.json first to see who said what; then:\npython scripts/relabel_speakers.py /tmp/watch-joe-vs-naval \\\n --names \"S0=Joe Rogan,S1=Naval Ravikant\"\n# transcript.md and report.md/.html/.docx are atomically rewritten in place.\n\n# LLM-driven highlight selection (default model is Claude Haiku 4.5)\npython scripts/watch_video.py PROJ-1234 \\\n --highlights-prompt \"highlight only bug-related parts\"\n\n# Post analysis back to the ticket (opt-in, confirmation prompt)\npython scripts/watch_video.py PROJ-1234 --dedup --ocr --post-to-jira\n\n# Bulk: process a sprint's worth of bug tickets\npython scripts/watch_batch.py \\\n --jira-jql \"project = PROJ AND labels = video-bug AND created >= -7d\" \\\n --dedup --ocr\n```\n\nEvery flag is documented in [docs/configuration.md](docs/configuration.md).\n\n---\n\n## Safety model\n\nThe skill ships with safety guardrails for the one action that leaves the machine — posting to Jira:\n\n- **`--post-to-jira` is opt-in.** Default behavior never writes anything anywhere.\n- **Interactive confirmation prompt.** When `--post-to-jira` is set, the CLI prints the planned comment and asks for `y/N` before sending. The default is `N`.\n- **`--post-to-jira-dry-run`** previews the comment without sending — for sanity-checking the formatting in CI or before a real run.\n- **Local writes are confined.** Pipeline artifacts go to the workdir, and the MCP `fetch_jira_attachment` tool only accepts an `outdir` under the tmp root — a ticket cannot steer downloads at your home directory. Attachment filenames are flattened to a single safe path component before use.\n- **No unsolicited Jira writes from any context.** Skill / plugin / MCP / direct API: all paths require explicit per-invocation consent. The MCP `post_to_jira` tool defaults to `confirm=False` (dry-run); MCP hosts MUST surface the planned action to the user before passing `confirm=True`.\n\n---\n\n## Architecture\n\nCLI scripts are the canonical implementation. Everything else (plugin, skill, MCP server) is a thin adapter that calls these scripts under the hood.\n\n```\nscripts/ ← canonical implementation\n├── watch_video.py ← top-level orchestrator\n├── fetch.py ← URL / file / Jira input\n├── probe.py ← ffprobe metadata + audio analysis\n├── frames.py ← ffmpeg keyframe extraction\n├── transcribe.py ← captions / local / Groq / OpenAI / Deepgram\n├── dedup.py ← pHash + transcript-aware protection\n├── ocr.py ← Tesseract on kept frames\n├── report.py ← report.md / .html / .docx generation\n├── highlights.py ← LLM-driven moment picker (6 providers)\n└── post_to_jira.py ← opt-in, confirmation-gated Jira posting\n\nmcp-server/server.py ← thin async wrapper over the CLI scripts\n```\n\nEach step writes to disk atomically (staged then `os.replace`'d). Per-step fingerprint cache; re-run with a tweaked flag and only the affected tail executes. Workdir is a contract — every artifact has a known filename and shape ([SKILL.md → file layout](SKILL.md)).\n\n---\n\n## Versioning + Roadmap\n\nLatest release: **v2.3.2** ([changelog](https://github.com/MarcinSufa/claude-watch-video/releases)) — adds `--whisper whisperx`, a free + offline local-diarization alternative to Deepgram. Same `speakers.json` schema, so `relabel_speakers.py` works against it unchanged.\n\nWhat's queued:\n- **v2.4.0+** — OCR cross-correlation for screen-recording name overlays (auto-label speakers from Zoom/Teams/Meet name tags, eliminating the manual relabel step entirely on those sources)\n\nFull roadmap: [ROADMAP.md](ROADMAP.md).\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE). Free for any use, commercial OK, just keep the copyright notice.\n\n## Acknowledgements\n\nBuilt on [ffmpeg](https://ffmpeg.org/), [yt-dlp](https://github.com/yt-dlp/yt-dlp), [faster-whisper](https://github.com/SYSTRAN/faster-whisper), [Tesseract](https://github.com/tesseract-ocr/tesseract), [Pillow](https://pillow.readthedocs.io/), [imagehash](https://github.com/JohannesBuchner/imagehash), [python-docx](https://github.com/python-openxml/python-docx), and the [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python). LLM highlight picking optionally uses [OpenAI](https://github.com/openai/openai-python), [Groq](https://groq.com/), [DeepSeek](https://api.deepseek.com), or [Google Gemini](https://ai.google.dev/) via their OpenAI-compatibility endpoints. Speaker diarization via [Deepgram Nova-3](https://deepgram.com).\n",
"bytes": 17884,
"sha": "25ffca208bf9809be78d04995d4d6b36c2d3b01606b8eb795321454eb547015a",
"repo_slug": "marcinsufa/claude-watch-video",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/plg_marcinsufa_claude_watch_video_watch_vide_b7c23fb8/readme"
}