{
  "markdown": "# /watch\n\n**Give Claude the ability to watch any video.**\n\nClaude Code (recommended — auto-updates via marketplace):\n```\n/plugin marketplace add bradautomates/claude-video\n/plugin install watch@claude-video\n```\n\nCodex, Cursor, Copilot, Gemini CLI, or any of 50+ [Agent Skills](https://agentskills.io) hosts:\n```bash\nnpx skills add bradautomates/claude-video -g\n```\n(`-g` installs globally for your user, available across all projects. Drop it to scope per-project.)\n\nMore install options (claude.ai web, manual) in the [Install](#install) section below.\n\nZero config to start — `yt-dlp` and `ffmpeg` install on first run via `brew` on macOS (Linux/Windows print exact commands). Captions cover most public videos for free. Whisper API key is only needed when a video has no captions.\n\n---\n\nClaude can read a webpage, run a script, browse a repo. What it can't do, out of the box, is *watch a video*. You paste a YouTube link and it has to either guess from the title or pull a transcript that's missing 90% of what's on screen.\n\nWith Claude Video `/watch` you can paste a URL or a local path, ask a question, and Claude fetches captions first, downloads only what it needs, extracts frames (scene-aware, or fast keyframes at `efficient` detail), pulls a timestamped transcript (free captions when available, Whisper API as fallback), and `Read`s every frame as an image. By the time it answers, it has *seen* the video and *heard* the audio.\n\n```\n/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?\n```\n\n## What people actually use it for\n\n**Analyze someone else's content.** `/watch https://youtu.be/<viral-video> what hook did they open with?` Claude looks at the first frames, reads the opening transcript, breaks down the structure. Same for ad creative, competitor launches, podcast intros, anything where the *how* matters as much as the *what*.\n\n**Diagnose a bug from a video.** Someone sends you a screen recording of something broken. `/watch bug-repro.mov what's going wrong?` Claude watches the recording, finds the frame where the issue appears, describes what's on screen, often catches the cause without you ever opening the file.\n\n**Summarize a video.** `/watch https://youtu.be/<long-thing> summarize this` does the obvious thing — pulls the structure, the key moments, what was actually said and shown. Faster than watching at 2x.\n\n**Cut the hype out of an update video.** `/watch https://youtu.be/<launch-video> what's actually new — skip the hype` Strip a \"game-changer\" feature drop down to the few things that matter, so you get the substance without ten minutes of intro and overselling.\n\n**Turn a playlist into notes.** `/watch https://youtu.be/<video> summarize this to a note` Run it across a series and file a per-video summary, so a channel or course becomes a searchable set of notes instead of hours you have to sit through.\n\n## How it works\n\n1. **You paste a video and a question.** URL (anything yt-dlp supports — YouTube, Loom, TikTok, X, Instagram, plus a few hundred more) or a local path (`.mp4`, `.mov`, `.mkv`, `.webm`).\n2. **`yt-dlp` checks captions first.** At `transcript` detail, captioned URLs return without downloading video. Otherwise, or when Whisper needs audio, it downloads only what the run needs.\n3. **`ffmpeg` extracts frames at the chosen detail.** `efficient` decodes keyframes only (near-instant); `balanced`/`token-burner` prefer scene-change frames and fall back to the duration-aware uniform sampler when they under-produce. JPEGs are 512px wide by default and clamped to 1998px tall for Claude Read compatibility.\n4. **The transcript comes from one of two places.** First try: `yt-dlp` pulls native captions (manual or auto-generated) from the source. Free, instant, accurate-ish. Fallback: extract a mono 16 kHz 64 kbps mp3 audio clip (~480 kB/min) and ship it to Whisper — Groq's `whisper-large-v3` (preferred — cheaper and faster) or OpenAI's `whisper-1`.\n5. **Frames + transcript are handed to Claude.** The script prints frame paths with `t=MM:SS` markers and the transcript with timestamps. Claude `Read`s each frame in parallel — JPEGs render directly as images in its context.\n6. **Claude answers grounded in what's actually on screen and in the audio.** Not \"based on the description\" or \"according to the title.\" It saw the frames. It heard the transcript. It answers the way someone who watched the video would.\n7. **Cleanup.** The script prints a working directory at the end. If you're not asking follow-ups, Claude removes it.\n\n## Frame budget — why it matters\n\nToken cost is dominated by frames. Every frame is an image; image tokens add up fast. The script's auto-fps logic exists so you don't blow your context budget on a sparse scan of a 30-minute video that would have been better answered by a focused 30-second window.\n\n| Duration | Default frame budget | What you get |\n|----------|---------------------|--------------|\n| ≤30 s | ~30 frames | Dense — basically every key moment |\n| 30 s - 1 min | ~40 frames | Still dense |\n| 1 - 3 min | ~60 frames | Comfortable |\n| 3 - 10 min | ~80 frames | Sparse but workable |\n| > 10 min | 100 frames (capped modes) | \"Sparse scan\" warning — re-run focused, or `--detail token-burner` for full uncapped coverage |\n\nWhen the user names a moment (\"around 2:30\", \"the last 30 seconds\", \"from 0:45 to 1:00\"), pass `--start` / `--end`. Focused mode gets denser per-second budgets, capped at 2 fps. Far more useful than a sparse pass over the whole thing.\n\n## Frame deduplication\n\nFrame selection — keyframes (`efficient`), scene-change detection (`balanced`/`token-burner`), or the uniform sampler it falls back to — can still surface near-identical frames: a screen recording that holds one slide for 90 seconds produces a dozen, each billed as a separate image. A dedup pass drops them before frames reach Claude. It runs by default on every frame mode (`--no-dedup` turns it off):\n\n1. One `ffmpeg` call scales each extracted JPEG to a 16×16 grayscale thumbnail. Everything after is pure-stdlib Python — no image libraries.\n2. For each frame, compute the **mean absolute difference** against the *last frame that was kept* (average per-pixel brightness change, 0–255 scale).\n3. If that difference is at or below the threshold (`2.0`), the frame is a near-duplicate and is dropped. Otherwise it's kept and becomes the new reference.\n4. The frame-budget cap applies *after* dedup, so the budget is spent on distinct frames.\n\nComparing against the last *kept* frame (not the previous one) catches slow fades that never trip a frame-to-frame threshold. The threshold is deliberately low and measures absolute brightness rather than structure, so a one-line code diff, a terminal scrolling a row, or two differently-colored flat slides all survive.\n\nThe **Frames** line reports what was collapsed, e.g. `6 selected from 14 candidates (… 8 near-duplicates dropped …)`. On always-moving footage nothing is dropped and you pay what you would have anyway.\n\n## Detail modes — measured\n\nThe `--detail` dial trades speed and token cost for visual fidelity. Numbers below are from a real run against a **49:08** YouTube video (1280×720, English auto-captions) — a long, mostly-static screen recording, the case that stresses the caps hardest. Extraction times are local CPU against a pre-downloaded copy; the one-time download was **~37 s** / 76 MB, shared by the three frame modes.\n\n| Mode | Engine | Frames | Cap | Extraction time | Temporal coverage | Est. image tokens |\n|------|--------|--------|-----|-----------------|-------------------|-------------------|\n| `transcript` | none (captions) | 0 | — | **~4.5 s** (one yt-dlp call, no download) | full (text) | 0 (≈26.6k text tokens) |\n| `efficient` | keyframe (`-skip_frame nokey`) | 50 | 50 | **~0.5 s** | 0:00 → 49:04 (full) | **~9.8k** |\n| `balanced` | scene-change | 100 | 100 | **~20.9 s** | 0:00 → 48:38 (full) | **~19.7k** |\n| `token-burner` | scene-change | 116 | uncapped | **~21.0 s** | 0:00 → 48:38 (full) | **~22.8k** |\n\n- **Image tokens** use Anthropic's `(width × height) / 750` — at the default 512px width these 720p frames are 512×288, **≈197 tokens/frame**; `--resolution 1024` roughly 4×s that. The transcript is surfaced in every captioned mode and on long videos is often the larger cost.\n- **One sampling rule across frame modes.** Each detects all candidates across the full range, then even-samples (first + last always kept) down to its cap. The modes differ only in candidate *source* (keyframes vs. scene cuts) and cap, never in how coverage is spread — so the last frame always lands at the end, not partway through.\n- **`efficient` is the speed tier** (~0.5 s) — it only reconstructs keyframes, so it's ~40× faster than the scene modes, which decode every frame to find cuts. It can also return *more* frames than `balanced` on low-motion footage (keyframes outnumber scene cuts); \"efficient\" means fast extraction, not fewer frames.\n- **`token-burner` only diverges from `balanced` past the cap.** This clip had 116 cuts, so `balanced` sampled 100 and `token-burner` kept all 116. On high-motion video with hundreds of cuts, `token-burner` keeps everything (and trips the >250-frame token warning) while `balanced` thins to 100.\n\nEnd-to-end from a cold URL, `transcript` is the cheapest mode by far; the frame modes add the shared ~37 s download on top of the extraction times above.\n\n## Install\n\n| Surface | Install |\n|---------|---------|\n| **Claude Code** | `/plugin marketplace add bradautomates/claude-video` then `/plugin install watch@claude-video` |\n| **Codex, Cursor, Copilot, Gemini CLI, +50 more** | `npx skills add bradautomates/claude-video -g` |\n| **claude.ai** (web) | [Download `watch.skill`](https://github.com/bradautomates/claude-video/releases/latest) → Settings → Capabilities → Skills → `+` |\n| **Manual / dev** | `git clone` then symlink `skills/watch` into your host's skills dir (see below) |\n\n### Claude Code\n\n```\n/plugin marketplace add bradautomates/claude-video\n/plugin install watch@claude-video\n```\n\nUpdate later with `/plugin update watch@claude-video`.\n\n### Codex, Cursor, Copilot, Gemini CLI, and 50+ other hosts\n\nThe [Agent Skills](https://agentskills.io) CLI installs the skill into whatever agents it detects:\n\n```bash\nnpx skills add bradautomates/claude-video -g\n```\n\n`-g` installs globally for your user (`~/.codex/skills`, `~/.cursor/skills`, etc.); drop it to install into the current project instead. Useful flags:\n\n- `-a, --agent <names…>` — target specific hosts, e.g. `-a codex -a cursor`\n- `-l, --list` — list the skills in this repo without installing\n- `--copy` — copy files instead of symlinking (for filesystems without symlink support)\n\nThe CLI discovers the skill from `skills/watch/SKILL.md` and copies the whole folder — `SKILL.md` plus its `scripts/` runtime — as a self-contained unit. `SKILL.md` resolves its own scripts relative to wherever it was installed, so it works the same on every host.\n\nUpdate later with `npx skills update watch -g`.\n\n### claude.ai (web)\n\n1. [Download `watch.skill`](https://github.com/bradautomates/claude-video/releases/latest) from the latest release.\n2. Go to Settings → Capabilities → Skills.\n3. Click `+` and drop the file in.\n\nEnable \"Code execution and file creation\" under Capabilities first — the skill shells out to `ffmpeg` and `yt-dlp`, so it won't run without it.\n\n### Manual (developer)\n\nClone the repo and symlink the self-contained skill folder into your host's skills directory — the symlink keeps the install in sync with your working tree as you edit:\n\n```bash\ngit clone https://github.com/bradautomates/claude-video.git\nln -s \"$(pwd)/claude-video/skills/watch\" ~/.claude/skills/watch   # or ~/.codex/skills/watch\n```\n\nFor claude.ai, build the `.skill` bundle from source: `bash skills/watch/scripts/build-skill.sh` produces `dist/watch.skill`.\n\n## First run\n\nOn the first `/watch` call, the skill runs `scripts/setup.py --check`. If `ffmpeg` / `yt-dlp` aren't on your PATH, or no Whisper API key is set, it walks you through fixing it:\n\n- **macOS** — auto-runs `brew install ffmpeg yt-dlp`.\n- **Linux** — prints the exact `apt` / `dnf` / `pipx` commands.\n- **Windows** — prints the `winget` / `pip` commands.\n- **API key** — scaffolds `~/.config/watch/.env` (mode `0600`) with commented placeholders for `GROQ_API_KEY` (preferred) and `OPENAI_API_KEY`.\n\nAfter setup, preflight is silent and `/watch` just works. The check is a sub-100ms lookup, so it doesn't slow you down on subsequent runs.\n\n## Bring your own keys\n\nCaptions cover the majority of public videos for free. The Whisper fallback only kicks in when a video genuinely has no caption track — typically local files, TikToks, some Vimeos, and the occasional caption-less YouTube upload.\n\n| Capability | What you need | Cost |\n|------------|---------------|------|\n| Download + native captions | `yt-dlp` + `ffmpeg` | Free |\n| Whisper fallback (preferred) | [Groq API key](https://console.groq.com/keys) — `whisper-large-v3` | Cheap, fast |\n| Whisper fallback (alt) | [OpenAI API key](https://platform.openai.com/api-keys) — `whisper-1` | Standard pricing |\n| Disable Whisper entirely | `--no-whisper` | Free, frames-only when no captions |\n\n## Usage\n\n```\n/watch https://youtu.be/dQw4w9WgXcQ what happens at the 30 second mark?\n/watch https://www.tiktok.com/@user/video/123 summarize this\n/watch ~/Movies/screen-recording.mp4 when does the UI break?\n/watch https://vimeo.com/123 what tools does she mention?\n```\n\nFocused on a specific section — denser frame budget, lower token cost:\n```\n/watch https://youtu.be/abc --start 2:15 --end 2:45\n/watch video.mp4 --start 50 --end 60\n/watch \"$URL\" --start 1:12:00            # from 1h12m to end\n```\n\nOther knobs (passed to `scripts/watch.py`):\n\n- `--detail transcript|efficient|balanced|token-burner` — fidelity/speed dial. `transcript` skips frames (transcript only); `efficient` uses fast keyframes (cap 50); `balanced` uses scene-aware frames (cap 100); `token-burner` is scene-aware and uncapped.\n- `--timestamps T1,T2,…` — grab a frame at each absolute timestamp (`SS`/`MM:SS`/`HH:MM:SS`). Claude reads the transcript first, then targets the moments the presenter flags (\"look here\", \"as you can see\"). Added on top of the detail frames (reserved against the cap); out-of-window cues are dropped in focus mode; with `--detail transcript` these become the only frames.\n- `--max-frames N` — lower the frame cap for a tighter token budget.\n- `--resolution W` — bump frame width to 1024 px when Claude needs to read on-screen text (slides, terminals, code).\n- `--fps F` — override the auto-fps calculation (still capped at 2 fps).\n- `--whisper groq|openai` — force a specific Whisper backend.\n- `--no-whisper` — disable transcription entirely; frames only.\n- `--no-dedup` — keep near-duplicate frames. By default a frame-delta pass drops frames that are visually near-identical to the one before them (held slides, static screen recordings, paused video), so the frame budget is spent on distinct content; this flag turns that off.\n- `--out-dir DIR` — keep working files somewhere specific (default: auto-generated tmp dir).\n\n## Limits\n\n- **Long-video accuracy depends on the detail mode.** On the capped modes (`efficient`, default `balanced`) coverage thins out past ~10 minutes — the frame cap spreads across the whole clip, so the script prints a \"sparse scan\" warning and you're better off re-running focused with `--start`/`--end`. `token-burner` lifts the cap and keeps *every* scene-change frame across the full video, so it stays complete on longer clips at the cost of more image tokens. The 10-minute mark is guidance for the capped modes, not a hard ceiling.\n- **Detail is one dial.** Defaults are balanced: scene-aware frames, 2 fps max, 100-frame cap. Use `--detail efficient` for a fast 50-frame keyframe pass, or `--detail token-burner` for uncapped scene candidates. Set `WATCH_DETAIL` in `~/.config/watch/.env` to change the default.\n\n## Structure\n\n```\n.\n├── skills/watch/                 # self-contained skill — copied as a unit by every installer\n│   ├── SKILL.md                  # skill contract — the source of truth across all surfaces\n│   └── scripts/\n│       ├── watch.py              # entry point — orchestrates download → frames → transcript\n│       ├── download.py           # yt-dlp wrapper\n│       ├── frames.py             # ffmpeg frame extraction + auto-fps logic\n│       ├── transcribe.py         # VTT parsing + dedupe + Whisper orchestration\n│       ├── whisper.py            # Groq / OpenAI clients (pure stdlib)\n│       ├── config.py             # shared config (~/.config/watch/.env)\n│       ├── setup.py              # preflight + installer\n│       └── build-skill.sh        # build dist/watch.skill for claude.ai upload (dev-only)\n├── hooks/                        # SessionStart status hook (Claude Code only)\n├── .claude-plugin/               # plugin.json + marketplace.json (Claude Code)\n├── .codex-plugin/                # plugin.json — Codex/agents manifest (\"skills\": \"./skills/\")\n├── .agents/plugins/              # marketplace.json — Agent Skills marketplace listing\n├── AGENTS.md → CLAUDE.md         # generic-agent entry point\n├── tests/                        # pytest suite (ffmpeg-synthesized clips, no network)\n└── .github/workflows/            # release.yml — auto-builds watch.skill on tag push\n```\n\n## Develop\n\n```bash\n# Run the test suite (stdlib + pytest; ffmpeg required for frame tests):\npython3 -m pytest -q\n\n# Build the claude.ai upload bundle:\nbash skills/watch/scripts/build-skill.sh      # → dist/watch.skill\n```\n\nReleasing: tag `vX.Y.Z`, push the tag. The workflow builds `dist/watch.skill` and attaches it to the GitHub release. Keep the version in sync across `skills/watch/SKILL.md`, `.claude-plugin/plugin.json`, and `.codex-plugin/plugin.json`.\n\nSee [CHANGELOG.md](CHANGELOG.md) for version history.\n\n## Open source\n\nMIT license.\n\nBuilt on `yt-dlp`, `ffmpeg`, and Claude's multimodal `Read` tool. Whisper transcription via [Groq](https://groq.com) or [OpenAI](https://openai.com).\n\nBuilt by Brad Bonanno — I make content about building with AI on [YouTube (@bradbonanno)](https://www.youtube.com/@bradbonanno), and build AI operating systems for businesses at [Solaris Automation](https://www.solarisautomation.io/). If `/watch` saves you from scrubbing through a video, come say hi on the channel.\n\n## Star History\n\n<a href=\"https://www.star-history.com/?repos=bradautomates%2Fclaude-video&type=date&legend=top-left\">\n <picture>\n   <source media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/chart?repos=bradautomates/claude-video&type=date&theme=dark&legend=top-left\" />\n   <source media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/chart?repos=bradautomates/claude-video&type=date&legend=top-left\" />\n   <img alt=\"Star History Chart\" src=\"https://api.star-history.com/chart?repos=bradautomates/claude-video&type=date&legend=top-left\" />\n </picture>\n</a>\n\n---\n\n[github.com/bradautomates/claude-video](https://github.com/bradautomates/claude-video) · [@bradbonanno](https://www.youtube.com/@bradbonanno) · [Solaris Automation](https://www.solarisautomation.io/) · [LICENSE](LICENSE)\n",
  "bytes": 19227,
  "sha": "ebea63b6bebde45e717177e19a09c9570d64fcd00b203b0cbb026f63fc01c089",
  "repo_slug": "bradautomates/claude-video",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/skl_bradautomates_claude_video_watch_fc3d1b65/readme"
}