Meta Agent Tools
PT

Back to the catalog

heimdall

Analyze video files: especially bug report screen recordings, directly in Claude Code. Extracts frames via ffmpeg, transcribes audio via whi

Open source Repository Open in the app JSON README (API)

About

Analyze video files: especially bug report screen recordings, directly in Claude Code. Extracts frames via ffmpeg, transcribes audio via whisper-cpp, and produces structured reports with steps-to-reproduce, error messages, and timestamps. All heavy work runs in a subagent so your main context stays clean.

Details

Kind
Plugins
Topic
AI, RAG & memory
Publisher
pranavtotla
Origin
marketplace
Category
ferramentas
Last push
2026-07-13T06:30:45Z
Repository state
ativo
Language
Python
License
MIT
Added
2026-08-30 01:48:58
Updated
2026-08-30 01:48:58
Origin id
pranavtotla/heimdall/heimdall

README

# heimdall

Video analysis plugin for Claude Code. Feed it a bug report recording, get back steps-to-reproduce, error messages, and timestamps.

**Homepage:** [totla.tech/work/heimdall](https://totla.tech/work/heimdall)

## Install

```bash
git clone --depth 1 https://github.com/pranavtotla/heimdall.git ~/.claude/skills/heimdall && ~/.claude/skills/heimdall/setup
```

Restart Claude Code. Commands: `/heimdall-watch`, `/heimdall-setup`.

### Via Plugin Marketplace

```bash
claude plugin marketplace add pranavtotla/heimdall
claude plugin install heimdall@heimdall
```

Then run `/heimdall:setup` to install dependencies (ffmpeg, whisper-cpp).

## Usage

```
/heimdall-watch path/to/bug-recording.mp4
```

Or via marketplace install: `/heimdall:watch path/to/bug-recording.mp4`

Heimdall will:
1. Extract frames adaptively by duration — interval sampling for short clips
   (< 1 min), scene detection for 1–30 min videos, keyframes for longer ones
   (override with `--strategy interval|scene|keyframe`)
2. Transcribe audio via whisper-cpp (if available)
3. Correlate frames with transcript timestamps
4. Produce a structured bug report

The agent may auto-drill into interesting time ranges (error dialogs, state transitions) for higher-density frame extraction.

### Extraction strategies

Frame density adapts to video length: short clips are sampled at a fixed
interval; 1–30 minute videos use two-pass scene detection (frames are chosen
where the picture actually changes, with periodic anchors so static stretches
stay covered); videos over 30 minutes fall back to keyframes. Tune scene
sensitivity with `--scene-threshold` (default `0.08`). Every extraction writes
a `frames.json` manifest recording each frame's timestamp, scene score, and
origin.

### Example Output

```markdown
## Video Analysis: Save button returns 403

**Source:** bug.mp4 | **Duration:** 45 sec | **Resolution:** 1920x1080 | **Transcript:** base.en

### Summary
User clicks the save button on the settings page. The save request fails with
a 403 Forbidden error. The user's session token appears to have expired.

### Steps to Reproduce
1. [0:00] Settings page loaded, all fields populated
2. [0:12] User modifies the "display name" field
3. [0:18] User clicks "Save Changes" button
4. [0:19] Error dialog appears: "Failed to save: 403 Forbidden"

### Error Messages
- [0:19] "Failed to save: 403 Forbidden" (error dialog)
```

## Architecture

```
scripts/extract.py          ← Core engine (harness-agnostic CLI)
agents/heimdall.md          ← Subagent (frame analysis + report synthesis)
commands/                   ← /heimdall:watch, /heimdall:setup (marketplace)
skills/                     ← /heimdall-watch, /heimdall-setup (git clone)
hooks/hooks.json            ← SessionStart dependency check
setup                       ← One-command installer + skill linker
```

All heavy work (frames, transcripts, analysis) runs inside the `heimdall` subagent — the main agent's context window only sees the final structured report.

## Requirements

- **ffmpeg** (required) — frame extraction and audio processing
- **whisper-cpp** (optional) — audio transcription. Setup downloads the `base.en` model (148MB, checksum-verified) to `~/.cache/heimdall/models/`; higher-accuracy opt-in: `./setup --model small.en`, or per-run `--model`/`$HEIMDALL_WHISPER_MODEL`.
- **Python 3.8+** — no external Python dependencies

## License

MIT

More