Meta Agent Tools
PT

Back to the catalog

io.github.RayAKaan/ocular-audio-mcp

MCP server for video transcripts, screenshots, and OCR on YouTube and web videos.

Open source Open in the app JSON README (API)

About

MCP server for video transcripts, screenshots, and OCR on YouTube and web videos.

Details

Kind
MCP servers
Topic
Files & documents
Publisher
rayakaan
Origin
official
Category
ferramentas
Transport
local
Version
1.1.2
Stars
1
Last push
2026-07-13T12:30:21Z
Repository state
ativo
Language
Python
License
MIT
Added
2026-08-29 03:02:11
Updated
2026-08-29 03:02:11
Origin id
io.github.RayAKaan/ocular-audio-mcp

README

# OcularAudio MCP

![OcularAudio MCP](OcularAudioMCP.png)

An asynchronous Model Context Protocol (MCP) server that gives AI models "eyes and ears" to process web videos. It extracts transcripts and captures screenshots from YouTube and other video platforms.

## Features

- **Hybrid transcript extraction**: Fetches YouTube captions instantly, falls back to local Whisper ASR
- **On-demand video screenshots**: Captures frames at any timestamp without downloading the full video
- **OCR on screenshots**: Extract visible text from frames using Tesseract (optional, `--ocr` flag)
- **Cookie authentication**: Supports age-restricted and private videos via cookies.txt
- **Local caching**: Processed videos are cached for instant subsequent lookups
- **Async architecture**: Non-blocking design keeps MCP clients responsive
- **Flexible output**: Clipboard, stdout, file, or JSON — your choice

## Benchmark

See [BENCHMARK.md](BENCHMARK.md) for performance benchmarks and a deep competitive analysis against all major video transcript, screenshot, and OCR tools in the MCP and CLI ecosystem.

## Requirements

- **Python 3.9+** (required for `list[int]` type hints)
- **FFmpeg** (required by yt-dlp and OpenCV)
- **Node.js 18+** (only for the CLI wrapper)
- **Tesseract** (optional, only for `--ocr` flag)

## Installation

### 1. Install system dependencies

**macOS:**
```bash
brew install ffmpeg python3
# Optional (for OCR):
brew install tesseract
```

**Windows:**
```bash
choco install ffmpeg python
# Optional (for OCR):
choco install tesseract
```

**Linux:**
```bash
sudo apt update && sudo apt install ffmpeg python3 python3-pip
# Optional (for OCR):
sudo apt install tesseract-ocr
```

### 2. Install Python packages

```bash
pip install -r requirements.txt
```

Or manually:
```bash
pip install mcp youtube-transcript-api yt-dlp opencv-python-headless faster-whisper requests pytesseract
```

### 3. Install Node.js CLI (optional)

```bash
npm install
```

## Usage

### Option A: MCP Server (Recommended)

The MCP server gives AI models direct access to video transcripts and screenshots.

**Quick Install** — No installation needed. Just add the config to your MCP client below.

#### Claude Desktop

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS) or `%APPDATA%\Claude\claude_desktop_config.json` (Windows)

#### Cursor

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `.cursor/mcp.json` (project) or `~/.cursor/mcp.json` (global)

#### Claude Code

```bash
claude mcp add ocular-audio-mcp -- npx ocular-audio-mcp
```

#### Codex CLI (OpenAI)

```bash
codex mcp add ocular-audio-mcp -- npx ocular-audio-mcp
```

#### Gemini CLI

```bash
gemini mcp add ocular-audio-mcp npx ocular-audio-mcp --scope user
```

#### Windsurf

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `~/.codeium/windsurf/mcp_config.json`

#### Zed

```json
{
  "context_servers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `~/.config/zed/settings.json`

#### VS Code (GitHub Copilot)

```json
{
  "servers": {
    "ocular-audio-mcp": {
      "type": "stdio",
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `.vscode/mcp.json`

#### OpenCode

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```
Config: `~/.opencode/config.json`

#### Cline (VS Code Extension)

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "npx",
      "args": ["ocular-audio-mcp"]
    }
  }
}
```

#### Local Development (from source)

```json
{
  "mcpServers": {
    "ocular-audio-mcp": {
      "command": "python",
      "args": ["/path/to/ocular_audio_mcp.py"]
    }
  }
}
```

### Option B: CLI

```bash
npx ocular-audio "https://www.youtube.com/watch?v=dQw4w9WgXcQ"
```

By default, the transcript is printed to stdout and copied to your clipboard. Paste it into Claude Web, ChatGPT, or any AI chat.

#### CLI Options

| Flag | Description |
|------|-------------|
| `-h`, `--help` | Show help message |
| `-v`, `--version` | Show version number |
| `--stdout` | Print transcript to stdout only (no clipboard, no file) |
| `--no-clipboard` | Skip clipboard copy |
| `--output <file>` | Write context to a specific file path |
| `--json` | Output raw JSON (metadata + transcript) for programmatic use |
| `--detail <level>` | Screenshot capture mode: `overview`, `balanced`, `deep`, `auto` (default: `auto`) |
| `--ocr` | Extract text from screenshots using Tesseract OCR |
| `--force` | Bypass cache and re-process the video |
| `--verbose` | Show detailed progress information |
| `--quiet` | Suppress summary and status messages |
| `--check` | Check system dependencies (Python, FFmpeg, Whisper, Tesseract) |
| `--list-cached` | List all cached videos with titles |
| `--cache-info` | Show cache statistics (count, size, oldest/newest) |
| `--clear-cache` | Delete all cached transcripts and screenshots |

#### Examples

```bash
# Basic usage — prints to stdout + copies to clipboard
npx ocular-audio "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Stdout only — great for piping to other tools
npx ocular-audio --stdout "https://www.youtube.com/watch?v=dQw4w9WgXcQ" | head -50

# Write to a specific file
npx ocular-audio --output transcript.txt "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Raw JSON for programmatic consumption
npx ocular-audio --json "https://www.youtube.com/watch?v=dQw4w9WgXcQ" | jq .metadata.title

# Transcript only, no screenshots
npx ocular-audio --detail overview "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Maximum screenshots
npx ocular-audio --detail deep "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# No clipboard copy, just print to terminal
npx ocular-audio --no-clipboard "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Screenshots with OCR — extract visible text from frames
npx ocular-audio --ocr "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# JSON output with OCR
npx ocular-audio --json --ocr "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Force re-process (bypass cache)
npx ocular-audio --force "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Verbose mode — see all progress details
npx ocular-audio --verbose "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Quiet mode — minimal output
npx ocular-audio --quiet --stdout "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

# Check system capabilities
npx ocular-audio --check

# List cached videos
npx ocular-audio --list-cached

# Show cache stats
npx ocular-audio --cache-info

# Clear all cached data
npx ocular-audio --clear-cache
```

## Cookie Setup (for age-restricted/private videos)

YouTube may block transcript access for age-restricted or private videos. To fix this, export your browser cookies:

1. Install a browser extension like "Get cookies.txt LOCALLY" (Chrome/Firefox)
2. Go to youtube.com while logged in
3. Export cookies to a file named `cookies.txt`
4. Place the file in one of these locations:
   - `~/.cache/ocular_audio_mcp/cookies.txt`
   - `~/.config/ocular_audio_mcp/cookies.txt`
   - `./cookies.txt` (in the project directory)

The server will automatically detect and use the cookies file.

## MCP Tools

### `get_ocular_audio_capabilities`

Returns system capabilities and dependency status. Use this to check what features are available.

**Parameters:** None

**Returns:** System info including Python version, FFmpeg, Whisper, Tesseract, OpenCV, and cookie status.

### `get_ocular_audio_metadata`

Extracts only video metadata (title, creator, duration, views, chapters) without transcript. Much faster than getting the full transcript.

**Parameters:**
- `url` (string): Video URL

### `get_ocular_audio_transcript`

Extracts the complete transcript, video chapters, and metadata from a video.

**Parameters:**
- `url` (string): Video URL
- `use_local_whisper` (boolean, default: true): Enable Whisper fallback if captions unavailable

### `get_ocular_audio_chapters`

Extracts only video chapters with timestamps. Returns chapter titles with start times in [MM:SS] format.

**Parameters:**
- `url` (string): Video URL

### `get_ocular_audio_video_screenshots`

Captures screenshots at specific timestamps.

**Parameters:**
- `url` (string): Video URL
- `timestamps_secs` (array of integers): Timestamps to capture (e.g., `[45, 120, 300]`)
- `enable_ocr` (boolean, default: false): If true, run OCR on each captured frame to extract visible text

### `get_ocular_audio_video_context`

Extracts transcript, metadata, and intelligent screenshots in one call. Automatically analyzes the transcript to find visually important moments and captures screenshots at those timestamps.

**Parameters:**
- `url` (string): Video URL
- `detail_level` (string, default: "auto"): Controls screenshot capture mode:
  - `"auto"` - Adapts to video length and content importance
  - `"overview"` - Transcript and metadata only, no screenshots (fastest)
  - `"balanced"` - Screenshots only at visually important moments (strong signals)
  - `"deep"` - Screenshots at every visually significant moment (all signals)
- `use_local_whisper` (boolean, default: true): Enable Whisper fallback if captions unavailable
- `enable_ocr` (boolean, default: false): If true, run OCR on captured screenshots to extract visible text

### `list_ocular_audio_cache`

Lists all cached videos with their metadata (title, uploader, duration, when cached).

**Parameters:** None

### `clear_ocular_audio_cache`

Clears cached video data.

**Parameters:**
- `video_id` (string, optional): Video ID to clear specific video. If empty, clears all cache.

## Cache Management

Processed videos are cached in `~/.cache/ocular_audio_mcp/` for 7 days. Use the CLI flags to manage the cache:

```bash
npx ocular-audio --list-cached     # See what's cached
npx ocular-audio --cache-info      # Storage stats
npx ocular-audio --clear-cache     # Wipe everything
```

Or manually:
```bash
rm -rf ~/.cache/ocular_audio_mcp/*.json
```

## Troubleshooting

### "No local ASR engines found"
Install a Whisper engine:
```bash
pip install faster-whisper
```

### "Audio track download failed"
- Check your network connection
- For age-restricted videos, add a cookies.txt file (see Cookie Setup above)
- Ensure FFmpeg is installed: `ffmpeg -version`

### "Failed to extract a playable video stream"
- The video may be private or geo-blocked
- Try adding cookies.txt
- Check if the video is still available

### Python not found on Windows
Ensure Python is in your PATH. Try:
```bash
python --version
```
If not found, reinstall Python from python.org and check "Add Python to PATH" during installation.

### MCP server not connecting
- Verify the path in your MCP client config is correct
- Test the server manually: `python /path/to/ocular_audio_mcp.py`
- Check that all dependencies are installed: `pip list | grep -E "mcp|whisper|yt-dlp"`

## License

MIT

More