Meta Agent Tools
PT

Back to the catalog

io.github.MarvinRey7879/framefetch

Any social-video URL → transcript, metadata, frames, OCR, summary, search, Q&A. MCP server + x402.

Open source Repository Open in the app JSON README (API)

About

Any social-video URL → transcript, metadata, frames, OCR, summary, search, Q&A. MCP server + x402.

Details

Kind
MCP servers
Topic
Finance & crypto
Publisher
marvinrey7879
Origin
official
Category
ferramentas
Transport
http
Version
1.9.1
Last push
2026-07-31T21:06:33Z
Repository state
ativo
Language
JavaScript
License
MIT
Added
2026-08-29 03:02:05
Updated
2026-08-29 03:02:05
Origin id
io.github.MarvinRey7879/framefetch

README

<p align="center">
  <img src="logo.svg" width="84" height="84" alt="FrameFetch logo">
</p>

<h1 align="center">FrameFetch</h1>

<p align="center">
  <b>Any social-video URL → answers, transcript, metadata, insights, frames &amp; on-screen text (OCR).</b><br>
  Agent-first video data API + MCP server. Pay per call, or with x402 (USDC) — no account.
</p>

<p align="center">
  <a href="https://www.npmjs.com/package/framefetch"><img src="https://img.shields.io/npm/v/framefetch?color=ff5a36" alt="npm"></a>
  <a href="https://framefetch.net"><img src="https://img.shields.io/badge/website-framefetch.net-ff8562" alt="website"></a>
  <a href="https://framefetch.net/status"><img src="https://img.shields.io/badge/status-live-4ad8a0" alt="status"></a>
  <img src="https://img.shields.io/badge/license-MIT-9a9fa6" alt="MIT">
</p>

---

FrameFetch turns one **YouTube, YouTube Shorts, TikTok, Instagram Reels, Pinterest, or Reddit** video URL into a single JSON response: a **direct answer to a question about the video**, **metadata**, **engagement insights**, a **transcript** (captions or Whisper), an **LLM digest** (text or spoken mp3), **structured JSON** (chapters/entities/products/claims), **comments + sentiment**, **parametrically-sampled frames** (every Nth / 1-per-second / a time range, at any width), and the **on-screen text burned into those frames** (OCR — captions, price tags, signage). Plus keyword **search** when you don't have a URL yet, and **batch** for up to 10 URLs in one call. Built API-first and MCP-first for AI agents.

> This repo is the **open-source client + docs**. The service itself runs at **[framefetch.net](https://framefetch.net)** — you bring a free API key (or pay per call with x402); the backend stays hosted.

## Why

An LLM can't watch a video. To reason about one it needs the video turned into text and images first — an answer, a transcript, metadata, a few frames. FrameFetch returns all of that from a URL, across six platforms, through one schema.

## Install

```bash
npm install framefetch
```

Node 18+ (uses built-in `fetch`). Get a free key: [framefetch.net](https://framefetch.net).

> **Version note.** This repo is at **0.4.0**. The newest version currently on npm is **0.3.0** — `npm install framefetch` still gives you that one, and it has only `extract`/`metadata`/`transcript`/`frames`/`platforms`/`status`/`demo`/`createKey`. Everything else documented below is live on the API today and available from this repo; from npm 0.3.0 you can reach the same data through `extract({ fields: [...] })`.

## Ask a question — get an answer, not a transcript dump

A direct question about a video returns a short, grounded answer with timestamped quotes, instead of you parsing a 25,000-token transcript yourself.

```js
import { FrameFetch } from 'framefetch';

const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });

const { ask } = await ff.ask(
  'https://www.youtube.com/watch?v=jNQXAC9IVRw',
  'What does the presenter say to do first?',
);

console.log(ask.answer);          // short, direct answer
console.log(ask.confidence);      // 'high' | 'medium' | 'low'
for (const q of ask.quotes) {     // verbatim, timestamped supporting quotes
  console.log(`[${q.t_sec}s] ${q.text}`);
}
console.log(ask.coverage);        // which part of the transcript was analyzed
```

Charged only when an answer is actually produced. A repeat question about an already-extracted video reuses the cached transcript, so it answers fast without a re-download or re-transcription — but the answer itself is always freshly generated, never cache-served.

**Frames-based answers:** when a video has no transcript (e.g. Pinterest, or transcription failed), the answer is grounded in sampled keyframe images instead. Then `coverage.mode` is `"frames"`, `quotes` is `[]` (no transcript text to quote), and `confidence` is capped at `"medium"`.

## Quick start

```js
import { FrameFetch } from 'framefetch';

const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY });

const r = await ff.extract({
  url: 'https://www.youtube.com/watch?v=jNQXAC9IVRw',
  fields: ['metadata', 'transcript', 'frames', 'text_overlay'],
  frames: { mode: 'fps', fps: 1, width: 480 },
});

console.log(r.metadata.title);         // "Me at the zoo"
console.log(r.transcript.text);        // "All right, so here we are, in front of the elephants…"
console.log(r.frames.length);          // 19 — frames is an array
console.log(r.textOverlay?.[0]?.text); // on-screen text detected in the first frame, if any
```

Note the two spellings: `text_overlay` is the **request** field name, `textOverlay` is the **response** key.

### Scoped helpers

```js
await ff.metadata(url);          // title, author, duration, views, likes…
await ff.transcript(url);        // captions, else Whisper
await ff.frames(url, { mode: 'fps', fps: 1, width: 512 });
await ff.ask(url, 'What product is being reviewed?'); // grounded Q&A, see above
await ff.digest(url);            // LLM summary of the transcript
await ff.audioDigest(url, { voice: 'nova' }); // spoken mp3 briefing (signed URL, 24h)
await ff.structured(url);        // chapters/entities/products/claims/key_moments
await ff.comments(url, { comments_cap: 50 }); // top-level comments
await ff.commentSentiment(url);  // aggregated audience-mood rollup (+ the comments)
await ff.platforms();            // capability matrix (no key)
await ff.status();               // live service health (no key)

// on-screen text (OCR) — requires "frames" alongside it, use extract() directly:
await ff.extract({ url, fields: ['frames', 'text_overlay'], frames: { mode: 'fps', fps: 1 } });
```

Every helper above is a thin wrapper over `extract()`, so anything `extract()` accepts (`translate`,
`format`, extra `fields`, …) can be passed as the last argument and is forwarded unchanged.

### Search for videos, extract many at once

```js
// Find something to extract when you don't have a URL yet
const s = await ff.search('how to make sourdough', { limit: 5 });
for (const hit of s.results) {
  console.log(hit.title, hit.url, hit.durationSec);
}

// Then extract up to 10 of them in ONE call. Shared options apply to every url.
const b = await ff.batch(s.results.slice(0, 3).map((r) => r.url), {
  fields: ['metadata', 'digest'],
});
for (const item of b.results) {
  if (!item.ok) { console.error(item.url, item.error?.code); continue; }
  console.log(item.metadata.title, '→', item.digest.gist);
}
```

One failing URL never fails the batch — each entry carries its own `ok` flag and, when `ok` is false,
an `error` with `code`/`message`/`hint`. Per-URL `frames` specs are not accepted in a batch; use
`extract()` for those.

### Translate the transcript, export subtitles

```js
// translate the transcript into 1 of 25 languages (surfaced as transcript_translated)
const r = await ff.transcript(url, { translate: 'ja' });
console.log(r.transcript_translated.text);

// export subtitles directly — format is sent as a query param, response comes back as a string
const srt = await ff.transcript(url, { format: 'srt' });         // source-language subtitles
const vttJa = await ff.transcript(url, { translate: 'ja', format: 'vtt' }); // translated subtitles
```

### No signup

```js
const ff = new FrameFetch();                              // no key
await ff.demo('https://youtu.be/jNQXAC9IVRw');            // instant metadata, rate-limited
const { key } = await ff.createKey('you@example.com');    // self-serve key + free credit
```

## Use it from an MCP agent

FrameFetch ships an MCP server (Streamable HTTP) with four tools: `framefetch_extract`,
`framefetch_platform_capabilities`, `framefetch_search` and `framefetch_account`. Add it to Claude,
Cursor, or any MCP client:

```json
{
  "mcpServers": {
    "framefetch": {
      "url": "https://framefetch.net/mcp",
      "headers": { "Authorization": "<YOUR_FRAMEFETCH_KEY>" }
    }
  }
}
```

Or one line:

```bash
claude mcp add --transport http framefetch https://framefetch.net/mcp \
  --header "Authorization: <YOUR_FRAMEFETCH_KEY>"
```

MCP lives at `https://framefetch.net/mcp` and speaks JSON-RPC over Streamable HTTP. REST lives under
`/v1/*` and takes plain JSON (`{"url": "…"}`). Crossing the two is the single most common first-call
mistake, so both directions answer clearly: a REST body POSTed to `/mcp` comes back as a JSON-RPC
parse error, and a JSON-RPC body POSTed to `/v1/extract` comes back as `400 WRONG_ENDPOINT` naming
the right URL for your client.

### Local stdio bridge

Prefer a local stdio server (Claude Desktop, sandboxes, no inbound HTTP)? This package
ships `framefetch-mcp`, a zero-dependency stdio↔HTTP bridge that exposes the same tools
and forwards calls to `framefetch.net`:

```json
{
  "mcpServers": {
    "framefetch": {
      "command": "npx",
      "args": ["-y", "framefetch-mcp"],
      "env": { "FRAMEFETCH_API_KEY": "<YOUR_FRAMEFETCH_KEY>" }
    }
  }
}
```

`tools/list` works with no key; tool calls use `FRAMEFETCH_API_KEY` (or x402). Override the
endpoint with `FRAMEFETCH_MCP_URL`.

## Pay without an account (x402)

Autonomous agents can pay per call in **USDC via x402** on Base — no signup, no human in the loop. Discoverable in the x402 Bazaar and at [`/.well-known/x402.json`](https://framefetch.net/.well-known/x402.json). Humans can use a free tier, prepaid credits, or a Stripe card.

## Errors

Failed calls throw `FrameFetchError` with `.status`, `.code`, and `.hint`:

```js
import { FrameFetchError } from 'framefetch';
try {
  await ff.transcript(url);
} catch (e) {
  if (e instanceof FrameFetchError && e.status === 402) {
    // out of credit — top up at framefetch.net or via x402
  }
}
```

## API surface

| Method | Endpoint | Auth |
| --- | --- | --- |
| `extract({ url, fields, frames, … })` | `POST /v1/extract` | key |
| `ask(url, question)` | `POST /v1/extract` (`ask` param) | key |
| `metadata(url)` | `POST /v1/metadata` | key |
| `transcript(url, { translate, format })` | `POST /v1/transcript` | key |
| `frames(url, spec)` | `POST /v1/frames` | key |
| `digest(url)` | `POST /v1/extract` (`digest`) | key |
| `audioDigest(url, { voice })` | `POST /v1/extract` (`audio_digest`) | key |
| `structured(url)` | `POST /v1/extract` (`structured`) | key |
| `comments(url, { comments_cap })` | `POST /v1/extract` (`comments`) | key |
| `commentSentiment(url)` | `POST /v1/extract` (`comment_sentiment`) | key |
| `search(query, { limit })` | `POST /v1/search` | key |
| `batch(urls, { fields })` | `POST /v1/batch` | key |
| `platforms()` | `GET /v1/platforms` | — |
| `status()` | `GET /v1/status` | — |
| `demo(url)` | `POST /v1/demo` | — |
| `createKey(email)` | `POST /v1/keys` | — |

### Full `extract()` request shape

```ts
ff.extract({
  url: string,
  fields?: Field[],            // 'metadata' | 'insights' | 'transcript' | 'frames' | 'text_overlay'
                               // | 'digest' | 'audio_digest' | 'structured' | 'comments'
                               // | 'comment_sentiment' | 'delta'
  frames?: { mode, n, fps, from, to, format, width },
  translate?: string,          // ISO-639-1 target language (25 supported)
  voice?: string,              // TTS voice for audio_digest
  comments_cap?: number,       // 1-200, default 100
  ask?: string,                // 3-500 char question — see ff.ask() above
  publish?: boolean,           // opt in to a public per-video SEO page
  format?: 'md' | 'markdown' | 'srt' | 'vtt', // alternate egress rendering (returns a string, not JSON)
});
```

See [`index.d.ts`](index.d.ts) for the complete typed response shape (`ExtractResult`, `Ask`,
`VideoStructured`, `VideoComments`, `CommentSentiment`, `AudioDigest`, `SearchResult`,
`BatchResult`, …).

Full OpenAPI: [framefetch.net/openapi.json](https://framefetch.net/openapi.json) · Docs: [framefetch.net/docs](https://framefetch.net/docs)

## Links

[Website](https://framefetch.net) · [Docs](https://framefetch.net/docs) · [Pricing](https://framefetch.net/pricing) · [Status](https://framefetch.net/status) · [Guide: giving an agent video data](https://framefetch.net/ai-agent-video-data) · [Compare vs alternatives](https://framefetch.net/compare-video-data-apis)

## License

MIT

More