Meta Agent Tools
PT

Back to the catalog

WebLens

Scrape, crawl, map and extract the web. Pay per call in USDC, no account or API key.

Open source Repository Open in the app JSON README (API)

About

Scrape, crawl, map and extract the web. Pay per call in USDC, no account or API key.

Details

Kind
MCP servers
Topic
Web search, scraping & browser
Publisher
dev.weblens
Origin
official
Category
ferramentas
Transport
http
Version
2.1.0
Last push
2026-09-04T08:54:15Z
Repository state
ativo
Language
TypeScript
Added
2026-08-29 03:01:31
Updated
2026-08-29 03:01:31
Origin id
dev.weblens/weblens

README

# WebLens

Premium web intelligence API for AI agents, with x402 micropayments. No accounts, no API keys — pay per request in USDC on Base.

🌐 **Live API:** https://api.weblens.dev
📖 **Interactive docs:** https://api.weblens.dev/docs
🤖 **Agent guide:** https://api.weblens.dev/llms.txt
🔎 **Marketplace:** [listed on x402scan](https://www.x402scan.com/server/529d8bbf-63d0-481c-93b2-21a92e2060d8)

## Try it free, right now

No wallet, no signup — these are rate limited to 10 requests/hour per IP:

```bash
curl https://api.weblens.dev/r/https://example.com     # any page as markdown
curl https://api.weblens.dev/s/cloudflare+workers      # web search
```

## Preview before you pay (free)

An agent shouldn't have to buy a call to learn what it returns. `POST /preview` is free and
answers that up front — the live price, a one-line summary, and a real sample of the exact
response shape:

```bash
curl -X POST https://api.weblens.dev/preview \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/answer"}'
```

```jsonc
{
  "endpoint": "/answer",
  "price": "$0.05",
  "currency": "USD",
  "summary": "A direct answer with inline [n] citations to real sources.",
  "sample": { "query": "...", "answer": "...", "citations": [], "confidence": 0.92 },
  "sampleType": "recorded",
  "livePreviewAvailable": false,
  "livePreviewHint": "This endpoint calls a paid upstream provider, so free live previews are not offered — the recorded sample shows the exact response shape."
}
```

**Live vs recorded.** A real, truncated live preview runs only for endpoints whose marginal cost
is a plain fetch — currently `/fetch/basic`, `/contents` and `/map` — and only when you pass a
`url`. Everything else is backed by a metered upstream (SerpAPI, Anthropic); running those free
would burn upstream credits, so they return the recorded sample, which still shows every field
name and type. An endpoint that isn't sold returns `404`.

```bash
# live: the first 500 chars of the real result, for free
curl -X POST https://api.weblens.dev/preview \
  -H "Content-Type: application/json" \
  -d '{"endpoint": "/fetch/basic", "url": "https://example.com"}'
```

Rate limited to 10 requests/hour per IP like the rest of the free tier.

## ERC-8004: receipts and feedback

ERC-8004 (Trustless Agents) keeps compact signals on-chain and the detailed documents off-chain.
WebLens hosts the **off-chain half** — the part a service operator can run without deploying a
contract. To be precise about what that is and isn't: **WebLens is not registered on-chain, holds
no agent id, and writes nothing to any registry.**

| Endpoint | What it gives you |
|----------|-------------------|
| `GET /.well-known/agent-registration.json` | The ERC-8004 registration document: name, description, image, services, `x402Support`, payment info, feedback endpoints. `registrations` is empty (no on-chain registration) and `supportedTrust` is `["feedback"]`. |
| `GET /receipts/{requestId}` | The receipt for a paid call — endpoint, status, outcome, price, payment method, network, pay-to. Every paid response returns `X-Receipt-Id` and `X-Receipt-Url` headers pointing here. Kept 30 days. |
| `POST /feedback` | Host a feedback document *you* author; returns `{feedbackURI, feedbackHash}` (keccak-256), the pair `giveFeedback()` expects. Required fields: `agentRegistry`, `agentId`, `clientAddress`, `createdAt`, `value`, `valueDecimals` — a missing one returns `400` naming it. |
| `GET /feedback/{id}` | Serves that document byte-for-byte, so its keccak-256 hash matches the `feedbackHash` you were given. This URL *is* the `feedbackURI`. |

Two more things worth stating plainly:

- A receipt's `mac` is a **symmetric HMAC tag** (the same construction as proof-of-context). Only a
  holder of the key can verify it — it is not a third-party-verifiable signature, and nothing here
  is trustless.
- The **buyer** authors the feedback document and posts `giveFeedback()` themselves. WebLens only
  hosts the document verbatim and returns its hash; it never authors, edits, or submits feedback.

## Endpoints

All paid endpoints are `POST` with a JSON body. Prices are per request in USDC.

### Core

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/fetch/basic` | Fetch a webpage as clean markdown | $0.002 |
| `/fetch/pro` | Fetch with full JavaScript rendering (SPAs) | $0.006 |
| `/fetch/resilient` | Auto-fallback: native scraper → headless Chromium | $0.008 |
| `/contents` | Bulk page text for 1–20 URLs | $0.0015/URL |
| `/screenshot` | Capture a webpage screenshot (PNG) | $0.008 |
| `/batch/fetch` | Fetch 2–20 URLs in parallel | $0.0015/URL |
| `/map` | Discover a site's URLs (sitemaps + links, no page fetches) | $0.004 |
| `/domain` | Registration + DNS + SaaS stack + risk signals for a domain | $0.005 |
| `/tech` | Detect a site's framework, CMS, CDN, analytics and payments stack | $0.005 |
| `/package` | npm/PyPI package health: deprecation, downloads, maintenance | $0.003 |
| `/discussions` | Hacker News stories on a topic, with aggregates | $0.004 |
| `/intel/project` | Off-chain project due diligence: domain age, team/whitepaper, contract cross-check, A–F grade | $0.05 |
| `/crawl` | Bounded whole-site crawl → markdown per page | $0.0015/page |

### Search

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/search` | Real-time web search (`includeContent` adds page markdown, +$0.0015/result) | $0.015 |
| `/search/news` | Google News articles with source and date | $0.015 |
| `/search/images` | Image results with dimensions and source pages | $0.015 |
| `/search/places` | Local businesses: address, rating, phone, coordinates | $0.045 |
| `/search/shopping` | Products with prices, sellers, ratings | $0.015 |
| `/search/scholar` | Academic papers with citation counts | $0.015 |
| `/search/autocomplete` | Query suggestions (keyword/intent research) | $0.015 |
| `/search/trends` | Interest-over-time timeline | $0.015 |

### Social

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/social/youtube/transcript` | Full video transcript with timestamps | $0.03 |

### Extraction & research

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/extract` | Structured extraction against a JSON schema | $0.03 |
| `/extract/smart` | Natural-language extraction (no schema needed) | $0.035 |
| `/pdf` | Extract text and metadata from a PDF | $0.004 |
| `/answer` | Grounded answer with inline `[n]` citations | $0.05 |
| `/research` | Search + fetch + AI summary with sources | $0.08 |
| `/research/deep` | Multi-step cited research: sub-questions → search each → answer with inline `[n]` citations | $0.20–$0.35 |
| `/compare` | Compare 2–3 webpages with AI analysis | $0.05 |

`/research/deep` is the only long-running endpoint: it plans sub-questions, runs a web search per
sub-question, fetches and dedupes the sources, then synthesizes a cited answer — all in one
synchronous call. A `standard` run (3 sub-questions, 8 sources, $0.20) takes roughly **30–60
seconds**; `deep` (5 sub-questions, 12 sources, $0.35) takes longer. Set a generous HTTP timeout
(120s+) and don't retry on a client timeout, or you'll pay twice.

### Intelligence

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/intel/site-audit` | SEO / performance / security audit | $0.75 |
| `/intel/company` | Company deep dive | $1.00 |
| `/intel/market` | Market research report | $5.00 |
| `/intel/competitive` | Competitive analysis with SWOT | $8.00 |

### Utility

| Endpoint | Description | Price |
|----------|-------------|-------|
| `/memory/set` | Persistent key-value storage for agents | $0.001 |
| `/monitor/create` | URL change monitor with webhooks | $0.01 + $0.001/check |
| `/credits/buy` | Prepay credits (bonus 20–40% at $10/$50/$100) | $2–$1000 |

Free and unauthenticated: `/`, `/health`, `/docs`, `/openapi.json`, `/llms.txt`, `/discovery`, `/.well-known/x402`, `/mcp`, `/r/{url}`, `/s/{query}`, `/free/fetch`, `/free/search`, `/preview`, `/.well-known/agent-registration.json`, `/receipts/{requestId}`, `/feedback`, `/feedback/{id}`.

Dynamically-priced endpoints (`/fetch/pro`, `/extract`) apply a complexity multiplier — up to 3× for bot-protected domains. Cached responses on the fetch family are **70% cheaper**. The exact price is always in the `402` challenge.

## Use with AI agents (MCP)

**Remote HTTP — nothing to install:**
```json
{ "mcpServers": { "weblens": { "url": "https://api.weblens.dev/mcp" } } }
```

**Local with automatic payment:**
```json
{
  "mcpServers": {
    "weblens": {
      "command": "npx",
      "args": ["-y", "@weblens/mcp"],
      "env": { "PRIVATE_KEY": "0xYourPrivateKeyHere" }
    }
  }
}
```

See [mcp-server/README.md](./mcp-server/README.md) for the full tool list and setup.

## API usage

```bash
# Fetch a page as markdown
curl -X POST https://api.weblens.dev/fetch/basic \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

# Search, with page content included
curl -X POST https://api.weblens.dev/search \
  -H "Content-Type: application/json" \
  -d '{"query": "latest AI news", "limit": 5, "includeContent": true}'

# Structured extraction
curl -X POST https://api.weblens.dev/extract \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/product",
       "schema": {"title": {"type": "string"}, "price": {"type": "number"}},
       "instructions": "Extract the product title and price"}'

# Crawl a site (pay for the page budget you request)
curl -X POST https://api.weblens.dev/crawl \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "limit": 10, "maxDepth": 2}'

# Deep research with inline citations (slow — 30-60s, so raise the client timeout)
curl --max-time 120 -X POST https://api.weblens.dev/research/deep \
  -H "Content-Type: application/json" \
  -d '{"query": "How are AI agents using micropayments in 2026?", "depth": "standard"}'
```

Each returns `402 Payment Required` with the price until you attach payment — see below.

## How payments work

WebLens implements [x402](https://x402.org) v2:

1. `POST` any paid endpoint with no payment.
2. Get `402 Payment Required` with a `PAYMENT-REQUIRED` header carrying the price, network, asset, and pay-to address.
3. Sign a USDC transfer authorization with your wallet.
4. Retry the same request with a `Payment-Signature` header.
5. The payment is verified on-chain and you get your data.

Settlement runs through the [PayAI facilitator](https://facilitator.payai.network) on Base mainnet. Any x402 v2 client works — `@x402/axios`, `@x402/fetch`, or the bundled MCP server, which handles the whole flow for you.

**Prepaid credits** are an alternative to per-request payment: buy credits with `/credits/buy`, then send `X-CREDIT-WALLET` / `X-CREDIT-SIGNATURE` / `X-CREDIT-TIMESTAMP` headers. Failed requests are refunded automatically.

## Development

```bash
pnpm install
pnpm run dev          # local dev server on :8787
pnpm run build        # tsc --noEmit (type check)
pnpm run lint         # eslint
pnpm run test         # vitest (unit + property + workerd integration)
pnpm run deploy       # wrangler deploy (production)
```

Secrets are set with wrangler, never committed:

```bash
wrangler secret put SERP_API_KEY        # search + all verticals + transcripts
wrangler secret put ANTHROPIC_API_KEY   # AI extraction, research, answers, intel
wrangler secret put CDP_API_KEY_ID      # optional: CDP facilitator fallback
wrangler secret put CDP_API_KEY_SECRET
```

| Variable | Purpose |
|----------|---------|
| `PAY_TO_ADDRESS` | Wallet that receives payments (`wrangler.toml` var) |
| `NETWORK` | `base` (production) or `base-sepolia` (testnet) |
| `SERP_API_KEY` | SerpAPI key — search, verticals, YouTube transcripts |
| `ANTHROPIC_API_KEY` | Claude — extraction, research, answers, intel |
| `SIGNING_PRIVATE_KEY` | Optional: HMAC key for proof-of-context tags |
| `PAYAI_FACILITATOR_URL` | Optional: override the facilitator endpoint |

### Testing against testnet

```bash
wrangler dev --env testnet          # Base Sepolia, fake USDC
API_URL=http://localhost:8787 PRIVATE_KEY=0x... npx tsx scripts/test-payment-testnet.ts
```

Get free testnet USDC from the [Circle faucet](https://faucet.circle.com/) on Base Sepolia.

### Discovery

```bash
pnpm run verify-bazaar    # check the x402 discovery extension on every route
```

`/openapi.json` doubles as the discovery document for [x402scan](https://www.x402scan.com/discovery/spec) — it carries `info.x-guidance`, per-operation `x-payment-info`, and `security: []` on free operations. WebLens is listed on x402scan and indexed in the PayAI facilitator catalog.

> **Note on the Coinbase CDP Bazaar:** it only indexes services whose payments the CDP facilitator itself settles. WebLens uses PayAI as its primary facilitator (CDP has an open Base-mainnet settlement bug, [x402#1065](https://github.com/x402-foundation/x402/issues/1065)), so it does not appear there.

## Tech stack

- **Cloudflare Workers** — edge runtime, Durable Objects, KV, Browser Rendering
- **Hono** — web framework
- **x402** — HTTP-native micropayments
- **Zod** — request validation
- **Vitest** + fast-check — unit, property, and real-workerd integration tests

## Links

- [API documentation](https://api.weblens.dev/docs)
- [LLM-optimized guide](https://api.weblens.dev/llms.txt)
- [Service discovery](https://api.weblens.dev/discovery)
- [x402 protocol](https://x402.org)

## License

MIT

More