{
  "markdown": "# AgentScrape\n\n> **Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC.**\n\n[![x402](https://img.shields.io/badge/x402-v2-blue)](https://x402.org)\n[![MCP](https://img.shields.io/badge/MCP-native-purple)](https://modelcontextprotocol.io)\n[![Base](https://img.shields.io/badge/Network-Base%20mainnet-0052FF)](https://base.org)\n[![Cloudflare Workers](https://img.shields.io/badge/runtime-Cloudflare%20Workers-F38020)](https://workers.cloudflare.com)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![hshintelligence/agent-scrape MCP server](https://glama.ai/mcp/servers/hshintelligence/agent-scrape/badges/score.svg)](https://glama.ai/mcp/servers/hshintelligence/agent-scrape)\n\n**Live:** [`agent-scrape.healingsunhaven.workers.dev`](https://agent-scrape.healingsunhaven.workers.dev)\n\nAI agents discover AgentScrape, pay per call in USDC on Base, and get clean structured data back — no accounts, no API keys, no human in the loop.\n\nTwo protocols, one service:\n\n- **HTTP API** with x402 v2 payment gate — for agents using raw HTTP\n- **MCP server** (Streamable HTTP transport) — for Claude Desktop, Cursor, Continue.dev, and any MCP-compatible framework\n\n---\n\n## Why this exists\n\nTraditional scraping APIs assume a human:\n\n- Sign up with email\n- Add a payment method\n- Manage an API key\n- Commit to a monthly subscription\n\n**AI agents can't do any of that.** They have wallets. They have stablecoins. They need infrastructure built for them.\n\nAgentScrape speaks two open standards:\n\n- **[x402](https://x402.org)** — HTTP-native payment protocol. The server returns 402 with payment requirements; the agent signs a USDC transfer; the facilitator settles on-chain; the data flows back. End-to-end in under 2 seconds.\n- **[MCP](https://modelcontextprotocol.io)** — the standard agent tool interface. Agents browse, discover, and invoke tools without per-vendor SDKs.\n\nNo signup. No API keys. No subscription. Just USDC.\n\n---\n\n## Tools\n\nAll tools cost **$0.001 USDC** during the mainnet validation window. Pricing ramps after stability is proven (see [Pricing](#pricing)).\n\n| Tool | Description |\n|---|---|\n| `scrape_webpage` | Scrape any URL to markdown, HTML, text, or JSON |\n| `extract_structured_data` | AI-powered structured extraction with natural-language prompts and JSON schemas (Groq + Llama 4 Scout) |\n| `screenshot_webpage` | PNG screenshot with viewport control (desktop/mobile/tablet, optional full-page) |\n| `extract_metadata` | Title, description, Open Graph, Twitter cards, JSON-LD, canonical URL |\n| `create_browser_session` | Stateful browser session with cookie + localStorage persistence |\n| `run_workflow` | Multi-step atomic execution: navigate, click, type, wait, scroll, screenshot, extract, evaluate |\n\n---\n\n## Quick start — HTTP API\n\n```bash\n# Free service discovery (no payment)\ncurl https://agent-scrape.healingsunhaven.workers.dev/\n\n# Paid call — without payment, returns 402 with payment requirements\ncurl -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"url\": \"https://example.com\", \"format\": \"markdown\"}'\n\n# Free tier — first 10 calls per wallet, send your wallet in the X-402-Payer header\ncurl -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \\\n  -H \"Content-Type: application/json\" \\\n  -H \"X-402-Payer: 0xYourBaseWalletAddress\" \\\n  -d '{\"url\": \"https://example.com\", \"format\": \"markdown\"}'\n```\n\n---\n\n## Quick start — MCP\n\nAdd to any MCP-compatible client via Streamable HTTP:\n\n```json\n{\n  \"mcpServers\": {\n    \"agent-scrape\": {\n      \"url\": \"https://agent-scrape.healingsunhaven.workers.dev/mcp\",\n      \"transport\": \"streamable-http\"\n    }\n  }\n}\n```\n\nThe x402 payment metadata is broadcast inside the MCP `tools/list` response. Your client handles the payment handshake automatically when configured with an x402-compatible wallet.\n\n---\n\n## Discovery\n\nFor machine-driven discovery, four standard endpoints are exposed (free, no auth):\n\n| Endpoint | Purpose |\n|---|---|\n| `/` | Service profile JSON (tools, prices, network, payTo) |\n| `/.well-known/x402` (and `.json` alias) | x402 manifest — payTo, network, facilitator, all paid routes |\n| `/openapi.json` | OpenAPI 3.1 spec for traditional tooling |\n| `/llms.txt` | Agent-friendly plaintext description |\n\nThe 402 response itself carries the canonical payment requirements in the base64-encoded `Payment-Required` header, and the body includes a human-readable error with discovery links.\n\n---\n\n## Architecture\n\n```\n                ┌──────────────────────────────────┐\n                │      Cloudflare Workers          │\n                │      (global edge, 330+ POPs)    │\n                │                                  │\nAI Agent ───────┼─▶  GET  /                  (free)│\n(HTTP)          ├─▶  GET  /.well-known/x402  (free)│\n                ├─▶  GET  /openapi.json      (free)│\n                ├─▶  GET  /llms.txt          (free)│\n                ├─▶  POST /scrape         ($0.001) │\n                ├─▶  POST /extract        ($0.001) │\n                ├─▶  POST /screenshot     ($0.001) │\n                ├─▶  POST /metadata       ($0.001) │\n                ├─▶  POST /workflow       ($0.001) │\n                ├─▶  POST /session        ($0.001) │\n                │                                  │\nAI Agent ───────┼─▶  POST /mcp                     │\n(MCP)           │      • initialize                │\n                │      • tools/list                │\n                │      • tools/call (paid)         │\n                │                                  │\n                └──────────────┬───────────────────┘\n                               │\n              ┌────────────────┴─────────────────┐\n              │                                  │\n   Coinbase CDP facilitator       Browser Rendering API\n   (api.cdp.coinbase.com/         Groq API (Llama 4 Scout)\n    platform/v2/x402)             KV (sessions, free tier)\n   Ed25519 JWT auth\n   Settles USDC on Base\n              │\n              ▼\n   payTo wallet 0x3F33...9E9a\n```\n\n**Stack:**\n\n- **Runtime:** Cloudflare Workers (V8 isolates, global edge)\n- **HTTP framework:** [Hono](https://hono.dev) + [@x402/hono](https://github.com/coinbase/x402) v2 payment middleware\n- **MCP framework:** [@modelcontextprotocol/sdk](https://github.com/modelcontextprotocol/typescript-sdk) + [agents/mcp](https://github.com/cloudflare/agents) with `withX402` payment binding\n- **Bazaar discovery:** [@x402/extensions](https://github.com/coinbase/x402) — every paid route declares input schema + output example\n- **Browser:** Cloudflare Browser Rendering API + [@cloudflare/puppeteer](https://developers.cloudflare.com/browser-rendering/)\n- **AI extraction:** [Groq](https://groq.com) + Llama 4 Scout (17B, 16E)\n- **Storage:** Cloudflare KV (5-min response cache, browser sessions, free-tier counters)\n- **Network:** Base mainnet (`eip155:8453`)\n- **Asset:** Native USDC (`0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913`)\n- **Facilitator:** Coinbase CDP (`api.cdp.coinbase.com/platform/v2/x402`) with Ed25519 JWT auth\n\n---\n\n## Free tier\n\nEvery wallet address gets **10 free calls per 30 days** on the HTTP API. Pass your Base wallet in the `X-402-Payer` header. After 10 calls, payment via x402 is required.\n\nFree tier is HTTP-only. MCP calls go through the standard x402 payment flow.\n\n---\n\n## Pricing\n\nLaunch window (first 48h on mainnet): **$0.001 flat per tool call.**\n\nAfter validation, pricing ramps to:\n\n| Tool | Production price |\n|---|---|\n| `scrape_webpage` | $0.003 |\n| `extract_structured_data` | $0.005 |\n| `screenshot_webpage` | $0.003 |\n| `extract_metadata` | $0.002 |\n| `run_workflow` | $0.008 |\n| `create_browser_session` | $0.001 |\n\nBulk discounts (20% at 1K calls/month, 40% at 10K) are planned.\n\n---\n\n## Why x402 + MCP\n\n| Traditional API | x402 + MCP |\n|---|---|\n| Sign up at scraperapi.com | Discover via MCP `initialize` |\n| Add credit card | Wallet holds USDC |\n| Get API key | No keys |\n| $49/month minimum | $0.001 per call |\n| Manage rate limits | Pay-per-use, no limits |\n| Human in the loop | Agent fully autonomous |\n\n---\n\n## Verifying the deployment\n\n```bash\n# Service identity\ncurl https://agent-scrape.healingsunhaven.workers.dev/ | jq .\n\n# x402 manifest\ncurl https://agent-scrape.healingsunhaven.workers.dev/.well-known/x402 | jq .\n\n# 402 challenge with x402 v2 headers\ncurl -i -X POST https://agent-scrape.healingsunhaven.workers.dev/scrape \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"url\":\"https://example.com\"}'\n# → HTTP 402, Payment-Required header (base64 JSON), Cache-Control: private, no-store\n\n# CORS preflight (browser-based agents)\ncurl -i -X OPTIONS https://agent-scrape.healingsunhaven.workers.dev/scrape \\\n  -H \"Origin: https://example.com\" \\\n  -H \"Access-Control-Request-Method: POST\" \\\n  -H \"Access-Control-Request-Headers: Content-Type,X-Payment\"\n# → HTTP 204, allow-headers includes X-Payment and Payment-Signature\n```\n\n---\n\n## Roadmap\n\n**Q3 2026:**\n\n- **AgentParse** — document/PDF/OCR-to-markdown service (same x402 + MCP pattern)\n- **AgentSearch** — multi-backend web search aggregation\n- LangChain integration package (`langchain-agent-scrape`)\n\n**Q4 2026:**\n\n- Subscription bundles (1000 scrapes for $2.50)\n- Multi-asset support (USDT on Base, USDC on Arbitrum)\n- Webhook + async response option for long-running workflows\n\n---\n\n## Run locally\n\nRequires Node 22+, a Cloudflare account with Workers + Browser Rendering enabled, a Groq API key, and a Coinbase CDP API key.\n\n```bash\ngit clone https://github.com/hshintelligence/agent-scrape.git\ncd agent-scrape\nnpm install\n\n# Set secrets\nnpx wrangler secret put GROQ_API_KEY\nnpx wrangler secret put CDP_API_KEY_ID\nnpx wrangler secret put CDP_API_KEY_SECRET\n\n# Local dev\nnpx wrangler dev\n\n# Deploy\nnpx wrangler deploy\n```\n\nTo use your own wallet as `payTo`, edit the `PAY_TO` constant in `src/index.ts`.\n\n---\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n\n---\n\n## Contact\n\nBuilt by [HSH Intelligence](https://hshintelligence.com), operated by Healing Sun Haven LLC (Wyoming, USA).\n\n- **Service:** https://agent-scrape.healingsunhaven.workers.dev\n- **Repo:** https://github.com/hshintelligence/agent-scrape\n- **General inquiries:** sales@healingsunhaven.com\n\n---\n\n**Pay-per-call web scraping for AI agents — no signup, no API keys, just USDC.**\n\n\n## Using AgentScrape from ElizaOS\n\nElizaOS supports remote MCP servers natively via the [`@elizaos/plugin-mcp`](https://www.npmjs.com/package/@elizaos/plugin-mcp) plugin. AgentScrape works out of the box — no custom integration needed.\n\n### Install the MCP plugin\n\n```bash\nbun add @elizaos/plugin-mcp\n# or\nnpm install @elizaos/plugin-mcp\n```\n\n### Configure your character to use AgentScrape\n\nIn your ElizaOS character JSON, add the plugin and point the MCP server config at AgentScrape's Streamable HTTP endpoint:\n\n```json\n{\n  \"name\": \"ResearchAgent\",\n  \"plugins\": [\"@elizaos/plugin-mcp\"],\n  \"settings\": {\n    \"mcp\": {\n      \"servers\": {\n        \"agentscrape\": {\n          \"type\": \"streamable-http\",\n          \"name\": \"AgentScrape\",\n          \"url\": \"https://agent-scrape.healingsunhaven.workers.dev/mcp\",\n          \"timeout\": 60\n        }\n      }\n    }\n  }\n}\n```\n\n### Available tools (auto-discovered by ElizaOS)\n\nOnce configured, the agent automatically discovers all six AgentScrape tools:\n\n| Tool | Cost | Purpose |\n|---|---|---|\n| `scrape_webpage` | $0.003 USDC | Markdown/HTML/text/JSON scrape |\n| `extract_structured_data` | $0.005 USDC | AI extraction via Groq + Llama 4 Scout |\n| `screenshot_webpage` | $0.003 USDC | PNG screenshot with viewport control |\n| `extract_metadata` | $0.002 USDC | Title, OG, Twitter, JSON-LD |\n| `create_browser_session` | $0.001 USDC | Stateful browser session |\n| `run_workflow` | $0.008 USDC | Multi-step atomic workflow up to 20 steps |\n\n### Free tier\n\nThe first 10 calls per wallet in the first 30 days are free. After that, calls are paid in USDC on Base mainnet (`eip155:8453`) via the x402 v2 payment protocol. To pay, supply the `X-PAYMENT-RESPONSE` header with an x402 payment receipt on retry — the standard x402 flow.\n\n### Example prompt\n\nOnce the plugin is wired up, your ElizaOS agent can use AgentScrape transparently:\n\n> User: \"Scrape https://www.x402.org and give me a two-sentence summary.\"\n\nElizaOS routes the request through `@elizaos/plugin-mcp`, which calls AgentScrape's `scrape_webpage` tool, and the agent answers based on the live page content.\n\n### Resources\n\n- AgentScrape Worker: https://agent-scrape.healingsunhaven.workers.dev\n- x402 manifest: https://agent-scrape.healingsunhaven.workers.dev/.well-known/x402.json\n- A2A Agent Card: https://agent-scrape.healingsunhaven.workers.dev/.well-known/agent.json\n- Source: https://github.com/hshintelligence/agent-scrape\n",
  "bytes": 12745,
  "sha": "8795b8322cdfaa1c2221ed36641b581250f738db7b1dcc0a8d9b5c2507dddd0b",
  "repo_slug": "hshintelligence/agent-scrape",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_hshintelligence_agent_scrape_cb85b44b/readme"
}