{
  "markdown": "# agent-web\n\n**URL → LLM-ready markdown.** A charter-clean, *polite, robots-respecting* web reader for AI\nagents — honors `robots.txt`, identifies honestly, and **never** bypasses anti-bot/CAPTCHA or\npaywalls. No account, no API key — free per-call access.\n\n🌐 Live: **https://agent-web.foomworks.workers.dev** · 🔌 MCP: **`https://agent-web.foomworks.workers.dev/mcp`**\n\n---\n\n## What it gives an agent\n\nGive it one publicly reachable URL; get back the page as clean, **LLM-ready markdown** —\ntitle, word count, and the text — without parsing raw HTML yourself. Server-rendered (static)\nHTML pages. Ideal for summarizing an article, extracting docs, or feeding a page into an LLM.\n\n## Connect as an MCP server (recommended)\n\nStreamable HTTP, JSON-RPC 2.0:\n\n```\nhttps://agent-web.foomworks.workers.dev/mcp\n```\n\n| Tool | Cost | Returns |\n|---|---|---|\n| `read_url` | free | fetch a URL → full LLM-ready markdown (title, word count, markdown) |\n| `read_url_preview` | free | the first ~600 characters of the markdown — a cheap look before the full pull |\n\n## Or call it over HTTP\n\n```bash\nBASE=https://agent-web.foomworks.workers.dev\ncurl -s \"$BASE/read?url=https://example.com/&ref=readme\"          # full markdown\ncurl -s \"$BASE/read/preview?url=https://example.com/&ref=readme\"  # first ~600 chars\n```\n\nDiscovery: [`/.well-known/mcp.json`](https://agent-web.foomworks.workers.dev/.well-known/mcp.json) (MCP) ·\n[`/.well-known/agent-card.json`](https://agent-web.foomworks.workers.dev/.well-known/agent-card.json) (A2A) ·\n[`/openapi.json`](https://agent-web.foomworks.workers.dev/openapi.json) ·\n[`/policy`](https://agent-web.foomworks.workers.dev/policy) (Acceptable-Use Policy)\n\n## Charter-clean by design (the bright line)\n\nThis is a *polite* reader, not a scraper:\n\n- Honors the origin's `robots.txt` for our user-agent — a disallowed path is **refused without fetching**; an unconfirmable `robots.txt` (5xx/error) is treated conservatively as disallowed.\n- Identifies honestly with a descriptive user-agent — **never** spoofs a browser.\n- Read-only `GET` of the **single** URL you supply — no crawl, no proxy rotation.\n- **Never** bypasses anti-bot/CAPTCHA, paywalls, or login walls.\n- SSRF-guarded: private/loopback/link-local/internal hosts are blocked and every redirect hop is re-validated.\n\nYou are responsible for your right to fetch the URL you submit.\n\n## Limits\n\n- **Static HTML only** for now — JavaScript-rendered pages, screenshots, and PDF are a later increment.\n- Body cap ~2 MB, fetch timeout ~12 s.\n- The service holds no keys and never initiates payments.\n\n## Install the AgentSkill\n\n[`SKILL.md`](./SKILL.md) is a portable AgentSkill (ClawHub / skills.sh / Claude Skills).\n\n## About\n\nagent-web is operated by **FOOM** — an AI-operated, human-supervised autonomous service.\nLicense: [MIT](./LICENSE).\n",
  "bytes": 2820,
  "sha": "2ddb3ad0ca181fdd71600b8bbc1898df50371999b3e2c0179b1151bed0a7fea3",
  "repo_slug": "foomworks/agent-web",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_foomworks_agent_web_cfaea37c/readme"
}