{
  "markdown": "<!-- mcp-name: io.github.Retio-ai/pagemap -->\n\n# PageMap\n\nPageMap converts raw HTML (100K+ tokens) into structured, AI-readable page maps (2-5K tokens) — a **97% token reduction**. It works as an MCP server, Python SDK, and CLI, supporting 16 page types and 30+ e-commerce sites. Agents can read, click, type, and navigate any web page.\n\n> *\"Give your agent eyes and hands on the web.\"*\n\n[![CI](https://github.com/Retio-ai/Retio-pagemap/actions/workflows/ci.yml/badge.svg)](https://github.com/Retio-ai/Retio-pagemap/actions/workflows/ci.yml)\n[![PyPI](https://img.shields.io/pypi/v/retio-pagemap)](https://pypi.org/project/retio-pagemap/)\n[![Python](https://img.shields.io/pypi/pyversions/retio-pagemap)](https://pypi.org/project/retio-pagemap/)\n[![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)\n[![Docker](https://img.shields.io/docker/v/retio1001/pagemap?label=Docker)](https://hub.docker.com/r/retio1001/pagemap)\n[![Awesome MCP Servers](https://img.shields.io/badge/Awesome-MCP%20Servers-fc60a8?logo=awesomelists&logoColor=white)](https://github.com/punkpeye/awesome-mcp-servers)\n\n---\n\n<!-- ============================================================ -->\n<!--  HUMAN GUIDE                                                  -->\n<!-- ============================================================ -->\n\n## Why PageMap?\n\nPlaywright MCP dumps 50-540KB accessibility snapshots per page, overflowing context windows after 2-3 navigations. Firecrawl and Jina convert HTML to markdown — read-only, no interaction.\n\nPageMap gives your agent a **compressed, actionable** view of any web page:\n\n| | PageMap | Playwright MCP | Firecrawl | Jina Reader |\n|--|:------:|:---------:|:-----------:|:--------:|\n| **Tokens / page** | **2-5K** | 6-50K | 10-50K | 10-50K |\n| **Interaction** | **click / type / select / hover** | Raw tree parsing | Read-only | Read-only |\n| **Multi-page sessions** | **Unlimited** | Breaks at 2-3 pages | N/A | N/A |\n| **Task success (94 tasks)** | **84.7%** | 61.5% | 64.5% | 57.8% |\n| **Avg tokens / task** | **2,710** | 13,737 | 13,888 | 11,424 |\n| **Cost / 94 tasks** | **$1.06** | $4.09 | $3.98 | $2.26 |\n\n> Benchmarked across 11 e-commerce sites, 94 static tasks, 7 conditions. 8,100+ tests passing.\n\n---\n\n## Quick Start\n\nChromium is auto-installed on first use — no manual `playwright install` needed.\n\n### Install\n\n```bash\npip install retio-pagemap\n```\n\n### MCP Client Config\n\nAdd to Claude Code, Cursor, Windsurf, or Claude Desktop:\n\n```json\n{\n  \"mcpServers\": {\n    \"pagemap\": {\n      \"command\": \"uvx\",\n      \"args\": [\"retio-pagemap\"]\n    }\n  }\n}\n```\n\n> **Claude Desktop (macOS)**: Use the absolute path to `uvx` — run `which uvx` (e.g. `/opt/homebrew/bin/uvx`).\n\n> **VS Code (Copilot)**: Use `\"servers\"` instead of `\"mcpServers\"` in `.vscode/mcp.json`.\n\n### Docker\n\n```bash\ndocker run -p 8000:8000 retio1001/pagemap --transport http\n```\n\n---\n\n## Features\n\n### 13 MCP Tools — Read + Interact\n\nNot just reading — your agent can click buttons, fill forms, select options, manage tabs, and navigate across pages. 13 tools cover the full browsing workflow:\n\n`get_page_map` · `execute_action` · `fill_form` · `scroll_page` · `wait_for` · `take_screenshot` · `get_page_state` · `navigate_back` · `batch_get_page_map` · `open_tab` · `switch_tab` · `list_tabs` · `close_tab`\n\n### 16 Page Types, Auto-Detected\n\nPageMap automatically classifies pages and applies optimized extraction for each type:\n\n`product_detail` · `listing` · `search_results` · `article` · `news` · `video` · `login` · `form` · `checkout` · `dashboard` · `help_faq` · `settings` · `error` · `documentation` · `landing` · `blocked`\n\n### E-Commerce Deep Coverage\n\nBuilt-in support for **30+ major e-commerce sites** across 4 tiers:\n\n- **Global mega-platforms** — Amazon, eBay, AliExpress, SHEIN, Walmart, Rakuten\n- **Global fashion** — Zara, H&M, Nike, Uniqlo, ASOS, Zalando, SSENSE, Farfetch, COS\n- **Korea** — Coupang, Naver Shopping, Musinsa, 29CM, W Concept, SSG, 11st\n- **Japan/China** — ZOZO, Tmall, JD.com, Taobao\n\nStructured extraction of prices, options (size/color), ratings, availability — with automatic cookie consent handling and login barrier detection.\n\n### Smart Recovery\n\nPageMap detects problems and tells your agent what to do:\n\n- **Barrier detection** — Login required? Bot blocked? Out of stock? Age verification? Popup overlay? PageMap adds a `barrier` field with the diagnosis and suggested next steps\n- **Cookie consent auto-dismiss** — 7 CMP providers auto-detected (Cookiebot, OneTrust, TrustArc, Didomi, Quantcast, Usercentrics, generic fallback). 5-tier dismiss cascade: CMP JS API → Reject → Accept → Dismiss → Close symbol. GDPR reject-first default policy\n- **Popup overlay detection** — AX tree `role=\"dialog\"` + HTML regex 2-phase detection. Promotional popups (newsletter, exit-intent) auto-dismissed\n- **Bot detection awareness** — Detects Cloudflare, Turnstile, reCAPTCHA, hCaptcha, and Akamai. Reports the provider and suggests wait/retry strategies\n- **Stale ref recovery** — When DOM changes invalidate refs, PageMap returns clear guidance to re-fetch\n\n### Content Intelligence\n\n- **8 JSON-LD schemas** — Product, NewsArticle, VideoObject, FAQPage, Event, LocalBusiness, BreadcrumbList, and ItemList\n- **Metadata extraction** — Prices, ratings, reviews, descriptions, images from structured data and DOM fallbacks\n- **2-layer caching** — Cache hit (~10ms), content refresh (~500ms), full rebuild (~1.5s). Diff-based updates for unchanged sections\n- **Delta evidence packet output** - Optional `to_delta_packet()` serializer emits digest-bound evidence units, claim candidates, provenance, and authority flags for downstream memory/review systems without changing the default MCP output\n\n### 10 Languages\n\nLocale auto-detected from URL. Token budgets adjusted for CJK scripts.\n\n| Language | Locale | Language | Locale |\n|----------|:------:|----------|:------:|\n| English | `en` | Chinese | `zh` |\n| Korean | `ko` | Spanish | `es` |\n| Japanese | `ja` | Italian | `it` |\n| French | `fr` | Portuguese | `pt` |\n| German | `de` | Dutch | `nl` |\n\n---\n\n## Deployment\n\n### Local (STDIO)\n\nDefault mode. Runs as a local MCP server — no server setup needed.\n\n```bash\nretio-pagemap\n```\n\n### Docker\n\n```bash\ndocker run -p 8000:8000 retio1001/pagemap --transport http\n```\n\nMulti-architecture images (amd64/arm64) available on [Docker Hub](https://hub.docker.com/r/retio1001/pagemap) and GitHub Container Registry.\n\n---\n\n## Python API\n\n```python\nimport asyncio\nfrom pagemap.browser_session import BrowserSession\nfrom pagemap.delta_serializer import to_delta_packet\nfrom pagemap.page_map_builder import build_page_map_live\nfrom pagemap.serializer import to_agent_prompt, to_json\n\nasync def main():\n    async with BrowserSession() as session:\n        page_map = await build_page_map_live(session, \"https://example.com/product/123\")\n        print(to_agent_prompt(page_map))   # Agent-optimized text format\n        print(to_json(page_map))           # Structured JSON\n        print(to_delta_packet(page_map))   # Digest-bound evidence packet\n        print(page_map.page_type)          # \"product_detail\"\n        print(page_map.interactables)      # [Interactable(ref=1, role=\"button\", ...)]\n        print(page_map.metadata)           # {\"name\": \"...\", \"price\": \"...\"}\n\nasyncio.run(main())\n```\n\nFor offline processing (no browser):\n\n```python\nfrom pagemap.page_map_builder import build_page_map_offline\n\npage_map = build_page_map_offline(open(\"page.html\").read(), url=\"https://example.com/product/123\")\n```\n\n---\n\n## Security\n\nPageMap treats all web content as untrusted input:\n\n- **SSRF defense** — Multi-layer protection against server-side request forgery\n- **Prompt injection defense** — Content boundaries, role-prefix stripping, suspicious content flagging\n- **robots.txt compliance** — RFC 9309 compliant. `--ignore-robots` opt-out flag\n- **Resource guards** — DOM node limit, HTML size limit, response size limit\n- **Session isolation** — Each session has independent cookies and storage, automatically cleaned up\n\n**Local development**: Private IPs are blocked by default. Use `--allow-local` or `PAGEMAP_ALLOW_LOCAL=1`.\n\n### Disclaimer\n\nUsers are responsible for complying with the terms of service of target websites and all applicable laws when using PageMap.\n\n---\n\n## Troubleshooting\n\n**\"spawn uvx ENOENT\" (Claude Desktop on macOS)** — Claude Desktop does not inherit your shell PATH. Run `which uvx` and use the absolute path in your config.\n\n**First page takes a long time** — Chromium cold start takes ~10-30s on first navigation. Subsequent pages load in 1-3 seconds.\n\n**Localhost blocked** — Use `--allow-local` flag or set `PAGEMAP_ALLOW_LOCAL=1`.\n\n**Chromium not found** — Run `pip install retio-pagemap && playwright install chromium` to install manually.\n\n---\n\n## Requirements\n\n- Python 3.11+\n- Chromium (auto-installed on first use)\n\n## Community\n\nHave a question or idea? Join the conversation in [GitHub Discussions](https://github.com/Retio-ai/Retio-pagemap/discussions).\n\n## Development\n\n[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/Retio-ai/Retio-pagemap?quickstart=1)\n\n```bash\ngit clone https://github.com/Retio-ai/Retio-pagemap.git\ncd Retio-pagemap\nuv sync --group dev\nplaywright install chromium\nuv run pytest --tb=short -q\n```\n\n## Pricing\n\n**Local (STDIO)** — Free forever. Self-hosted, open source under AGPL-3.0.\n\n**Cloud API** — Hosted multi-tenant server with auth, rate limiting, and credit-based billing. Contact **retio1001@retio.ai** for access.\n\n## License\n\nAGPL-3.0-only — see [LICENSE](LICENSE) for the full text.\n\nFor commercial licensing options, contact **retio1001@retio.ai**.\n\n---\n\n<!-- ============================================================ -->\n<!--  AGENT REFERENCE                                              -->\n<!-- ============================================================ -->\n\n## For Agents\n\n*This section is written for AI agents using PageMap as an MCP tool.*\n\n### Tools\n\n| Tool | When to use |\n|------|-------------|\n| `get_page_map` | **Start here.** Navigate to a URL and get a full structured map with numbered refs. |\n| `execute_action` | Click, type, select, or hover using a ref number from the last `get_page_map`. |\n| `fill_form` | Fill multiple form fields in one call. More efficient than sequential `execute_action` calls. |\n| `get_page_state` | Check current URL and title without a full rebuild. Use after actions that may navigate. |\n| `scroll_page` | Scroll to reveal lazy-loaded content before calling `get_page_map` again. |\n| `wait_for` | Wait for dynamic content to appear (e.g. after a search or form submit). |\n| `take_screenshot` | Capture the visual state when the PageMap alone is ambiguous. |\n| `navigate_back` | Go back one step in browser history. |\n| `open_tab` | Open a new browser tab and navigate to a URL. |\n| `switch_tab` | Switch to a different open tab by index. |\n| `list_tabs` | List all open tabs with their URLs and titles. |\n| `close_tab` | Close a tab by index. |\n| `batch_get_page_map` | Fetch multiple URLs in parallel. Use for comparison tasks. |\n\n### Output Format\n\n```yaml\nURL: https://example.com/product/123\nTitle: Product Name\nType: product_detail          # auto-detected page type\n\n## Actions\n[1] button: Add to cart (click)\n[2] select: Size (select) — options: S, M, L, XL\n[3] link: See all reviews (click)\n...\n\n## Info\nPrice: $49.99\nRating: 4.5 / 5 (128 reviews)\nDescription: ...\n\n## Images\n  [1] https://cdn.example.com/product.jpg\n\n## Meta\nTokens: ~1,800 | Interactables: 24 | Generation: 380ms\n```\n\n- **`## Actions`** — Every interactive element on the page with a stable `ref` number.\n- **`## Info`** — Key page content extracted from HTML: prices, titles, ratings, descriptions.\n- **`## Images`** — Product/content image URLs.\n- **`## Meta`** — Token count, interactable count, generation time.\n\n### Barrier Detection\n\nWhen PageMap encounters a page-level obstacle, it includes a `barrier` field in the response:\n\n```yaml\nState:\n  barrier: login_required\n  barrier_hint: \"Login form detected with email + password fields. Use fill_form to authenticate.\"\n```\n\nPossible barriers: `cookie_consent`, `login_required`, `bot_blocked`, `out_of_stock`, `empty_results`, `error_page`, `age_verification`, `region_restricted`, `popup_overlay`.\n\n**When you see a barrier:** follow the `barrier_hint` guidance. For `bot_blocked`, wait and retry. For `login_required`, use `fill_form` with credentials.\n\n### Ref Lifecycle\n\nRefs are assigned by `get_page_map` and remain valid until the page state changes.\n\n**Refs are invalidated when:**\n- The page navigates to a new URL\n- A DOM mutation occurs (modal opens, SPA navigation, accordion toggles)\n- `execute_action` causes a page-level change\n\n**When you get a stale ref error:** call `get_page_map` again to get fresh refs before retrying.\n\n### Token Budget Behavior\n\nWhen a page exceeds the token budget, content is pruned in this order:\n1. Navigation menus, footers, sidebars removed first\n2. Secondary body content trimmed\n3. `## Actions` and `## Info` are always preserved\n\nIf key content seems missing, try `scroll_page` to load lazy content, then `get_page_map` again.\n\n### Recommended Workflow\n\n```\n1. get_page_map(url)          → read Actions + Info, pick refs\n2. execute_action(ref, ...)   → interact\n3. get_page_state()           → confirm navigation occurred\n4. get_page_map(new_url)      → get fresh refs for next step\n```\n\nFor pages with dynamic content (search results, filters):\n```\n1. get_page_map(url)\n2. execute_action(ref, \"click\")    → trigger search/filter\n3. wait_for(text=\"results\")        → wait for content\n4. get_page_map(url)               → get updated map\n```\n\n### Known Limitations\n\n- **Login-gated pages** — PageMap does not manage sessions or cookies. Authentication must be handled externally.\n- **Heavy bot detection** (Cloudflare, Akamai) — May block automated access. PageMap detects the provider and suggests strategies, but cannot bypass active bot mitigation.\n- **Private network access** — Blocked by default. Requires `--allow-local` flag.\n- **iframes** — Cross-origin iframes are not accessible due to browser security policies.\n\n---\n\n*PageMap — Structured Web Intelligence for the Agent Era.*\n",
  "bytes": 14315,
  "sha": "9422a76e5c8cdeaf471907661c99a28b1df078ebb6e5dc9d9014d451a6f84397",
  "repo_slug": "retio-ai/retio-pagemap",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_retio_ai_pagemap_c8c0eaca/readme"
}