{
  "markdown": "# Groundhog\n\n[![Conformance](https://github.com/dmytrome/groundhog/actions/workflows/conformance.yml/badge.svg)](https://github.com/dmytrome/groundhog/actions/workflows/conformance.yml)\n[![Glama](https://glama.ai/mcp/servers/dmytrome/groundhog/badges/score.svg)](https://glama.ai/mcp/servers/dmytrome/groundhog)\n\n**Web search, read and research for AI agents — through a real, stealth-patched Chrome.**\nGroundhog is an [MCP](https://modelcontextprotocol.io) server that finds pages, reads them,\nand researches across them, returning clean Markdown a model can trust: text no human could\nsee is **stripped by default before the model reads it**, every source comes back with a\n**provenance receipt**, and a real browser reads pages that block plain fetchers — without\nthe SSRF holes of naive fetch tools.\n\n```text\nagent / crawler  ──MCP──▶  Groundhog (search, read_url, research)  ──CDP──▶  stealth Chrome  ──▶  the web\n```\n\n## Measured against other fetch layers\n\nEighteen pages, each carrying text a human reader cannot see, plus one clean control. A\ncase passes when the payload does not reach the model **and** the article text still does —\na fetcher that returns nothing contains every payload and is useless.\n\n| | contained | kept the article | reported it |\n| --- | --- | --- | --- |\n| `requests` + trafilatura | 16/18 | 18/18 | 0/18 |\n| Scrapling (HTTP) | 10/18 | 18/18 | 0/18 |\n| Jina Reader | 8/18 | 18/18 | 0/18 |\n| Firecrawl | 4/18 | 18/18 | 0/18 |\n| **Groundhog** | **18/18** | **18/18** | **18/18** |\n\nContainment happens by accident all the time — an article extractor prunes a hidden `<div>`\nbecause its heuristics dislike it, not because anything asked whether a reader could see\nit. That is why the last column matters: **nothing else tested tells the caller that\nanything was removed**, so an empty threat list and a page with nothing hidden look\nidentical.\n\nThe corpus is published at\n[dmytrome.github.io/groundhog](https://dmytrome.github.io/groundhog/), so any fetcher can\nbe measured against the same pages. Harness, per-case results and the limits of what this\nproves are in [`benchmark/`](benchmark/) — including that hosted services are a snapshot of\nthe date recorded in [`RESULTS.md`](benchmark/RESULTS.md).\n\n## Quick start\n\nAdd Groundhog to your MCP client — that's it. On the first fetch, Groundhog pulls and\nstarts the stealth-browser container for you (Docker or Podman required); no repo checkout,\nno manual steps. When the default (non-compose) auto-start path has to run, any stale\ncontainer named `groundhog-browser` is removed first; a reachable browser is never touched.\n\nClaude Code:\n\n```bash\nclaude mcp add groundhog -- uvx groundhog-mcp\n```\n\nClaude Desktop / Cursor / Windsurf (`claude_desktop_config.json` or equivalent):\n\n```json\n{\n  \"mcpServers\": {\n    \"groundhog\": {\n      \"command\": \"uvx\",\n      \"args\": [\"groundhog-mcp\"]\n    }\n  }\n}\n```\n\n`uvx` fetches `groundhog-mcp` from PyPI on first run. The first fetch pulls the browser\nimage (once, a few minutes); later fetches are instant. No container runtime? The `status`\ntool and any error say how to install one — or point `CDP_URL` at a hosted browser for\nzero-install use.\n\n**Prefer to manage the browser yourself?** Start it and Groundhog will just use it:\n\n```bash\ndocker run -d --rm --name groundhog-browser --shm-size 512m \\\n  -p 127.0.0.1:9222:9222 -- ghcr.io/dmytrome/groundhog:latest\n# or, from a repo checkout: docker compose up --build -d\ncurl -s http://localhost:9222/json/version    # CDP is live\n```\n\nSet `GROUNDHOG_AUTO_START_BROWSER=false` to disable auto-start. To run the MCP server from\nsource: `cd mcp && uv sync && uv run groundhog-mcp`.\n\nAll four tools are annotated `readOnlyHint`, which is what lets a client run them without a\nper-call confirmation. That describes what they do to *your data*: nothing is written, and no\nremote state is changed. Worth knowing, because it is the one exception: with auto-start on,\nthe first call may pull and run the browser container, and remove an **unreachable** container\nnamed `groundhog-browser` first. A reachable one is never touched, and\n`GROUNDHOG_AUTO_START_BROWSER=false` turns the whole path off.\n\n## What makes it different\n\n- **Hidden text is stripped before the model reads it.** Groundhog renders a real DOM, so it\n  can judge what a *human* would actually see and strip what they could not, reporting each\n  occurrence in `threats`. A strong heuristic, not a proof — see\n  [the limits of hidden-text detection](#limits-of-hidden-text-detection), and\n  [the benchmark](#measured-against-other-fetch-layers) for how that compares. The twelve signals,\n  the `threats` caveat and the `include_hidden` exception are documented under `read_url`.\n- **Every source carries a receipt.** SHA-256 hash of the extracted content, canonical URL,\n  language, word count, and author/date when the page declares them — so a downstream claim\n  traces back to exactly what was read. `read_url` returns the fetch time alongside it as\n  `fetched_at`.\n- **Safe by default.** The SSRF guard resolves each host before navigating and refuses to\n  return content from a URL that redirects into a private address. Read-only, with per-domain\n  rate limiting. This matters most in `research`, where a *third party* chooses the URLs.\n  See [Security](#security) for the full blocklist and the guard's limits.\n- **No automation tell.** Puppeteer/Playwright/Selenium enable the CDP `Runtime` domain,\n  which anti-bots detect (`isAutomatedWithCDP`). Groundhog drives the browser over raw CDP\n  and never enables `Runtime`/`Console`, so that signal is absent — a clean session that\n  full automation libraries can't produce over `connect_over_cdp`.\n- **A real fingerprint.** It's real Chrome, run headful under Xvfb (no `HeadlessChrome`\n  token) — authentic TLS/HTTP2 fingerprint, real WebGL/canvas — not a Python HTTP client,\n  so fingerprint-driven blocks go away and cheap proxies work where they otherwise wouldn't.\n- **No model, no API key.** `research` returns extracts, not summaries; your agent does the\n  synthesis. Self-hosted and MIT — the pages you fetch never leave your infrastructure.\n\n## Tools\n\n### `read_url(url, format=\"markdown\", max_tokens=None, query=None, include_hidden=False)`\n\nFetches a page and returns clean content plus provenance.\n\n| Key          | Meaning                                                                                          |\n| ------------ | ------------------------------------------------------------------------------------------------ |\n| `markdown`   | Extracted content (article-first, falls back to full text); `format` may be `markdown` or `text` |\n| `title`      | Page title                                                                                       |\n| `url`        | The URL you asked for                                                                            |\n| `final_url`  | The URL after redirects (re-checked against the SSRF guard). Never rewritten: if the page's own final URL is unusable, the requested URL is reported and a `final_url_suppressed` threat says so |\n| `fetched_at` | UTC ISO-8601 timestamp                                                                           |\n| `status`     | What actually came back: `ok`, or `challenge` / `blocked` / `rate_limited` / `not_found` / `server_error` / `unsupported_content` when the content is not the page you asked for — so a Cloudflare interstitial, a 403, or a PDF is not read as if it were the real page. `blocked` covers `401`/`403` and every other 4xx that serves an error page (`451`, `400`, `405`…), with the exact code in `http_status`. `unknown` means no response — or no usable status — was observed for the document that was read: it is reported rather than assumed to be fine. A `challenge` is recognised from vendor **mitigation markers** — a header that exists only to announce it, or a request for an asset only a challenge loads — so it works whatever language the page is in; page wording is a last resort and only counts on a page too empty to be content. See [the limits of block detection](#limits-of-block-detection) The verdict describes the document the text came from, so a page that redirects client-side (meta-refresh, `location.href`) is judged on where it landed, not where it started |\n| `http_status`| The top-level response's HTTP status code, or `null` when it could not be observed              |\n| `truncated`  | Whether the content was cut to fit the token budget                                              |\n| `threats`    | Signals detected: hidden-CSS nodes and invisible-character classes; empty when none found |\n| `matches`    | When `query` is set: ranked passages with `heading`, `offset`, and `score` for citation          |\n| `provenance` | Content hash, canonical URL, language, word count, and author/date metadata when present         |\n\nBecause Groundhog renders a real DOM, it can evaluate computed styles. Text invisible to\nhumans is **stripped by default** and each occurrence reported in `threats` with its signal\ntype and a short excerpt: `display:none`/`visibility:hidden`, `content-visibility: hidden`\n(the subtree is skipped from layout while the element keeps an ordinary box, so no other\nsignal sees it), `opacity ≤ 0.05`, `font-size < 4 px`, zero-size elements, an element that generates no box\nof its own (`display: contents`) whose contents render nothing, the sub-pixel box\nused by `.sr-only`/`.visually-hidden` accessibility utility classes (a pattern attackers now\nmimic), the legacy `clip: rect(...)` hiding technique, fully transparent text color, text\ncolor matching the background color (near-1:1 contrast), elements positioned entirely\noutside the rendered page (e.g. `left: -9999px`), and content the browser lays out but\nrefuses to render — a collapsed `<details>`, whose hiding lives on a pseudo-element no\nper-element style read can see. Non-trivial HTML comments are reported too — they never reach the\nextracted content either way, but a page embedding instructions this way is worth knowing\nabout. `<template>` content is treated the same and for a sharper reason: it never renders,\nso no reader and no computed style sees it, yet it *is* serialized into the markup the\nextractor reads, and what a component actually shows is the copy it clones into its shadow\ntree — which is read from the live page instead. Text carried in an `alt`, `aria-label`, `aria-description` or `title` attribute is\ncleared from the markup at any length and reported above a length threshold: no computed\nstyle hides it, so the signals above cannot see it, and every image on a page carries an\n`alt` — reporting each one would spend the cap on captions. A second, character-level class is stripped and reported alongside these: zero-width\ncharacters, bidi marks and RTL overrides, and the Unicode Tag block — an invisible ASCII\nmirror that is the canonical prompt-injection smuggling channel. Pass `include_hidden=True`\nto keep the stripped text in the output; `threats` is still populated so you know it was\nthere.\n\n**Treat `threats` as untrusted.** Entries come in eight shapes (the character classes share one):\n\n| `type` | Carries |\n| ------ | ------- |\n| `hidden_css` | The hiding `reason`, an 80-char `excerpt` of the removed text, and the DOM `location`. All three are page-authored, so all three are stripped of invisible characters and length-capped — but they remain attacker-*chosen* text |\n| `hidden_attribute` | The same three fields, for text carried in an `alt`, `aria-label`, `aria-description` or `title` attribute rather than hidden by style. Reported above a length threshold, since every image carries an `alt`; cleared from the markup at any length |\n| `hidden_template` | The same three fields, for text inside a `<template>`. Inert markup: it never renders, never reaches `innerText`, and no computed style applies to it — but it is serialized into the markup the extractor reads. Reported above a length threshold; emptied from the markup either way |\n| `zero_width` / `bidi` / `tag` | A codepoint and count in `reason`, no excerpt. Detected on the text the page actually served — the extractor removes these characters on its way to Markdown, so scanning the extracted output would report none of them |\n| `report_truncated` | How many entries were dropped when the cap was hit. Its own type, so it cannot be miscounted as a finding |\n| `final_url_suppressed` | The page's own final URL was unusable (over-long, or carrying invisible characters) and was not returned; `final_url` reports the URL you requested instead |\n| `detection_degraded` | The collector had to run in the page's own JavaScript world, where the page can replace the DOM APIs it uses. A short list proves nothing on that page |\n| `strip_incomplete` | The rendered text was rebuilt from markup rather than read from layout. Either a flagged node could not be removed outright — it won the cascade against the hiding stylesheet (an inline `!important` does) or its recorded position did not resolve — or the page renders through open shadow roots, whose content has no layout to read. The second is by far the more common, and is routine rather than adversarial |\n\nThe value of stripping is that the payload is out of the content being reasoned over, not\nthat it is invisible to the model. At most **50 findings per page** are returned (**10 per\nsource** in `research`, since the fan-out multiplies the report); beyond that a\n`report_truncated` entry is appended stating how many were dropped, rather than truncating\nsilently. The two classes are capped independently, so a page cannot bury the findings that\ncarry its injection excerpt by flooding the report with decoys of the other kind. Notices are\nappended after the cap — so they can never themselves be dropped, and a capped list is up to\n50 findings plus at most two notices.\n\nPass `query` to replace blunt head-truncation with relevance-ranked passage selection:\ncontent is chunked on markdown structure, ranked by lexical (BM25) relevance, and the top\npassages within the token budget are returned; `matches` gives each passage's heading,\ncharacter offset, and score for downstream citation. Ranking runs on the sanitized content,\nso hidden-text injection payloads cannot influence which passages surface — with the one\nexception of `include_hidden=True`, which leaves the hidden text in the document and ranks it\nalong with everything else.\n\n### `search(query, limit=10)`\n\nFinds pages for a query and returns ranked hits — `title`, `url`, `snippet`, `engine`,\n`score`, `published` — plus the `backend` that answered. Hits are links only: nothing is\nfetched until you pass a URL to `read_url`.\n\nTwo backends, chosen automatically. Set `SEARXNG_URL` to use your own\n[SearXNG](https://docs.searxng.org) instance (best results; needs `formats: [html, json]`\nin its `settings.yml`, since JSON is off by default upstream). With no instance configured,\nGroundhog renders a search page through the stealth browser instead — no extra\ninfrastructure, at the cost of depending on that page's layout. Force one with\n`GROUNDHOG_SEARCH_BACKEND=searxng|serp`.\n\nEvery text field of a hit is attacker-influenceable — a poisoned page controls how it describes\nitself — so each passes through the same invisible-character stripping as page content, and\neach is length-capped. The URL is treated differently: it is what a model cites, so it is\nnever rewritten. A hit is dropped outright if cleaning would change its URL at all, if that\nURL is not `http`/`https`, if it carries credentials, or if it exceeds 2048 characters. Both matter on the DuckDuckGo path, which percent-decodes\nthe redirect wrapper and can therefore turn `%E2%80%8B` back into a real zero-width\ncharacter inside the link. A backend that is unreachable, has JSON disabled, or whose every upstream engine\nis rate-limited raises an actionable error rather than reporting an empty web.\n\n### `research(query, max_sources=5, max_tokens=None)`\n\nOne call for \"find out about X\": searches, reads the top sources through the stealth\nbrowser, and returns the passages most relevant to `query` — ranked across **all** sources\nin a single pass, so a passage from source 4 competes fairly with one from source 1.\n\nReturns `passages` (each with `text`, `source_url`, `heading`, `score`) and `sources` (each\nwith `url`, `title`, `status`, `page_status`, `threats`, `provenance`). `status` is the fetch\noutcome (`ok` / `blocked` / `timeout` / `error`); `page_status` is what a page that loaded\nactually was — the same classification `read_url` reports, so a source that returned a\nbot-challenge or a non-HTML body is visible rather than passing as `ok` — and is `null` when\nthe fetch never produced a page. A source whose `page_status` says its body is an interstitial\nor an error page contributes **no passages**: it would otherwise compete for your token budget\nagainst real content. It still appears in `sources`, saying why it contributed nothing. At most\none page per registrable domain, for source diversity. Passages are extracts, not summaries — nothing is generated,\nand no model or API key is involved. When a passage isn't enough, `read_url` its\n`source_url` for the whole page.\n\nA source that fails doesn't fail the call: it appears in `sources` with a status of\n`blocked` (SSRF guard), `timeout`, or `error`, so a partial answer is still usable and you\ncan see what was missed. Because search results are chosen by a third party — and\nSEO-poisoned results are a documented in-the-wild attack — every fetched URL goes through\nthe same SSRF guard and hidden-text stripping as `read_url`, and each source reports what\nwas stripped from it. A source that failed carries `provenance: null` — only sources that\nwere actually read are hashed. `threats` is per-source here and capped at 10 entries per\nsource, lower than `read_url`'s 50, because the fan-out multiplies it. `max_sources` is\ncapped at 10.\n\nIt's slower than an API-backed research tool: a real browser renders every source. That's\nthe trade for reading pages that block plain fetchers, and for being able to tell you what\nwas hidden in them.\n\n### `status()`\n\nReports whether Groundhog can reach the stealth browser. Returns `browser_reachable`,\n`cdp_url` and a `hint` with remediation steps when it isn't reachable. The endpoint is\nreported as scheme, host and port only — a hosted browser often carries a credential in\nits URL, and this value reaches the model.\n\n## Configuration\n\n**MCP server** (`mcp/`):\n\n| Env var                          | Default                 | Purpose                                                                                  |\n| -------------------------------- | ----------------------- | ---------------------------------------------------------------------------------------- |\n| `CDP_URL`                        | `http://127.0.0.1:9222` | CDP endpoint of the stealth browser. May be remote (a DNS name or IP); auto-start is skipped for non-local values. The endpoint is unauthenticated — keep it on a private network or a tunnel. |\n| `GROUNDHOG_BLOCK_PRIVATE_IPS`    | `true`                  | Enforce the SSRF guard (resolve + block private ranges)                                  |\n| `GROUNDHOG_MIN_DELAY_MS`         | `5000`                  | Minimum delay between requests to the same domain                                        |\n| `GROUNDHOG_MAX_TOKENS`           | `20000`                 | Token budget before truncation                                                           |\n| `GROUNDHOG_MAX_CONCURRENT_PAGES` | `4`                     | Cap on concurrent open tabs                                                              |\n| `SEARXNG_URL`                    | _(unset)_               | Your SearXNG instance for `search`, e.g. `http://searxng:8080`. Needs `formats: [html, json]`. Unset → SERP via the stealth browser. |\n| `GROUNDHOG_SEARCH_BACKEND`       | `auto`                  | `auto` (SearXNG when `SEARXNG_URL` is set, else SERP), or force `searxng` / `serp`        |\n| `GROUNDHOG_AUTO_START_BROWSER`   | `true`                  | Auto-pull-and-run the browser container when it isn't reachable (needs Docker/Podman); `false` to manage it yourself |\n| `GROUNDHOG_BROWSER_IMAGE`        | `ghcr.io/dmytrome/groundhog:latest` | Image used for auto-start                                                    |\n| `GROUNDHOG_COMPOSE_FILE`         | _(none)_                | Use `docker compose -f <file> up -d` for auto-start instead of `docker run` (local repo) |\n\n**Dependencies:** `py3langid` (which pulls in numpy) is used for language detection in the\n`provenance` result. It is installed in the MCP server package only — not in the browser\ncontainer.\n\n**Browser container:**\n\n| Env var       | Default                       | Purpose                                                           |\n| ------------- | ----------------------------- | ----------------------------------------------------------------- |\n| `USER_AGENT`  | derived from installed Chrome | UA set at launch, so it is clean in every scope including workers        |\n| `PROXY`       | _(none)_                      | Upstream proxy (`http://user:pass@host:port`); auth is relayed and timezone/locale auto-align to the exit IP |\n| `TZ`          | `UTC`                         | Fallback timezone; auto-derived from the exit IP when `PROXY` is set     |\n| `WINDOW_SIZE` | `1920,1080`                   | Initial Chrome window size                                               |\n| `XVFB_WHD`    | `1920x1080x24`                | Virtual display geometry                                                 |\n\n## Under the hood: the stealth Chrome container\n\nA minimal Docker container running **headful Chrome under Xvfb** with a remote CDP\nendpoint. Any CDP-speaking client (Puppeteer, Playwright, Selenium, chromedp, raw\nDevTools) can drive it — Groundhog is one such client.\n\n- **Headful under Xvfb**, not `--headless=new` — the browser reports `Chrome`, not\n  `HeadlessChrome`, avoids headless-specific tells, and engages the real GPU path.\n- **`--disable-blink-features=AutomationControlled`** — `navigator.webdriver` reads\n  `false`.\n- **UA set at launch** from the installed Chrome version (`USER_AGENT`), so it is clean\n  in every scope — main frame, network, and Web/Service Worker globals.\n- **Proxy geo-coherence.** When `PROXY` is set, the entrypoint geolocates the exit IP and\n  aligns the browser timezone and locale to it — a timezone or locale that disagrees with\n  the IP is itself a block signal. The country→locale table is CLDR likely-subtags. Chrome\n  can't authenticate to a proxy over `--proxy-server`, so credentials are relayed through a\n  local tinyproxy; WebRTC is pinned to the proxy path so the real IP can't leak.\n- **GPU-aware WebGL.** The entrypoint auto-detects a GPU (NVIDIA via the Container\n  Toolkit, or Intel/AMD via `/dev/dri`) and uses hardware acceleration; without one it\n  runs Mesa `llvmpipe`, a coherent software renderer that VMs and servers legitimately\n  emit. See the `gpus`/`devices` hints in [`docker-compose.yml`](docker-compose.yml).\n\n### Verified results\n\nMeasured against a freshly built container (Chrome 151, headful under Xvfb, no proxy),\ndriven over raw CDP:\n\n| Detector                                                               | Result                                  |\n| ---------------------------------------------------------------------- | --------------------------------------- |\n| [deviceandbrowserinfo](https://deviceandbrowserinfo.com/are_you_a_bot) | not a bot (`isBot: false`, zero flags)  |\n| [browserscan](https://www.browserscan.net/bot-detection)               | Normal                                  |\n| [bot.sannysoft.com](https://bot.sannysoft.com/)                        | 31 / 31 checks pass                     |\n\n[iphey](https://iphey.com/) is tracked informationally, not pass/fail: its one recurring\nflag is Location (\"looks like you're trying to hide your location\"), which fires on any\ndatacenter/hosting exit IP regardless of browser fingerprint or `TZ` correctness — it\npasses on a residential IP and fails in CI (a cloud runner) and behind most proxies alike.\n\nSee [`RESULTS.md`](RESULTS.md) for the full live table (regenerated by\n[`tests/antibot.py`](tests/antibot.py) and the Conformance workflow).\n\nThese reflect the raw-CDP client. Full automation libraries (Puppeteer/Playwright/Selenium)\nenable the CDP `Runtime` domain and are flagged as automated even against this container —\nsee [`examples/`](examples/) for which need patched (rebrowser) variants.\n\n### Examples\n\n| Client              | Path                                                       |\n| ------------------- | ---------------------------------------------------------- |\n| Puppeteer (Node)    | [`examples/puppeteer`](examples/puppeteer)                 |\n| Playwright (Node)   | [`examples/playwright-node`](examples/playwright-node)     |\n| Playwright (Python) | [`examples/playwright-python`](examples/playwright-python) |\n| Selenium (Python)   | [`examples/selenium-python`](examples/selenium-python)     |\n| chromedp (Go)       | [`examples/go-chromedp`](examples/go-chromedp)             |\n| Raw CDP (Python)    | [`examples/python-raw-cdp`](examples/python-raw-cdp)       |\n\nSee [`examples/OTHER_TOOLS.md`](examples/OTHER_TOOLS.md) for crawl4ai, Scrapy +\nPlaywright, go-rod, Crawlee, and nodriver pointers.\n\n## Security\n\nThe CDP endpoint is **unauthenticated** — anyone who can reach the port has full control\nof the browser. Bind it to localhost or a trusted private network; never expose it to the\npublic internet. `--no-sandbox` is used because Chrome's sandbox does not work in an\nunprivileged container; keep the container isolated. To report a vulnerability, see\n[`SECURITY.md`](SECURITY.md).\n\n### Limits of hidden-text detection\n\nWorth knowing before treating an empty `threats` list as a clean bill of health. Nothing is\nremoved from the live page — the markup is stripped inside a separate inert document, which\nis imported rather than cloned (`cloneNode` is itself `[CEReactions]`), and the rendered text\ncomes from the live page with the flagged nodes hidden by an adopted stylesheet. So a page\ngets no synchronous hook to react to the strip. What that does *not* cover:\n\n- **The style signals are thresholds, and the character set is a denylist.** Those are the\n  real limits — see below. The detector itself runs in an isolated world\n  (`Page.createIsolatedWorld`), so a page cannot suppress it by replacing the DOM APIs it\n  uses; if the browser ever declines to provide one, the result carries a\n  `detection_degraded` threat rather than quietly weaker detection.\n- **Thresholds can be sat just inside.** `opacity: 0.06`, `font-size: 4px`, a contrast ratio\n  just above 1.15 — all pass, as do hiding techniques the twelve signals don't model\n  (`clip-path`, `text-indent`, `transform: scale(0)`).\n- **Invisible-character coverage is a set, not a rule.** Zero-width, bidi and the Unicode Tag\n  block are stripped and reported; codepoints outside that set are not.\n- **When the text is rebuilt, line breaks are guessed from tag names.** In the two cases\n  above the rendered text is taken from the stripped markup, which has no layout — so an\n  element the page styled `display:inline` still gets a break, and a block-level tag outside\n  the list gets none. Word boundaries are preserved; exact line structure is not.\n- **Closed shadow roots are not read.** Open ones are: their content is scanned for hidden\n  text and composed into the output as the flat tree a reader sees, slots included. A\n  closed root is unreachable from the isolated world, so it cannot be scanned — and what\n  cannot be scanned is not composed in. Its content stays out of the result entirely rather\n  than arriving unexamined.\n- **A page can win the cascade against the hiding sheet, or hide its own `<body>`.** An\n  inline `!important` beats an author stylesheet, and `innerText` returns raw text when\n  nothing renders at all. In either case the rendered text is abandoned for the stripped\n  markup, which is a weaker guarantee than reading real layout — reported as\n  `strip_incomplete` rather than left to look like a clean strip.\n\n### Limits of block detection\n\n`status` tells you a fetch returned a challenge or an error page rather than the content\nyou asked for. Worth knowing what it does and does not cover.\n\n- **Mitigation markers are a list, and the list is not exhaustive.** A challenge is called\n  with certainty when the response carries a header that exists only to announce it\n  (`cf-mitigated`, `x-vercel-mitigated`, `x-amzn-waf-action`, `x-dd-b`, `x-datadome-cid`) or\n  when the page requests an asset only a challenge loads (Cloudflare's orchestrator,\n  DataDome, PerimeterX, Imperva). A vendor absent from that list, or one that changes its\n  endpoint, falls through to the weaker signals below.\n- **The wording fallback is English, and only fires on an empty page.** A challenge with no\n  recognised marker is caught only if it renders almost no text *and* matches a known\n  phrase. A localized interstitial from an unlisted vendor is the gap — it is why the\n  markers exist, and why they are preferred over any amount of phrase tuning.\n- **A soft block is undetectable here.** A page that returns 200 with a plausible body but\n  the data quietly withheld looks exactly like content. Nothing in this classification sees\n  it; only comparing against what the page *should* contain would.\n- **`unknown` is not `ok`.** It means no response, or no usable status, was observed for the\n  document that was read. It is reported rather than assumed fine, and it is not treated as\n  a block — a source carrying it still contributes passages in `research`.\n- **Vendor *presence* is not a block.** `server: cloudflare`, `cf-ray` and `__cf_bm` are on\n  a large share of the web on every page it serves normally, so they are deliberately not\n  used; nor is `cf_clearance`, which is issued when a challenge is *passed*.\n\n**What the SSRF guard blocks.** Each host is resolved and rejected if it lands in loopback,\nRFC-1918 private, link-local (incl. `169.254.169.254`), reserved, multicast, unspecified,\nCGNAT `100.64.0.0/10`, or IPv4-mapped IPv6 ranges. Only `http` and `https` are allowed, and\ncredentials in URLs are rejected. The check runs again immediately before navigation, and\nonce more against `final_url` after redirects.\n\n**Limits of the SSRF guard.** It is a strong default, not a sandbox. Know these before\npointing it at untrusted URLs:\n\n- The guard resolves and checks the host *before* navigation and re-checks `final_url`\n  *after* the page loads. A redirect into a private address is therefore still requested by\n  Chrome — its content is never returned, but a blind SSRF or a state-changing internal `GET`\n  has already landed. Intermediate hops in a longer redirect chain are not individually\n  checked.\n- Sub-resource requests the page itself issues (`img`, `script`, `iframe`, `fetch`) are not\n  intercepted; only the top-level navigation is checked.\n- Groundhog resolves DNS in its own process while Chrome resolves independently at navigate\n  time, so a short-TTL rebinding window remains open. Closing these properly needs\n  request-level interception (CDP `Fetch`).\n- Fetches share the browser's default profile — targets are created without a separate\n  browser context — so cookies and storage set by one page persist into later fetches.\n  \"Read-only\" describes Groundhog's own API, not the JavaScript on a fetched page, which can\n  issue requests of its own from that shared profile.\n\nSet `GROUNDHOG_BLOCK_PRIVATE_IPS=false` only on a network where reaching internal addresses\nis intended.\n\n## A note on \"stealth\"\n\nBest-effort, not a guarantee. It defeats common open-source detectors and lets cheap\nproxies work on many mid-tier targets, but it does not beat sophisticated commercial\nanti-bot systems that gate on IP reputation, TLS/HTTP2 fingerprints, and behavioral\nanalysis. Use it for legitimate, authorized automation and testing.\n\n## Privacy Policy\n\nGroundhog is self-hosted software, not a service. It runs on your machine, and the project\noperates no servers that it talks to.\n\n**What is collected: nothing.** There is no telemetry, no analytics, no crash reporting and\nno licence check. The maintainers receive no data about you, the URLs you fetch, or the\ncontent you read. There is no account to create.\n\n**How data is used and stored.** Pages are fetched by a browser running on your own machine\nand returned to the MCP client that asked for them. The server keeps no database, writes no\nlogs to disk, and persists nothing between calls — with one exception worth knowing: fetches\nshare the browser container's profile, so cookies and storage set by one fetched page remain\nin that container and are visible to later fetches. Removing the container discards them\n(`docker rm -f groundhog-browser`), and the container is removed automatically if you started\nit with `--rm`.\n\n**Third parties your traffic reaches.** Only those you direct it to, plus two you should know\nabout:\n\n- **The sites you fetch**, which see the request as an ordinary browser visit from your IP —\n  or from your proxy's exit IP if `PROXY` is set.\n- **The search backend.** With `SEARXNG_URL` set, your queries go to the SearXNG instance you\n  chose. Without it, `search` renders a public search engine's results page through the\n  browser, so that engine sees the query.\n- **An IP-geolocation lookup, only when `PROXY` is set.** The container asks an external\n  service for the proxy exit IP's country so it can align the browser's timezone and locale.\n  That request carries the exit IP and nothing else. It does not happen without a proxy.\n- **PyPI and GHCR at install time**, to download the package and browser image.\n\n**Retention.** Nothing is retained by the project. On your machine, the browser container\nholds cookies and cache for its lifetime; deleting the container deletes them.\n\n**Contact.** Questions and security reports: [`SECURITY.md`](SECURITY.md), or open an issue at\n<https://github.com/dmytrome/groundhog/issues>.\n\n## License\n\n[MIT](LICENSE)\n",
  "bytes": 34185,
  "sha": "528558b747addf0a38926eb8f64b3851882463310fbbfa537707ce54f1435191",
  "repo_slug": "dmytrome/groundhog",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_dmytrome_groundhog_mcp_a47835f4/readme"
}