{
  "markdown": "<h2 align=\"center\">Kloakt</h2>\n\n<p align=\"center\">\n  <strong>Cloaked headless browser for AI agents.</strong><br>\n  Lightweight, stealthy, built in Rust. Based on <a href=\"https://github.com/h4ckf0r0day/obscura\">Obscura</a>.\n</p>\n\n---\n\n<!-- mcp-name: io.github.KultMember6Banger/kloakt -->\n\nKloakt is a headless browser built for AI agents. It runs JavaScript via V8, extracts clean markdown from any page (including SPAs), and exposes tools via MCP for Claude Code and other AI systems.\n\nBeyond one-shot extraction it can drive **persistent, stateful sessions** (click, type, navigate — cookies *and* page/JS state persist across calls), emit an **accessibility/structure snapshot** as an agent-vision substitute, and capture **real screenshots** via system Chrome — 12 MCP tools in all.\n\n### Why Kloakt?\n\n| Metric       | Kloakt       | Headless Chrome |\n|--------------|--------------|------------------|\n| Memory       | **30 MB**    | 200+ MB          |\n| Binary size  | **70 MB**    | 300+ MB          |\n| Anti-detect  | **Built-in** | None             |\n| Page load    | **85 ms**    | ~500 ms          |\n| Startup      | **Instant**  | ~2s              |\n| SPA extract  | **Yes**      | Manual           |\n\n## Install\n\n### Prebuilt binary (recommended)\n\nOne-line install (Linux & macOS). Downloads the right binary for your OS/arch from the latest GitHub Release and installs it to `~/.local/bin` (or `/usr/local/bin` when run as root):\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/KultMember6Banger/kloakt/main/install.sh | sh\n```\n\nYou can pin a version or override the install dir:\n\n```bash\nKLOAKT_VERSION=v0.1.2 INSTALL_DIR=/usr/local/bin \\\n  sh -c \"$(curl -fsSL https://raw.githubusercontent.com/KultMember6Banger/kloakt/main/install.sh)\"\n```\n\nWindows: download `kloakt-x86_64-windows.zip` from the [Releases page](https://github.com/KultMember6Banger/kloakt/releases) and extract `kloakt.exe` onto your `PATH`.\n\n### Homebrew (macOS)\n\n```bash\nbrew install KultMember6Banger/kloakt/kloakt\n# or, from a local checkout:\nbrew install --formula ./Formula/kloakt.rb\n```\n\n(Until a dedicated tap exists, `brew tap KultMember6Banger/kloakt https://github.com/KultMember6Banger/kloakt` then `brew install kloakt`.)\n\n### cargo install\n\nBuilds the CLI from crates.io (requires Rust toolchain; first build compiles V8, ~5 min):\n\n```bash\ncargo install obscura-cli\n```\n\nThis installs the `kloakt` binary. To build with stealth mode, add `--features stealth`.\n\n### Build from source\n\n```bash\ngit clone https://github.com/KultMember6Banger/kloakt.git\ncd kloakt\ncargo build --release\n\n# With stealth mode (anti-detection + tracker blocking)\ncargo build --release --features stealth\n```\n\nRequires Rust 1.75+ ([rustup.rs](https://rustup.rs)). First build takes ~5 min (V8 compiles from source, cached after).\n\n## Quick Start\n\n### Extract content (AI agent use)\n\n```bash\n# Clean markdown from any page\nkloakt extract https://example.com --main\n\n# Structured JSON with metadata\nkloakt extract https://example.com --main --json\n\n# Cap output for agent context windows\nkloakt extract https://en.wikipedia.org/wiki/Rust --main --json --max-chars 3000\n\n# Wait for SPA hydration\nkloakt extract https://example.com --delay 2000 --json\n```\n\n### Fetch a page\n\n```bash\n# Get the page title\nkloakt fetch https://example.com --eval \"document.title\"\n\n# Extract all links\nkloakt fetch https://example.com --dump links\n\n# Render JavaScript and dump markdown\nkloakt fetch https://news.ycombinator.com --dump markdown\n\n# Wait for dynamic content\nkloakt fetch https://example.com --wait-until networkidle0\n```\n\n### Start the CDP server\n\n```bash\nkloakt serve --port 9222\n\n# With stealth mode\nkloakt serve --port 9222 --stealth\n```\n\n### Scrape in parallel\n\n```bash\nkloakt scrape url1 url2 url3 ... \\\n  --concurrency 25 \\\n  --eval \"document.querySelector('h1').textContent\" \\\n  --format json\n```\n\n### Snapshot page structure (agent vision)\n\n```bash\n# Indexed accessibility/structure tree — tags, text, roles, what's clickable, visibility\nkloakt snapshot https://example.com\n\n# Only the actionable elements (links, buttons, inputs), with id/name/label for targeting\nkloakt snapshot https://example.com --interactive\n```\n\nkloakt has no rasterizer, so this is the lightweight \"what's on the page and what can I act on\" view for agents that work from structure rather than pixels.\n\n### Screenshot (via system Chrome)\n\n```bash\n# Real PNG — delegates to a locally-installed Chrome/Chromium/Edge\nkloakt screenshot https://example.com --output shot.png --width 1280 --height 800\n```\n\n### Persistent sessions\n\nDrive a named session whose cookies **and** page/JS state survive across separate\ninvocations, backed by a running `kloakt serve` daemon:\n\n```bash\nkloakt serve --port 9222 &                        # start the daemon once\n\nkloakt session open shop --url https://example.com\nkloakt session snapshot shop --interactive        # see the page structure\nkloakt session type shop 'input[name=q]' 'hello'  # fill a field\nkloakt session click shop 'button[type=submit]'   # click an element\nkloakt session text shop                          # read the body text\nkloakt session eval shop 'document.title'         # run JS, get the value back\nkloakt session close shop                         # tear down (drops cookies + page)\n```\n\n## Smart Extraction\n\nThe `extract` command uses a multi-phase pipeline optimized for AI agents:\n\n1. **Noise removal** — strips cookie banners, ads, popups, nav, social widgets\n2. **Content scoring** — text-density algorithm (Readability-like) finds the main content block\n3. **Markdown conversion** — DOM-to-markdown with absolute URL resolution\n4. **SPA fallback** — when JS rendering fails, extracts from meta tags, Open Graph, JSON-LD, and noscript content\n\nWorks on static HTML, server-rendered pages, and pure client-side SPAs (React, Vue, etc.).\n\n## Python API\n\n```python\nfrom kloakt import (\n    extract, extract_fields, fetch, scrape, search, crawl,\n    snapshot, screenshot, session_open, session_close,\n)\n\n# Extract clean markdown\npage = extract(\"https://example.com\")\nprint(page.title, page.content, page.meta)\n\n# Cap output length\npage = extract(\"https://example.com\", max_chars=3000)\n\n# Wait for SPA content\npage = extract(\"https://example.com\", delay=2000)\n\n# Structured field extraction via CSS selectors\ndata = extract_fields(\"https://news.ycombinator.com\", {\n    \"title\": \"title\",\n    \"stories\": \".titleline > a[]\",   # [] => list of all matches\n    \"links\": \".titleline > a[]@href\" # @href => an attribute\n})\nprint(data[\"data\"][\"stories\"])\n\n# Raw fetch\nhtml = fetch(\"https://example.com\", dump=\"html\")\ntitle = fetch(\"https://example.com\", eval_js=\"document.title\")\n\n# Parallel scrape\nresults = scrape([\"https://a.com\", \"https://b.com\"], concurrency=5)\n\n# Discover links, or breadth-first crawl a small section of a site\nlinks = search(\"https://news.ycombinator.com\", same_domain=True)\npages = crawl(\"https://example.com\", max_pages=5, max_depth=1)\n\n# Structure snapshot (agent vision) and a real screenshot via system Chrome\nsnap = snapshot(\"https://example.com\", interactive=True)\nscreenshot(\"https://example.com\", output=\"shot.png\")\n\n# Persistent session — auto-starts a daemon if one isn't running; cookies + page\n# state persist across calls. (session_nav / _click / _type / _eval / _text / _snapshot)\nsession_open(\"shop\", url=\"https://example.com\")\n# ... drive the page across calls ...\nsession_close(\"shop\")\n```\n\n## MCP Server (Claude Code)\n\nKloakt includes an MCP server for use as a Claude Code tool:\n\n```json\n{\n  \"mcpServers\": {\n    \"kloakt\": {\n      \"command\": \"python3\",\n      \"args\": [\"/path/to/kloakt/mcp_server.py\"]\n    }\n  }\n}\n```\n\nExposes 12 native tools:\n\n| Tool | What it does |\n|------|--------------|\n| `kloakt_extract` | Clean markdown, or structured fields via `schema` |\n| `kloakt_fetch` | Low-level fetch (html/text/links/markdown, or JS eval) |\n| `kloakt_scrape` | Many URLs in parallel |\n| `kloakt_search` | Discover outbound links on a page |\n| `kloakt_crawl` | Budget/depth-limited breadth-first crawl |\n| `kloakt_snapshot` | Accessibility/structure tree (agent vision) |\n| `kloakt_screenshot` | Real PNG via system Chrome |\n| `kloakt_session_open` | Open a persistent named session |\n| `kloakt_session_act` | navigate / click / type / eval within a session |\n| `kloakt_session_read` | Read a session's page as text or snapshot |\n| `kloakt_session_list` | List open sessions |\n| `kloakt_session_close` | Close a session (drops its cookies + page) |\n\n## Puppeteer / Playwright\n\n### Puppeteer\n\nThe CDP server embeds a per-session token in the WebSocket path (like Chrome). Connect via\n`browserURL` so the client discovers the token from `/json/version` automatically — don't\nhardcode the `ws://.../devtools/browser` path.\n\n```javascript\nimport puppeteer from 'puppeteer-core';\n\nconst browser = await puppeteer.connect({\n  browserURL: 'http://127.0.0.1:9222', // discovers the tokenized ws endpoint\n});\n\nconst page = await browser.newPage();\nawait page.goto('https://news.ycombinator.com');\nconst stories = await page.evaluate(() =>\n  Array.from(document.querySelectorAll('.titleline > a'))\n    .map(a => ({ title: a.textContent, url: a.href }))\n);\nawait browser.disconnect();\n```\n\n### Playwright\n\n```javascript\nimport { chromium } from 'playwright-core';\n\nconst browser = await chromium.connectOverCDP({\n  endpointURL: 'http://127.0.0.1:9222', // discovers the tokenized ws endpoint\n});\n\nconst page = await browser.newContext().then(ctx => ctx.newPage());\nawait page.goto('https://en.wikipedia.org/wiki/Web_scraping');\nconsole.log(await page.title());\nawait browser.close();\n```\n\n## Stealth Mode\n\nEnable with `--features stealth`.\n\n- Per-session fingerprint randomization (GPU, screen, canvas, audio, battery)\n- Realistic `navigator.userAgentData` (Chrome 145, high-entropy values)\n- `event.isTrusted = true` for dispatched events\n- Native function masking (`Function.prototype.toString()` → `[native code]`)\n- `navigator.webdriver = undefined`\n- Realistic `Accept-Language` + Client Hints (`Sec-CH-UA`) request headers\n- Per-session randomized `navigator.languages`\n- TLS fingerprint (JA3) rotation across Chrome 145 Linux / Windows / macOS profiles\n- 3,520 tracker domains blocked\n\n## CLI Reference\n\n### `kloakt extract <URL>`\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--format` | `markdown` | Output: `markdown`, `text`, or `links` |\n| `--main` | off | Strip nav, header, footer, sidebar |\n| `--json` | off | Structured JSON: title, URL, content, meta |\n| `--max-chars` | unlimited | Truncate content to N characters |\n| `--delay` | `0` | Extra ms to wait after load |\n| `--stealth` | off | Anti-detection mode |\n| `--selector` | — | Wait for CSS selector |\n| `--wait-until` | `load` | `load`, `domcontentloaded`, `networkidle0` (bounded by `--wait`) |\n| `--schema` | — | Extract structured fields as JSON (see below) |\n| `--har` | — | Write captured network activity to a HAR file |\n| `--cache-ttl` | `0` | Cache the result on disk and reuse it for N seconds |\n\n#### Structured extraction with `--schema`\n\nPass a JSON object mapping field names to CSS selectors. Suffix a selector with `[]` to\nreturn **all** matches as a list, and with `@attr` to return an **attribute** instead of text:\n\n```bash\nkloakt extract https://news.ycombinator.com \\\n  --schema '{\"title\":\"title\",\"stories\":\".titleline > a[]\",\"first_link\":\".titleline > a@href\"}'\n# => { \"url\": ..., \"data\": { \"title\": \"...\", \"stories\": [...], \"first_link\": \"...\" }, \"elapsed_ms\": ... }\n```\n\nThis is also exposed through the MCP `kloakt_extract` tool via an optional `schema` argument.\n\n### `kloakt fetch <URL>`\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--dump` | `html` | Output: `html`, `text`, `links`, `markdown` |\n| `--eval` | — | JavaScript expression to evaluate |\n| `--wait-until` | `load` | Wait condition |\n| `--selector` | — | Wait for CSS selector |\n| `--stealth` | off | Anti-detection mode |\n| `--quiet` | off | Suppress banner |\n\n### `kloakt serve`\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--port` | `9222` | WebSocket port |\n| `--proxy` | — | HTTP/SOCKS5 proxy URL |\n| `--stealth` | off | Anti-detection + tracker blocking |\n| `--workers` | `1` | Parallel workers |\n\n### `kloakt scrape <URL...>`\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--concurrency` | `10` | Parallel workers |\n| `--eval` | — | JS expression per page |\n| `--format` | `json` | Output: `json` or `text` |\n\n### `kloakt snapshot <URL>`\n\nEmit an indexed accessibility/structure tree (always JSON) — an agent-vision substitute.\nEach node is compact: `i` (index), `tag`, `depth`, `vis` (visible), and when present `click`,\n`role`, `text`, `type`/`value`, `href`, `id`, `name`, `label`.\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--interactive` | off | Only actionable elements (links, buttons, inputs) |\n| `--max-nodes` | `1500` | Cap on nodes emitted |\n| `--stealth` | off | Anti-detection mode |\n| `--delay` | `0` | Extra ms to wait after load |\n| `--wait-until` | `load` | Wait condition |\n\n### `kloakt screenshot <URL>`\n\nCapture a real PNG by delegating to a locally-installed Chrome/Chromium/Edge (kloakt has no\nrasterizer). Errors clearly if none is found; override detection with `--chrome <path>` or the\n`KLOAKT_CHROME` env var.\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--output` | `screenshot.png` | Output PNG path |\n| `--width` | `1280` | Viewport width |\n| `--height` | `800` | Viewport height |\n| `--chrome` | auto-detect | Path to a Chrome/Chromium/Edge binary |\n\n### `kloakt session <COMMAND>`\n\nDrive a persistent, named session against a running `kloakt serve` daemon. Cookies **and**\npage/JS state survive across separate invocations (client state in `~/.kloakt/sessions/<name>.json`).\n\n| Command | Description |\n|---------|-------------|\n| `open <name> [--url URL] [--port]` | Open (or reattach to) a session and create a page |\n| `nav <name> <url>` | Navigate the session's page |\n| `eval <name> <expr>` | Evaluate a JS expression, print the JSON value |\n| `text <name>` | Print `document.body.innerText` |\n| `snapshot <name> [--interactive]` | Structure snapshot of the session's page |\n| `click <name> <selector>` | Click the first matching element |\n| `type <name> <selector> <text>` | Focus an element, set its value, fire input/change |\n| `list [--port]` | List open sessions on the daemon |\n| `close <name>` | Close the session (drops its pages + cookies) |\n\n> Multi-statement JS passed to `eval` returns `null` (the daemon evaluates a single\n> expression); wrap it in an IIFE — `(function(){ ...; return v })()` — to get a value back.\n\n### `kloakt benchmark <URL...>`\n\nMeasure load performance per URL — average/min/max load time, request count, bytes, and DOM\nnode count — as a table or `--json`.\n\n```bash\nkloakt benchmark https://example.com https://news.ycombinator.com --runs 3\n```\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--runs` | `1` | Runs per URL (reports the average) |\n| `--json` | off | Emit JSON instead of a table |\n| `--wait-until` | `load` | `load`, `domcontentloaded`, or `networkidle0` |\n\n### Challenge / bot-wall detection\n\n`kloakt extract --json` includes a `\"challenge\"` field reporting a detected captcha or bot\nwall (`recaptcha`, `hcaptcha`, `turnstile`, `cloudflare`, `datadome`, `perimeterx`) or\n`null`. This is **detection only** — kloakt tells you a page is gated so an agent can stop\nand back off; it does not attempt to solve or evade challenges. (Also surfaced via the MCP\n`kloakt_extract` output and the Python `Page.challenge` field.)\n\n### Global flags\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--obey-robots` | off | Respect `robots.txt` — refuse to fetch disallowed paths |\n| `--allow-private` | off | Allow private/internal/loopback hosts (disables the SSRF guard) |\n\n> **Security note:** by default kloakt refuses to fetch private, loopback, link-local, and\n> cloud-metadata addresses (SSRF protection), and rejects `file://` URLs. Use `--allow-private`\n> only when you intentionally need to reach internal services. The CDP server binds to\n> `127.0.0.1` and validates the `Host` header to block DNS-rebinding.\n\n## CDP API\n\nFull Chrome DevTools Protocol support for Puppeteer/Playwright compatibility.\n\n| Domain | Methods |\n|--------|---------|\n| **Session** | open, close, list — named, persistent browser contexts that keep cookies alive |\n| **Target** | createTarget (`browserContextName` binds a page to a session), closeTarget, attachToTarget, createBrowserContext, disposeBrowserContext |\n| **Page** | navigate, getFrameTree, addScriptToEvaluateOnNewDocument, lifecycleEvents |\n| **Runtime** | evaluate, callFunctionOn, getProperties, addBinding |\n| **DOM** | getDocument, querySelector, querySelectorAll, getOuterHTML, resolveNode |\n| **Network** | enable, setCookies, getCookies, setExtraHTTPHeaders, setUserAgentOverride |\n| **Fetch** | enable, continueRequest, fulfillRequest, failRequest |\n| **Storage** | getCookies, setCookies, deleteCookies |\n| **Input** | dispatchMouseEvent, dispatchKeyEvent |\n\n## License\n\nApache 2.0 — Based on [Obscura](https://github.com/h4ckf0r0day/obscura) by h4ckf0r0day.\n",
  "bytes": 17287,
  "sha": "861e5dcec96c4b097d8b85bf01e0798901796eacaedddb7d3dbd8d3fab76f8da",
  "repo_slug": "kultmember6banger/kloakt",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_kultmember6banger_kloakt_04ca640c/readme"
}