{ "markdown": "\n
\n \"groundcheck\"\n
\n\n# Groundcheck\n\n[![Groundcheck on x402-list](https://x402-list.com/badge/groundcheck.svg?data=uptime)](https://x402-list.com/services/groundcheck?utm_source=badge&utm_medium=referral&utm_campaign=embed)\n\n![Groundcheck — verify a factual claim against live sources, over MCP](assets/og-card.png)\n\n**The grounding check agents run before they commit to an answer.**\n\nGroundcheck verifies a factual claim against live sources and returns a **verdict**, a\n**confidence score**, and **citations**. Any agent — Claude Code, Cursor, your own — can call\nit mid-task, before it states a fact it isn't sure of.\n\nIt is also a **verification layer for agentic commerce**: when an agent pays another\nservice over x402, `attest_delivery` verifies what was delivered against what was\nadvertised and issues a signed, offline-verifiable **delivery receipt** binding payment →\ndelivery → grounded content — the neutral accountability trail the a2a-payments\nliterature calls the missing layer ([docs/delivery-attestation.md](docs/delivery-attestation.md)).\n\n## Architecture\n\nTwo parts, each in the language that fits it:\n\n```\nserver/ TypeScript MCP server — thin protocol layer (stdio). Holds no logic.\nengine/ Python FastAPI service — retrieval + stance classification + the verdict brain.\n```\n\nThe MCP server is spawned by your client over stdio and talks to the engine over HTTP\n(`GROUNDCHECK_ENGINE_URL`, default `http://127.0.0.1:8723`). The engine is the single source\nof truth for how a verdict is reached, and it classifies source stance through the canonical\nPython [`free-llm-router`](https://github.com/beepboop2025/free-llm-router) (free-tier providers).\n\n```\nverify_claim ─▶ TS MCP server ─HTTP▶ Python engine\n ├─ retrieval (Wikipedia, keyless; or your own search)\n ├─ stance (free-llm-router → supports/refutes/neutral)\n └─ verdict (refuses on conflict, saturating confidence)\n```\n\n## Tools\n\n| Tool | Use it when | Returns |\n|------|-------------|---------|\n| `verify_claim(claim, maxSources?)` | About to assert a fact you're unsure of | `{ verdict, confidence, rationale, sources }` |\n| `check_citations(text, maxClaims?)` | Before publishing an AI-generated draft | per-claim verdict report |\n| `attribution_badge()` | Want to mark content as checked | a Markdown badge |\n| `resolve_instrument(query, idType?, maxResults?)` | Text names a security and you need to know exactly which one | canonical FIGI records + provenance (Bloomberg open symbology) |\n| `extract_claims(text, maxClaims?)` | Want to see which claims a document makes before paying to ground them | atomic checkable claims + a signed receipt bound to the input hash |\n| `attest_delivery(service, response_text, …)` | You paid another service over x402 and will act on (or account for) its output | a signed **delivery receipt** binding payment → delivery → grounded content ([docs](docs/delivery-attestation.md)) |\n\n`verdict` is one of `supported` · `refuted` · `unverified`. Each verdict also\ncarries a `sufficiency` tag (`sufficient` · `insufficient` · `no_sources` ·\n`no_stance` · `conflict`) so an agent can tell \"I found nothing\" from \"sources\nexist but don't establish it\" from \"sources disagree\" — the three ways an\nabstention happens carry different meaning and are no longer collapsed\n([SURE-RAG](https://arxiv.org/abs/2605.03534)).\n\n**Compound claims are decomposed.** A claim like _\"Marie Curie won two Nobel\nPrizes and was born in Paris\"_ is split into atoms\n([Fact in Fragments](https://arxiv.org/abs/2506.07446)), each verified on its\nown evidence and recombined weakest-link: one false part refutes the whole, one\nunproven part blocks a `supported`. The true half can no longer carry the false\nhalf past the check. The atom breakdown is returned in `atoms`. (Decomposition\nis rule-based and high-precision — it splits only on clean conjunction\nboundaries and otherwise leaves the claim whole; disable with\n`GROUNDCHECK_DECOMPOSE=0`.)\n\n**Remote MCP (no install):** add `https://groundcheck.seiche.info/mcp` as a remote MCP server (Claude/ChatGPT/Cursor connectors, or a gateway like Smithery/Glama). Speaks streamable-HTTP JSON-RPC; `verify_claim` is free, the paid tools answer HTTP 402 with an x402 offer.\n\n## Quickstart\n\nThe MCP server **auto-starts the Python engine** if one isn't already running, so a single\nregistration is enough — no separate process to babysit.\n\n```bash\nmake install # deps for both halves (pip + npm)\nnpm --prefix server run build # compile the server\nexport GROQ_API_KEY=\"gsk_...\" # one free key for stance classification (Groq: ~2 min, 14,400/day)\n\n# register with your MCP client — the engine spawns on first use and stops with the server\nclaude mcp add groundcheck -- node \"$PWD/server/dist/server.js\"\n```\n\nAlready running the engine yourself (`make engine` or `docker compose up -d`)? The server\ndetects and **reuses** it — and won't touch an engine it didn't start. Set\n`GROUNDCHECK_NO_SPAWN=1` to stop it from ever spawning one.\n\n> Once published to npm, registration becomes `claude mcp add groundcheck -- npx -y groundcheck-mcp`.\n> Auto-spawn needs a local `engine/` + Python deps; for an npx-only install, run the engine via\n> `docker compose up -d` and the server connects to it over `GROUNDCHECK_ENGINE_URL`.\n\nWith **no** provider key the engine still runs — retrieval works, but every verdict is\n`unverified`. It degrades honestly: a disabled backend, a missing key, or conflicting sources\nall flow toward `unverified`. An unconfigured Groundcheck **cannot** return `supported`.\n\n> Note: OpenRouter's `:free` models are quota-throttled (HTTP 429) and make a poor sole\n> provider. Prefer Groq or Cerebras for the fast classification tier.\n\n## Why grounded verdicts, not LLM-judgment\n\nAsking an LLM to *judge* whether a claim is true is unreliable in a way that's easy to miss.\nIn **TraderBench** (Yuan et al., 2026), the *same* candidate responses re-scored by three\nfrontier LLM judges swung by ~29 points on the knowledge-retrieval section — while the\nperformance-grounded section, whose scoring is anchored to verifiable computation, swung 0.3.\nThe lesson: **the more you constrain a judgment with external evidence, the less it varies.**\n\nGroundcheck is built on that principle. It never asks a model \"is this true?\" from parametric\nmemory. Instead it:\n\n- **retrieves** sources first, then asks only the narrow, evidence-anchored question — does\n *this cited passage* support, refute, or stay neutral on the claim (stance classification);\n- **refuses on conflict** and saturates confidence, so disagreement flows to `unverified`\n rather than a confident guess;\n- **returns citations**, so the verdict is checkable, not taken on the model's word.\n\nThat's the difference between an LLM judge and a grounding check: the judge's discretion is the\nproduct; here it's deliberately fenced in by retrieved evidence.\n\n## Calibrated verdicts: the \"error ≤ α\" guarantee\n\nA confidence number without a promise attached is just vibes with decimals. When a\ncalibration artifact is deployed, Groundcheck attaches a `guarantee` object to\ndirectional verdicts, built with **split conformal prediction** (adapted from\n*Multi-LLM Adaptive Conformal Inference*, arXiv:2602.01285):\n\n- Stance classification runs as a **panel**: up to `GROUNDCHECK_ENSEMBLE_MAX` free\n providers judge the claim independently (different model families disagree on\n *which* claims they get wrong, so the ensemble beats any one of them). Per-source\n stances are majority-voted; each panelist also emits a probability the claim is\n true given only the snippets, combined into a weighted `ensemble_score`.\n- `scripts/calibrate.py` runs the real pipeline over a labeled claim set and stores\n finite-sample thresholds per claim group (`instrument` / `general`, `global`\n fallback) in `calibration/calibration.json`.\n- A verdict is **certified** (`guarantee.certified: true`) only when its score\n clears the threshold. The math guarantees that, for claims exchangeable with the\n calibration set, a **false claim is certified `supported` with probability ≤ α**\n (default 0.1), and symmetrically for `refuted`. No distributional assumptions,\n exact in finite samples.\n\nHonest degradation, as everywhere else: no artifact → no guarantee is ever claimed;\ntoo little calibration data for a given α → the threshold is refused rather than\nextrapolated. The guarantee is only as good as the exchangeability assumption —\nrecalibrate with domain claims before leaning on it in a new domain.\n\n## Configuration (engine)\n\n| Var | Default | Purpose |\n|-----|---------|---------|\n| `GROUNDCHECK_SEARCH_BACKEND` | _(unset)_ | `stub` to disable real retrieval |\n| `GROUNDCHECK_SEARCH_URL` | Wikipedia | custom JSON search endpoint (`{results:[{title,url,snippet,stance?}]}`) |\n| `GROUNDCHECK_SEARCH_KEY` | — | bearer token for the custom endpoint |\n| `GROUNDCHECK_ROUTER_PATH` | sibling checkout | path to the `free-llm-router` Python package |\n| `GROUNDCHECK_ENGINE_HOST` / `_PORT` | `127.0.0.1` / `8723` | engine bind address |\n| `GROQ_API_KEY` _(or any router provider key)_ | — | enables stance classification |\n| `GROUNDCHECK_ENSEMBLE` | `1` | multi-provider stance panel (`0` = single-router) |\n| `GROUNDCHECK_ENSEMBLE_MAX` | `3` | max concurrent panelists per claim |\n| `GROUNDCHECK_CALIBRATION` | `calibration/calibration.json` | conformal artifact path |\n\n### Machine-payable hosting (x402)\n\nA hosted engine can charge AI agents per call in USDC over the\n[x402 protocol](https://x402.org) — HTTP 402 + signed transfer authorization,\nno accounts or API keys. Dormant unless `GROUNDCHECK_X402_PAY_TO` is set;\n`/verify` stays free forever and is the way to evaluate output before paying;\nthe paid surface prices as a granular verification loop: **extract $0.005 → ground\n$0.02 → delivery-attestation bundle $0.05** (plus `/resolve` at $0.005).\nBoth protocol generations (v1 and v2) are accepted, and agents can read the\noffer at `GET /.well-known/x402`. Full operator guide: [docs/x402.md](docs/x402.md).\n\nServer side:\n\n| Var | Default | Purpose |\n|-----|---------|---------|\n| `GROUNDCHECK_ENGINE_URL` | `http://127.0.0.1:8723` | where the server finds the engine |\n| `GROUNDCHECK_NO_SPAWN` | _(unset)_ | set to disable auto-spawning the engine |\n| `GROUNDCHECK_ENGINE_DIR` | repo `engine/` | engine location for auto-spawn |\n| `GROUNDCHECK_PYTHON` | `python3` | interpreter used to spawn the engine |\n| `GROUNDCHECK_REPO_URL` | repo URL | URL used in the attribution footer/badge |\n\n## Development\n\n```bash\nmake test # engine pytest (verdict rule + x402 gating) + server typecheck\nmake engine # run the engine\nmake server # run the MCP server in dev (tsx)\nmake build # compile the server to server/dist\n```\n\nThe interesting logic is in `engine/groundcheck_engine/verdict.py`: how much source\nagreement counts as \"supported,\" how conflict is handled, and how confidence saturates.\n\nMIT.\n", "bytes": 11219, "sha": "2b8822d3365cf23c65e1aec18aa7dae150607ba74bc16d15f670e49a0d1a046c", "repo_slug": "beepboop2025/groundcheck", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/mcp_io_github_beepboop2025_groundcheck_3ca1361b/readme" }