{ "markdown": "\"polygraph\"\n\n# litmus\n\n**The open behavioral litmus harness for MCP servers — grade A–F, reproducible.**\n\n\n\n \n \n \n \n \n \n \n \n
\"npm\"\"CI\"\"license\"\"Glama\"\"Smithery\"\"graded
\n\n## Grade a server in one command\n\n```bash\n# -p is required: the package ships three bins, so npx must be told which to run\nnpx -y -p @polygraphso/litmus polygraphso-litmus litmus npm/@modelcontextprotocol/server-filesystem\n```\n\nPoint it at an npm ref, a pypi ref, a `github/owner/repo` ref (cloned, built, and run sandboxed —\nDocker required; the grade pins the commit SHA), an `https://` MCP endpoint, or a local entry\nfile. The harness connects\nthe way an agent would, fingerprints the exact tool surface, runs the four probe categories, and\nprints the grade with the findings behind it — plus a deterministic evidence bundle on disk.\nIt **runs the target's code** (egress is Docker-sandboxed; without Docker, C-02 is skipped and the\ngrade caps at B), takes ~20–60s, and exits non-zero on D/F so it scripts anywhere. To dispute any\npublished grade, re-run this same command against the same server — open and deterministic means a\nre-run reproduces the grade, or refutes it.\n\nLooking up a grade someone already published takes under a second and runs nothing: the\n[polygraph.so index](https://polygraph.so/rankings), or `check_server` from the MCP tools below.\n\n![litmus grading npm/@modelcontextprotocol/server-filesystem — five probe steps, then grade: A](docs/assets/polygraph-litmus-demo.gif)\n\n## Use it from your agent — MCP server + plugin\n\nFor grade lookups, point any MCP client at polygraph's hosted endpoint, no install:\n\n```bash\nclaude mcp add --transport http polygraph https://polygraph.so/api/mcp\n```\n\nor the raw config:\n\n```json\n{ \"mcpServers\": { \"polygraph\": { \"url\": \"https://polygraph.so/api/mcp\" } } }\n```\n\nThis serves the lookup tools only (`check_server`, `list_servers`, `request_grade`); grading a\nserver yourself (`run_litmus`, `run_skill_litmus`) executes its code, so it needs the local\nstdio install below.\n\nThe package also ships a stdio MCP server (`polygraphso-litmus-mcp`) with the full toolset, for\nany MCP-capable client:\n\n- **`check_server`** — read a server's *published* grade in under a second (no execution); the\n pre-flight check before recommending or installing a server.\n- **`list_servers`** — servers with a published grade, A first; paged (default 25 per call, with `grade`/`limit`/`offset` filters and a full-corpus summary).\n- **`request_grade`** — record a grade request with polygraph.so ($1 one-time fee; graded within 48h of payment — the response carries the payment link).\n- **`run_litmus`** — grade a server *now*: the full harness, grade + evidence returned to the agent.\n- **`run_skill_litmus`** — grade a Claude Code / Agent Skill (static scan, A/B/D/F).\n- **`verify_attestation`** — read the onchain proof behind a published grade (EAS on Base).\n\nIn **Claude Code**, the plugin wires the server plus two commands in one step:\n\n```\n/plugin marketplace add polygraphso/litmus\n/plugin install polygraph@polygraphso\n```\n\nthen `/polygraph:grade ` and `/polygraph:check `. Cursor and manual JSON setups\nare on [polygraph.so](https://polygraph.so/#install); full tool docs in\n[`packages/litmus/README.md`](packages/litmus/README.md#use-it-from-an-ai-agent-mcp-server).\n\n## Gate your CI on MCP grades — GitHub Action\n\nFail a build when an MCP **server** or an Agent **Skill** it ships grades **D/F** under the open\nbehavioral litmus. For servers it is hybrid — a fast lookup of the published grade, then the harness\nwhen ungraded; for skills it is a fast static scan. Un-gradeable targets warn unless `strict`.\n\nIt's on the **[GitHub Marketplace](https://github.com/marketplace/actions/polygraph-mcp-gate)** as\n`polygraphso/litmus@v1`. For a security gate, pin to a commit SHA rather than the mutable `@v1` tag:\n\n```yaml\n# .github/workflows/mcp-gate.yml\nname: mcp-gate\non: [pull_request] # NOT pull_request_target — that exposes secrets to fork PRs\npermissions:\n contents: read\njobs:\n gate:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v5\n - uses: polygraphso/litmus@ # pin to a SHA; resolve from the v1 release\n with:\n # Name the targets explicitly (recommended). Grading runs a server's code,\n # so on a public repo prefer an allowlist over discovering PR-controlled config:\n servers: |\n npm/@modelcontextprotocol/server-filesystem\n skills: |\n ./my-skill\n # discover: \"true\" # opt in to auto-discovery (.mcp.json/.vscode/.cursor) — trusted repos only\n # min-grade: B # stricter than the default D/F gate\n # strict: \"true\" # also fail on targets that cannot be graded\n```\n\n**Inputs:** `servers` · `skills` · `discover` (default `false`) · `min-grade` · `strict` · `working-directory` · `version` · `bearer`. **Outputs:** `result` · `failed` · `report`.\n\n**Security.** Grading a server **runs its code** (egress is Docker-sandboxed, but it still executes).\nTrigger on `pull_request`, never `pull_request_target`. Keep `discover` off on public repos and name\ntargets explicitly — auto-discovered config is pull-request-controllable. `bearer` is sent as an\n`Authorization` header to the target, so pass it only for an explicitly trusted, pinned remote — never\nwith discovery or on untrusted PRs, and keep it scoped and short-lived.\n\nNot on GitHub? The gate is a plain command — `npx @polygraphso/litmus@0.20.0 ci` (pin the version) —\nso it runs in any CI or as a pre-commit hook. A grade is a measurement, not a guarantee: re-run the\nopen harness to reproduce any result.\n\n## What litmus is\n\nThis is the source for **[`@polygraphso/litmus`](https://www.npmjs.com/package/@polygraphso/litmus)**,\nthe open behavioral litmus harness for MCP servers from [polygraph.so](https://polygraph.so).\n\nThe harness connects to an MCP server the way an agent would, fingerprints its exact\ntool surface, and runs four probe categories — **C-01** tool-output injection (static,\ndynamic, and second-order — one tool's output weaponized as another's input), **C-02**\npermission/egress (in a hardened default-deny Docker sandbox, matched host **and** port),\n**C-03** sensitive-data handling (planted canaries), **C-04** adversarial-input handling\n(malformed/oversized and jailbreak inputs) — then grades the server **A–F**. A passing grade is a\nmeasurement, not a guarantee; the methodology and its disclosed limits are at\n[polygraph.so](https://polygraph.so) (the open source here is the ground truth).\n\nAlongside the grade, an npm target's dependency tree is checked against the\n[osv.dev](https://osv.dev) vulnerability database and any vulnerable dependencies are reported as\n**dependency advisories**. This is a separate, **point-in-time** signal — it is *advisory only*: it\nnever affects the A–F grade and is not part of the reproducible evidence (vulnerability data changes\nover time, so folding it into the grade would break re-run reproducibility). It applies to npm\ntargets only; other target kinds report it as skipped. Resolution runs\n`npm install --package-lock-only --ignore-scripts`, which resolves the tree without downloading\ntarballs or running any package code. Opt out with `--no-deps-audit` (or `LITMUS_DEPS_AUDIT=0`).\n\nThe same package also grades **Claude Code / Agent Skills** (a `SKILL.md` + bundle) under a\n**separate** static litmus (`litmus-skill-v3`): a deterministic byte-scan — **S-01** prompt\ninjection, **S-03** data-exfiltration instructions, **S-04** dangerous commands in the SKILL.md\nbody or bundled scripts (incl. base64-obfuscated `curl | bash`) — graded **A/B/D/F** and anchored\nby a whole-directory **content hash**, plus a separate\nadvisory quality signal. It is *static* (no execution): an **A** is static-clean, not behavioral\nproof. See [`packages/litmus/README.md`](packages/litmus/README.md#grade-a-skill).\n\nThe hosted, operator-run grading **service** is **not** in this repo — it lives in a\nseparate private repo and consumes this package from npm like any other client.\n\n## Layout\n\nThis is a pnpm monorepo. Only **`@polygraphso/litmus`** is published; the\n`@polygraph/*` packages are private building blocks that tsup bundles into it.\n\n```\npackages/\n litmus/ # @polygraphso/litmus — the only published package (lib + 3 bins: CLI, skill CLI, MCP)\n core/ # contract types, canonical JSON, identity helpers\n probes/ # the harness: connect, fingerprint, grade, probe runners, sandbox\n onchain/ # EAS attestation read + encode/decode (Base) — read-only, no minting\n agent/ # agent-gate decision logic + live-fingerprint recheck\n mcp/ # MCP server wrapper\n cli/ # CLI commands + target/auth resolution\n demo-*-mcp/ # demo MCP servers used as test fixtures\n```\n\nSee [`packages/litmus/README.md`](packages/litmus/README.md) for the npm-facing usage docs,\nand [polygraph.so](https://polygraph.so) for the methodology and proof format.\n\n## Develop\n\n```bash\npnpm install\npnpm -r typecheck\npnpm -r test\npnpm --filter @polygraphso/litmus build # → packages/litmus/dist\n```\n\n## Release\n\n`@polygraphso/litmus` is versioned in `packages/litmus/package.json`. Tag to publish:\n\n```bash\ngit tag litmus-v && git push origin litmus-v\n```\n\nThe `Publish @polygraphso/litmus` workflow builds, typechecks, tests, and publishes with\nnpm provenance. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the full process and the\nlocal-development workflow for downstream consumers.\n\n## License\n\n[Apache-2.0](LICENSE) — © polygraph.so.\n", "bytes": 11286, "sha": "fef5a33871dca568cfb562fa019f22d20a5dc70653f0eb6bd2fc137a41b9257f", "repo_slug": "polygraphso/litmus", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/mcp_io_github_polygraphso_litmus_3f85fa2f/readme" }