io.github.polygraphso/litmus
Grade MCP servers A to F with the open behavioral litmus. npm: full toolset; hosted: lookups only.
Open source Repository Open in the app JSON README (API)
About
Grade MCP servers A to F with the open behavioral litmus. npm: full toolset; hosted: lookups only.
Details
- Kind
- MCP servers
- Topic
- Developer tools
- Publisher
- polygraphso
- Origin
- official
- Category
- ferramentas
- Transport
- http
- Version
- 0.36.0
- Stars
- 8
- Last push
- 2026-07-20T15:52:52Z
- Repository state
- ativo
- Language
- TypeScript
- License
- Apache-2.0
- Added
- 2026-08-29 04:01:12
- Updated
- 2026-08-29 04:01:12
- Origin id
io.github.polygraphso/litmus
README
<img src="https://www.polygraph.so/brand/mark.png" alt="polygraph" height="56" />
# litmus
**The open behavioral litmus harness for MCP servers — grade A–F, reproducible.**
<!-- Badges live in a one-row table so they stay on a single line on BOTH the repo page and the
GitHub Marketplace listing for the polygraph-mcp-gate action. The Marketplace stylesheet sets
`img { display: block }`, which stacks plain inline badges into a tall vertical column; table
cells lay out horizontally regardless. GitHub strips inline styles, so the cells keep their
default 1px border — that boxed look is expected. Don't revert to a plain badge line. -->
<table>
<tr>
<td><a href="https://www.npmjs.com/package/@polygraphso/litmus"><img alt="npm" src="https://img.shields.io/npm/v/@polygraphso/litmus?style=flat-square&labelColor=0d1117&color=6f42c1" /></a></td>
<td><a href="https://github.com/polygraphso/litmus/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/polygraphso/litmus/ci.yml?branch=main&style=flat-square&labelColor=0d1117&label=ci" /></a></td>
<td><a href="LICENSE"><img alt="license" src="https://img.shields.io/badge/license-Apache--2.0-555?style=flat-square&labelColor=0d1117" /></a></td>
<td><a href="https://glama.ai/mcp/servers/polygraphso/litmus"><img alt="Glama" src="https://glama.ai/mcp/servers/polygraphso/litmus/badges/score.svg" /></a></td>
<td><a href="https://smithery.ai/servers/ruben-sousa-dinis/polygraphso"><img alt="Smithery" src="https://smithery.ai/badge/ruben-sousa-dinis/polygraphso" /></a></td>
<td><a href="https://polygraph.so/mcp/npm/@polygraphso/litmus"><img alt="graded by polygraph" src="https://polygraph.so/api/badge?server=npm/@polygraphso/litmus" /></a></td>
</tr>
</table>
## Grade a server in one command
```bash
# -p is required: the package ships three bins, so npx must be told which to run
npx -y -p @polygraphso/litmus polygraphso-litmus litmus npm/@modelcontextprotocol/server-filesystem
```
Point it at an npm ref, a pypi ref, a `github/owner/repo` ref (cloned, built, and run sandboxed —
Docker required; the grade pins the commit SHA), an `https://` MCP endpoint, or a local entry
file. The harness connects
the way an agent would, fingerprints the exact tool surface, runs the four probe categories, and
prints the grade with the findings behind it — plus a deterministic evidence bundle on disk.
It **runs the target's code** (egress is Docker-sandboxed; without Docker, C-02 is skipped and the
grade caps at B), takes ~20–60s, and exits non-zero on D/F so it scripts anywhere. To dispute any
published grade, re-run this same command against the same server — open and deterministic means a
re-run reproduces the grade, or refutes it.
Looking up a grade someone already published takes under a second and runs nothing: the
[polygraph.so index](https://polygraph.so/rankings), or `check_server` from the MCP tools below.
## Use it from your agent — MCP server + plugin
For grade lookups, point any MCP client at polygraph's hosted endpoint, no install:
```bash
claude mcp add --transport http polygraph https://polygraph.so/api/mcp
```
or the raw config:
```json
{ "mcpServers": { "polygraph": { "url": "https://polygraph.so/api/mcp" } } }
```
This serves the lookup tools only (`check_server`, `list_servers`, `request_grade`); grading a
server yourself (`run_litmus`, `run_skill_litmus`) executes its code, so it needs the local
stdio install below.
The package also ships a stdio MCP server (`polygraphso-litmus-mcp`) with the full toolset, for
any MCP-capable client:
- **`check_server`** — read a server's *published* grade in under a second (no execution); the
pre-flight check before recommending or installing a server.
- **`list_servers`** — servers with a published grade, A first; paged (default 25 per call, with `grade`/`limit`/`offset` filters and a full-corpus summary).
- **`request_grade`** — record a grade request with polygraph.so ($1 one-time fee; graded within 48h of payment — the response carries the payment link).
- **`run_litmus`** — grade a server *now*: the full harness, grade + evidence returned to the agent.
- **`run_skill_litmus`** — grade a Claude Code / Agent Skill (static scan, A/B/D/F).
- **`verify_attestation`** — read the onchain proof behind a published grade (EAS on Base).
In **Claude Code**, the plugin wires the server plus two commands in one step:
```
/plugin marketplace add polygraphso/litmus
/plugin install polygraph@polygraphso
```
then `/polygraph:grade <server>` and `/polygraph:check <server>`. Cursor and manual JSON setups
are on [polygraph.so](https://polygraph.so/#install); full tool docs in
[`packages/litmus/README.md`](packages/litmus/README.md#use-it-from-an-ai-agent-mcp-server).
## Gate your CI on MCP grades — GitHub Action
Fail a build when an MCP **server** or an Agent **Skill** it ships grades **D/F** under the open
behavioral litmus. For servers it is hybrid — a fast lookup of the published grade, then the harness
when ungraded; for skills it is a fast static scan. Un-gradeable targets warn unless `strict`.
It's on the **[GitHub Marketplace](https://github.com/marketplace/actions/polygraph-mcp-gate)** as
`polygraphso/litmus@v1`. For a security gate, pin to a commit SHA rather than the mutable `@v1` tag:
```yaml
# .github/workflows/mcp-gate.yml
name: mcp-gate
on: [pull_request] # NOT pull_request_target — that exposes secrets to fork PRs
permissions:
contents: read
jobs:
gate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: polygraphso/litmus@<commit-sha> # pin to a SHA; resolve from the v1 release
with:
# Name the targets explicitly (recommended). Grading runs a server's code,
# so on a public repo prefer an allowlist over discovering PR-controlled config:
servers: |
npm/@modelcontextprotocol/server-filesystem
skills: |
./my-skill
# discover: "true" # opt in to auto-discovery (.mcp.json/.vscode/.cursor) — trusted repos only
# min-grade: B # stricter than the default D/F gate
# strict: "true" # also fail on targets that cannot be graded
```
**Inputs:** `servers` · `skills` · `discover` (default `false`) · `min-grade` · `strict` · `working-directory` · `version` · `bearer`. **Outputs:** `result` · `failed` · `report`.
**Security.** Grading a server **runs its code** (egress is Docker-sandboxed, but it still executes).
Trigger on `pull_request`, never `pull_request_target`. Keep `discover` off on public repos and name
targets explicitly — auto-discovered config is pull-request-controllable. `bearer` is sent as an
`Authorization` header to the target, so pass it only for an explicitly trusted, pinned remote — never
with discovery or on untrusted PRs, and keep it scoped and short-lived.
Not on GitHub? The gate is a plain command — `npx @polygraphso/litmus@0.20.0 ci` (pin the version) —
so it runs in any CI or as a pre-commit hook. A grade is a measurement, not a guarantee: re-run the
open harness to reproduce any result.
## What litmus is
This is the source for **[`@polygraphso/litmus`](https://www.npmjs.com/package/@polygraphso/litmus)**,
the open behavioral litmus harness for MCP servers from [polygraph.so](https://polygraph.so).
The harness connects to an MCP server the way an agent would, fingerprints its exact
tool surface, and runs four probe categories — **C-01** tool-output injection (static,
dynamic, and second-order — one tool's output weaponized as another's input), **C-02**
permission/egress (in a hardened default-deny Docker sandbox, matched host **and** port),
**C-03** sensitive-data handling (planted canaries), **C-04** adversarial-input handling
(malformed/oversized and jailbreak inputs) — then grades the server **A–F**. A passing grade is a
measurement, not a guarantee; the methodology and its disclosed limits are at
[polygraph.so](https://polygraph.so) (the open source here is the ground truth).
Alongside the grade, an npm target's dependency tree is checked against the
[osv.dev](https://osv.dev) vulnerability database and any vulnerable dependencies are reported as
**dependency advisories**. This is a separate, **point-in-time** signal — it is *advisory only*: it
never affects the A–F grade and is not part of the reproducible evidence (vulnerability data changes
over time, so folding it into the grade would break re-run reproducibility). It applies to npm
targets only; other target kinds report it as skipped. Resolution runs
`npm install --package-lock-only --ignore-scripts`, which resolves the tree without downloading
tarballs or running any package code. Opt out with `--no-deps-audit` (or `LITMUS_DEPS_AUDIT=0`).
The same package also grades **Claude Code / Agent Skills** (a `SKILL.md` + bundle) under a
**separate** static litmus (`litmus-skill-v3`): a deterministic byte-scan — **S-01** prompt
injection, **S-03** data-exfiltration instructions, **S-04** dangerous commands in the SKILL.md
body or bundled scripts (incl. base64-obfuscated `curl | bash`) — graded **A/B/D/F** and anchored
by a whole-directory **content hash**, plus a separate
advisory quality signal. It is *static* (no execution): an **A** is static-clean, not behavioral
proof. See [`packages/litmus/README.md`](packages/litmus/README.md#grade-a-skill).
The hosted, operator-run grading **service** is **not** in this repo — it lives in a
separate private repo and consumes this package from npm like any other client.
## Layout
This is a pnpm monorepo. Only **`@polygraphso/litmus`** is published; the
`@polygraph/*` packages are private building blocks that tsup bundles into it.
```
packages/
litmus/ # @polygraphso/litmus — the only published package (lib + 3 bins: CLI, skill CLI, MCP)
core/ # contract types, canonical JSON, identity helpers
probes/ # the harness: connect, fingerprint, grade, probe runners, sandbox
onchain/ # EAS attestation read + encode/decode (Base) — read-only, no minting
agent/ # agent-gate decision logic + live-fingerprint recheck
mcp/ # MCP server wrapper
cli/ # CLI commands + target/auth resolution
demo-*-mcp/ # demo MCP servers used as test fixtures
```
See [`packages/litmus/README.md`](packages/litmus/README.md) for the npm-facing usage docs,
and [polygraph.so](https://polygraph.so) for the methodology and proof format.
## Develop
```bash
pnpm install
pnpm -r typecheck
pnpm -r test
pnpm --filter @polygraphso/litmus build # → packages/litmus/dist
```
## Release
`@polygraphso/litmus` is versioned in `packages/litmus/package.json`. Tag to publish:
```bash
git tag litmus-v<x.y.z> && git push origin litmus-v<x.y.z>
```
The `Publish @polygraphso/litmus` workflow builds, typechecks, tests, and publishes with
npm provenance. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the full process and the
local-development workflow for downstream consumers.
## License
[Apache-2.0](LICENSE) — © polygraph.so.