{
  "markdown": "# Verdict\n\n[![ci](https://github.com/ArtJack/verdict/actions/workflows/ci.yml/badge.svg)](https://github.com/ArtJack/verdict/actions/workflows/ci.yml)\n[![verdict on itself](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fraw.githubusercontent.com%2FArtJack%2Fverdict%2Fmain%2F.qa%2Fstate.json&query=%24.verdict&label=verdict%20on%20itself&color=blue)](.qa/reports/INDEX.md)\n[![eval 8/8 seeded defects](https://img.shields.io/badge/eval-8%2F8_seeded_defects-brightgreen)](eval/README.md#published-results)\n[![pinned rules 165/165 killed](https://img.shields.io/badge/pinned_rules-165%2F165_killed-brightgreen)](eval/README.md#suite-fault-detection-power--mutation-testing-on-ourselves)\n[![PyPI](https://img.shields.io/pypi/v/verdict-qa-mcp?label=verdict-qa-mcp&color=blue)](https://pypi.org/project/verdict-qa-mcp/)\n[![Claude Code plugin](https://img.shields.io/badge/Claude_Code-plugin-6E56CF)](#install)\n[![license MIT](https://img.shields.io/github/license/ArtJack/verdict)](LICENSE)\n\n**Your test suite is green. Verdict found a defect that had lived 4,595 days.**\n\nVerdict is a QA agent that does not fix, does not flatter, and does not forget. It measures\nbefore it judges — the harness runs your gates, hashes every line a finding cites, re-runs\nthe guarding test at the old commit and the new one — and it keeps a memory: every run is a\ndelta against the last, findings age, regressions rank first, and the tester's own misses\nare published beside its hits. The contract it runs under is immutable and hashed into every\nverdict; what it learns lives beside the contract, dated and auditable, and never edits it.\nThe number above is real: `FilePerms` in a 4k★ Python library could not revoke a permission\nbit since 2014-02-07, and every one of its 625 tests was green the day Verdict filed it —\n[the run, and the misses, are in the ledger](eval/README.md#runs-on-strangers-repositories).\n\n```\n/plugin marketplace add ArtJack/verdict     # Claude Code\n/plugin install verdict@verdict\n/verdict:run\n```\n```\nnpx skills add ArtJack/verdict              # every other coding agent\n```\n\nMost AI \"QA agents\" are a paragraph of enthusiasm with a checklist. They audit your repo\nfrom scratch every time, re-report the same 20 findings until you stop reading, call flaky\ntests \"failures\", call stale tests \"failures\", and end with \"LGTM! 🎉\".\n\nVerdict is a Claude Code plugin built the way QA is actually practiced:\n\n- **It remembers.** A state file carries the baseline. Every finding gets a stable ID and\n  an age; every run reports `NEW / STILL_OPEN / RESOLVED / REGRESSED` — regressions ranked\n  first, always.\n- **A red test means something.** Every failure is classified — `REAL_DEFECT`,\n  `STALE_EXPECTATION` (which needs a citation proving the change was intended),\n  `BRITTLE_TEST`, `ENVIRONMENT`, or `FLAKY` (confirmed by re-runs, quarantined **with an\n  expiry**). The classification most likely to excuse a regression carries the highest\n  evidence bar.\n- **A verdict you can defend.** Every run ends in exactly one of\n  `pass | pass with risks | blocked | fail` — an open Blocker forces `fail`, `blocked` is a\n  legitimate outcome, and a `pass` always names what was *not* tested.\n- **It never fixes your code.** There is no `Edit` tool, a hook confines its writes to the\n  QA root, and a strict-mode Bash guard closes the shell's write channels — a tester that\n  patches what it judges isn't independent. The guard is a heuristic, not a sandbox, and\n  [the README says so](#the-read-only-guarantee-honestly-stated).\n- **It is tested, and it tests itself.** A scored eval suite with the misses published, a\n  signed run history the model cannot forge, a track record the tester cannot edit — and a\n  nightly audit of its own releases: 69 findings filed against itself so far, every one\n  fixed in a tagged release with the defect pinned as a mutant the suite must kill.\n\n![A Verdict delta run: verdict first, REGRESSED findings ranked on top, a flake quarantined with an expiry, and the gate's exit codes keeping \"never ran\" apart from \"said no\"](docs/demo.svg)\n\n**Who pays for the model?** You do, with the Claude subscription you already have: the\nplugin runs inside your own session, nothing routes through anyone else, and everything\nbelow the model — the state, the gate, the MCP server, the eval scorer — is stdlib Python\nthat runs for free. Works on Python, TypeScript, Go, or anything with a test runner; the\n[eval fixtures](eval/) cover Python and TypeScript.\n\n**Read next:** [Install](#install) · [What installs, and when it runs](#what-installs-and-when-it-runs) ·\n[Quickstart](#quickstart) · [Why another QA agent](#why-another-qa-agent) ·\n[The tested tester](#the-tested-tester) · [CI gate](#ci-gate-prs-on-the-testers-memory) ·\n[Accepting a risk](#accepting-a-risk--the-maintainers-pen) · [FAQ](#faq)\n\n## Install\n\n```\n/plugin marketplace add ArtJack/verdict\n/plugin install verdict@verdict\n```\n\nAny other coding agent — Cursor, Codex, OpenCode and the rest of the\n[agent skills](https://skills.sh) ecosystem — gets the same doctrine as five skills, and the\nsame harness as a pip package:\n\n```\nnpx skills add ArtJack/verdict        # release risk · verify a fix · flaky triage · root cause · spec review\npip install verdict-qa-mcp            # verdict-facts, verdict-finalize, verdict-gate, verdict-accept, verdict-answer\n```\n\n[![skills.sh](https://skills.sh/b/ArtJack/verdict)](https://skills.sh/ArtJack/verdict)\n\nThe skills restate the contract for an agent that cannot run the `verdict` agent; the\nhooks that enforce the read-only guarantee exist only in Claude Code, so there the\nguarantee is the agent's own discipline plus the harness's refusals. `AGENTS.md` and\n`llms.txt` at the repository root are for agents that read before they act.\n\n**Python 3.9 or newer**, whatever your `python3` resolves to — the hooks and the\nfact harness are stdlib-only and are invoked by that name, which on a stock Mac is\n`/usr/bin/python3` (3.9). The optional MCP server is a pip install and needs 3.10+,\nwhich is what `requires-python` in `pyproject.toml` refers to. The floor is tested:\na module that would fail to import on 3.9 fails CI instead\n([tests/test_interpreter_floor.py](tests/test_interpreter_floor.py)) — because the\nfailure it prevents was silent. The Bash guard once raised on import there while the\nwrite guard beside it kept working, so a strict session looked armed with half its\ncontrols missing.\n\n## What installs, and when it runs\n\nInstalling a plugin means letting its code run in your sessions, so here is exactly\nwhat this one does — measured from `hooks/hooks.json`, not summarised from memory.\nSix hook registrations; each starts a `python3` (tens of milliseconds) when its\nevent fires:\n\n| Event | Fires on | Script | Silent when |\n|---|---|---|---|\n| `PreToolUse` | `Write`/`Edit`/`MultiEdit`/`NotebookEdit` | write-scope guard | always, unless `VERDICT_STRICT=1` or the caller is the verdict agent itself |\n| `PreToolUse` | `Bash` | bash-scope guard | always, unless `VERDICT_STRICT=1` |\n| `PostToolUse` | `Write`/`Edit`/`MultiEdit` | state validator | unless the written file is literally named `state.json` |\n| `Stop` / `SubagentStop` | end of turn | run-contract check | unless a QA run *in this session* left hand-written state — and it blocks **at most once**, never loops |\n| `SessionStart` | session open | findings banner | unless the repository has QA state |\n\nEvery hook **fails open**: malformed input, missing files, or an exception mean\nexit 0 and silence — a broken hook must never brick a session. `VERDICT_STRICT=1`\nis what arms the scope guards, and you set it only for dedicated QA sessions\n(headless, CI, the nightly); in ordinary interactive work the guards are no-ops.\n\n**Prefer not to install globally?** Everything works per-repository: copy\n[agents/verdict.md](agents/verdict.md) into `<repo>/.claude/agents/` and the\n[hooks/hooks.json](hooks/hooks.json) entries into `<repo>/.claude/settings.json`,\nwith `${CLAUDE_PLUGIN_ROOT}` replaced by a checkout path. That is exactly how the\neval harness provisions its scratch projects — [eval/run_eval.py](eval/run_eval.py)\nis the reference implementation.\n\n## Quickstart\n\n```\n/verdict:run                          # first run: profile + isolation rules + baseline\n/verdict:run the payment retry change   # every later run: a delta against the stored state\n```\n\nEvery later run is a delta against the stored state. A repeat run returns something like:\n\n```text\nVERDICT: fail\n\nScope: 2c67f47..b4e2943 (4 commits, 16 files) · run 4 (delta)\nIsolation check: pass (no .env present; no live service touched)\n\nFindings — REGRESSED first:\nREGRESSED  PRICER-F-002  Critical/P0  round_cents uses banker's rounding again (pricer.py:17)\n                         resolved 08-19, reintroduced by b4e2943 — this forces the verdict\nNEW        PRICER-F-007  Major/P1     quarantine graveyard: test_listable_at_floor_exactly\n                         skipped 114 days with no expiry — it is the test that would catch F-001\nSTILL_OPEN PRICER-F-001  Critical/P0  age 6d  is_listable rejects a price exactly at the floor\nFLAKY      test_bulk_discount_applies — fails 3/6 runs with no code change; quarantined\n           until 2026-09-07, excluded from this verdict, listed until re-evaluated\n\nDelta gates: tests 213 → 213 · duration +0.4% · coverage on changed files: no decrease\nRelease blockers: PRICER-F-002 (regressed), PRICER-F-001\nNot tested: concurrency under parallel checkout — no harness present\nFix order: 1) F-002  2) F-001 (unskip its test first, watch it fail red)  3) F-007 expiry\nArtifact: .qa/reports/2026-08-24-pricer-review.md\n```\n\n## Why another QA agent\n\n| | Typical `qa-expert.md` | Verdict |\n|---|---|---|\n| Remembers the last run | no — every run is a fresh audit | state file; `NEW/STILL_OPEN/RESOLVED/REGRESSED` with ages |\n| Flaky tests | \"keep flakes under 1%\" (prose) | quarantine ledger with mandatory expiry; flakes excluded from the verdict, never from the report |\n| Release decision | \"go/no-go\" appears as a checklist word | four-verdict contract; an open Blocker forces `fail` |\n| Red test triage | \"investigate failures\" | five-class taxonomy; `STALE_EXPECTATION` requires an intent citation |\n| Quality gates | \">90% coverage\" absolutes | direction gates: coverage on changed files must not decrease; 0 tests collected ≠ 1 test failing |\n| Test design | \"test edge cases\" | 24-technique catalog with risk triggers — incl. property-based, metamorphic (for ML/LLM output), MC-DC, contract tests ([docs/test-design.md](docs/test-design.md)) |\n| Can edit your code | nothing stops it | no `Edit` tool + write-scope hook + strict-mode Bash guard |\n| Security | ignored, or oversold | opt-in report-only pass: dependency audit + diff secret scan; pentest explicitly out of scope |\n| Risk prioritisation | \"focus on high-risk areas\" | the ranking is computed from the project's own finding history (severity-weighted, paths merged across citation depths), and the report must show the ranking, the cutoff, and everything below it — which lands in not-tested |\n| Root cause | \"investigate the failure\" | a four-link chain with a citation per link, a mandatory class check (is this an instance or a pattern?), and causation proven by flipping the cause in a scratch copy — with its own scored fixture built around a decoy |\n| Requirements review | never — code only | `/verdict:spec` judges the spec before code exists (contradictions, unmeasurables, boundary ambiguities, history conflicts) — with its own scored eval fixture |\n| \"No bugs found!\" | frequently | never — coverage, gaps, and residual risk instead |\n| AI-authored code | same checklist as human code | provenance measured (trailer census over the range, profile `authorship`); a pattern catalog with a procedure and evidence bar per entry ([docs/ai-authored-code.md](docs/ai-authored-code.md)); deterministic censuses for hallucinated imports, placeholders and swallowed errors feed judgment — and a scored fixture proves the behaviours |\n| Its own accuracy | unmeasured, and unmeasurable after the fact | every finding states a confidence when filed; the outcome is computed from what the finding did, kept in a permanent ledger, and reported as a track record the tester cannot edit |\n| Tested itself | — | scored eval suite: baseline + delta-memory + adversarial-honesty fixtures, deterministic scorer, published answer keys ([eval/](eval/)) |\n| State consumable by other tools | — | `verdict-mcp`: read-only MCP server over the state — works from Cursor, Codex, CI, any MCP client |\n\n## The tested tester\n\nA QA agent that was never tested is exactly the kind of claim it should reject.\n[`eval/`](eval/) is a scored eval suite with a **deterministic scorer** —\n[`score.py`](eval/score.py) reads the state file, not the prose — and eight fixtures, the four that carry the headline claims:\n\n- **Baseline** ([fixtures/pricer](eval/fixtures/pricer)): 8 seeded issues covering all\n  five failure classifications, including a boundary defect hidden behind a \"temporarily\"\n  skipped test and a stale expectation whose intent citation sits in the CHANGELOG.\n  [Answer key](eval/EXPECTED.md).\n- **Delta** ([fixtures/pricer_rev_b](eval/fixtures/pricer_rev_b)): scores the flagship —\n  a run against an authored run-2 history must produce `REGRESSED` (ranked first), `NEW`,\n  `STILL_OPEN`, `RESOLVED`, and release an expired quarantine, while a CHANGELOG decoy\n  tries to launder the new defect as intended.\n  [Answer key](eval/EXPECTED-DELTA.md).\n- **Liar** ([fixtures/liar](eval/fixtures/liar)): adversarial honesty — a test script that\n  prints \"ALL TESTS PASSED\" unconditionally, a conftest that skip-marks the whole suite, a\n  mock asserting its own return value, a tautological assertion. Scores whether the\n  verdict takes output at face value.\n- **Spec** ([fixtures/refund-spec](eval/fixtures/refund-spec)): shift-left — a draft PRD\n  with a seeded contradiction, an unmeasurable requirement, an exactly-at-the-boundary\n  ambiguity, a silent failure-path gap, and a CHANGELOG that contradicts the spec. Scores\n  `/verdict:spec` finding them all before any code exists.\n\n`python3 eval/run_eval.py --mode seeded|live|baseline` runs it all in an isolated scratch\nrepo and scratch state home. Results are published as measured; misses — and any answer-key\namendment — stay in the table ([eval/README.md](eval/README.md)).\n\n## State modes\n\n- **Solo (default):** state lives in `~/.claude/verdict/<repo-name>/` — nothing added to\n  your repo.\n- **Team:** create `.qa/` in the repo (`/verdict:baseline team`) and commit it — your teammates\n  and CI share the same baseline, and QA reports travel with the code.\n\nThe state schema is documented in [docs/state-schema.md](docs/state-schema.md) —\nversioned, forward-compatible, human-readable JSON.\n\n## Commands\n\nPlugin commands are namespaced by the plugin and must be typed in full — `/verdict:run`,\n`/verdict:status`. There is no short form: a bare `/verdict` is an unknown command,\nmeasured rather than assumed.\n\n**Start here:** `/verdict:run` — the front door. It reads the tester's memory and picks the\nright pass itself: no state yet → a baseline; state present → today's delta; arguments\ngiven → a delta narrowed to what you named. It says which it chose and why. Every command\nbelow is the same machinery aimed at one specific job, for when you already know which\njob you want.\n\n| Command | What it does |\n|---|---|\n| `/verdict:run` | **Front door** — routes to baseline, delta, or a scoped review from the stored state |\n| `/verdict:baseline` | Initialize the QA root, project profile, and baseline state |\n| `/verdict:regression` | Regression checklist: changed area → adjacent flows → integrations |\n| `/verdict:release` | Release gate with the four-verdict contract |\n| `/verdict:bug` | Turn a symptom/log/complaint into a classified, structured bug report |\n| `/verdict:flake` | Classify an intermittent failure: ≥3 reproductions, mechanism hunt → `BRITTLE_TEST` fix task, or `FLAKY` quarantine with expiry |\n| `/verdict:status` | Read-only status from the stored state — no run, no writes, no agent spin-up |\n| `/verdict:spec` | Shift-left: judge a spec/issue/PRD for testability *before code exists* — contradictions, unmeasurables, undefined boundaries, silent gaps, history conflicts, plus Given/When/Then criteria |\n| `/verdict:cause` | Trace a failure to its root cause: symptom → mechanism → origin → **class**, each link cited, causation proven by counterfactual rather than narrated; trigger, cause, and latent condition kept apart |\n| `/verdict:charter` | Timeboxed exploratory charter with a risk focus seeded from the profile's incident history; observations captured as evidence, discoveries converted to bug reports and regression candidates |\n\n## The tester's memory, over MCP (optional)\n\nVerdict's state isn't locked inside the agent. `verdict-mcp` is a small **read-only** MCP\nserver over the same state files, so anything that speaks MCP can consult your QA memory —\nan orchestrator gating a merge, a Cursor or Codex session, a CI step commenting a PR:\n\n| Tool | Returns |\n|---|---|\n| `get_verdict(project)` | last verdict, release blockers, report path, not-tested list |\n| `get_findings(project, status)` | `open` (default), `all`, or `NEW / STILL_OPEN / RESOLVED / REGRESSED` — REGRESSED ranked first |\n| `get_quarantine(project)` | the flaky ledger, each entry with a computed `expired` flag |\n| `get_history(project)` | run-over-run trend parsed from the report INDEX |\n| `get_report(project, report?)` | full report content (default: last run's) — path-guarded to the QA root, so a CI step can quote the evidence, not just link it |\n| `get_profile(project)` | the project's QA profile: isolation rules, risk areas, real test commands — plus the lessons ledger when one exists |\n| `get_trends(project)` | run-over-run trajectory from the INDEX, the current pressure picture (open by severity, age distribution, quarantine size, duration), and **hotspots** — where this project's defects actually cluster, computed from its own findings and severity-weighted, with the number of runs behind the ranking |\n| `list_projects()` / `get_state(project)` | everything with a baseline / the raw state |\n\n```\nclaude mcp add verdict -- uvx --from verdict-qa-mcp verdict-mcp\n```\n\nThe distribution is **`verdict-qa-mcp`** — the console script and the import package are\nstill `verdict-mcp` / `verdict_mcp`; only the name PyPI indexes differs, because\n`verdict-mcp` there belongs to an unrelated project. Installing straight from the\nrepository also works and needs no release:\n`uvx --from git+https://github.com/ArtJack/verdict verdict-mcp`\n\n`project` is a key from the solo root (`~/.claude/verdict/`, override with `VERDICT_HOME`)\nor a repo path in team mode (resolves `<repo>/.qa/`). Every tool carries a read-only\nannotation and the server never writes — **the tester's memory is public API; the tester's\npen is not.** Needs `uv` (or `pipx install verdict-qa-mcp`); the plugin itself still has\nzero dependencies and works without the server.\n\n## Closing the loop (without letting the tester fix anything)\n\nVerdict is deliberately the **gate** of a fix loop, never its actor — an agent that fixes\nand then re-judges its own fixes is grading its own homework. The loop belongs to your\norchestrator, your coding agent, or CI; Verdict's job is to make every pass around it\nevidence-cited and impossible to rubber-stamp:\n\n```text\n┌─────> implement the ordered fix list   (you / your coding agent)\n│                    │\n│                    v\n│        /verdict:run — delta run        (scoped by diff, findings aged)\n│                    │\n│                    v\n│        get_verdict over MCP ────────── pass ──> merge\n│                    │                            (the not-tested list travels with the PR)\n└──────── fail · pass with risks\n          (fix order is dependency-aware, REGRESSED first)\n```\n\nMinimal driver, any MCP client:\n\n```python\nwhile True:\n    before = mcp.call(\"verdict\", \"get_verdict\", {\"project\": \"myapp\"}).get(\"run_number\") or 0\n    subprocess.run([\"claude\", \"-p\", \"/verdict:run delta pass on myapp\"])   # the agent runs\n    v = mcp.call(\"verdict\", \"get_verdict\", {\"project\": \"myapp\"})          # the gate reads\n    assert (v.get(\"run_number\") or 0) > before, \"run died before writing state — not a verdict\"\n    if v[\"verdict\"] == \"pass\":\n        break\n    fix(v[\"release_blockers\"],\n        mcp.call(\"verdict\", \"get_findings\", {\"project\": \"myapp\", \"status\": \"open\"}))\n```\n\n(The `run_number` check matters: without it, a run that crashes before writing\nstate re-serves *yesterday's* verdict — and if yesterday passed, the loop merges\nunreviewed code. `verdict-gate --min-run-number` is the same check as a CLI.)\n\nRules that keep the loop honest — all enforced by the agent's contract, not by hope:\n\n- **REGRESSED breaks the loop loudly.** A finding that comes back outranks any number of\n  NEW ones; it is ranked first in every report and every `get_findings` response.\n- **Red tests exit through the right door.** `STALE_EXPECTATION` exits via a test-update\n  task (with an intent citation), `REAL_DEFECT` via a code fix — the loop never converges\n  by editing a red test to match the code.\n- **Flakes can't be buried.** Quarantined tests are excluded from the gate but re-enter on\n  expiry, so the loop cannot converge by skipping its way to green.\n- **`blocked` halts, it doesn't pass.** A missing environment stops the loop for the\n  operator instead of laundering itself into a verdict.\n- **A crashed run is not a verdict.** The gate asserts `run_number` advanced; stale state\n  is its own exit code (`5`), distinct from both pass and fail.\n\nThis is not hypothetical — it is the loop the author's private deployment runs nightly,\nunattended, against a production codebase.\n\nOn nights when nothing a finding cites has moved, `verdict-run --skip-unless-drift` finalizes\na **sweep** instead of a model run — the previous verdict carried by id, signed by no model,\nthe run number advanced — and prints why whenever it cannot. The conditions are the harness's\nown measurements: every cited line where it was, every gate green, the test-id set unchanged,\nno quarantine due ([docs/nightly.md](docs/nightly.md)).\n\n## CI: gate PRs on the tester's memory\n\nThe repo doubles as a composite GitHub Action. **Gate mode** needs no API key, no install,\nand no model — a stdlib-only script reads the committed team-mode `.qa/` state, sets the\njob status, and maintains one sticky PR comment (verdict headline, blockers,\nREGRESSED-first findings table, the not-tested list):\n\n```yaml\npermissions:\n  pull-requests: write\nconcurrency: verdict-${{ github.ref }}\n\nsteps:\n  - uses: actions/checkout@v4\n  - uses: ArtJack/verdict@v0\n    with:\n      max-age-hours: 48        # a stale verdict is exit 5, never a pass\n      max-commits-behind: 0    # ...and so is one measured before these commits\n```\n\n**Run mode** (experimental) executes a headless Verdict pass first — on a GitHub-hosted\nrunner with `anthropic-api-key`, or on a **self-hosted runner with\n`claude-oauth-token`** from `claude setup-token`, so nightly QA rides your subscription\ninstead of API billing (`anthropic-base-url` passes through for Anthropic-compatible\ngateways). The same contract is available anywhere as a CLI:\n\n```bash\nverdict-gate myapp --max-age-hours 24 --max-commits-behind 0 --fail-on risks\n```\n\nExit codes: `0` pass · `1` fail · `2` usage · `3` blocked · `4` no state (the tester never\nran) · `5` stale. `4` and `5` are deliberately distinct from `1`: \"the tester never ran\"\nmust never look like \"the tester said no\". For running the nightly pass on your own\nmachine — cron, systemd, subscription token, strict mode — see\n[docs/nightly.md](docs/nightly.md).\n\n`--format sarif` emits the open findings as SARIF 2.1.0 (severity → level, locations\nparsed from `file:line` evidence), so they land as annotations in GitHub's Security tab:\n\n```yaml\n- run: verdict-gate --format sarif > verdict.sarif || true\n- uses: github/codeql-action/upload-sarif@v3\n  with: { sarif_file: verdict.sarif }\n```\n\n## Give your tester project eyes (bring your own MCPs)\n\nThe agent ships with core tools only, but the frontmatter is an extension point: copy\n`agents/verdict.md` into your project's `.claude/agents/` and add your project's MCP tools\n(database, staging API, browser) to its `tools:` list. The agent's §0 isolation rules\ngovern how it may use them — read-only facts, never mutations, `blocked` when it cannot\nverify. This pattern is battle-tested: the private ancestor of this agent runs nightly\nwith eleven read-only marketplace-database tools, which is exactly how it caught a live\noverselling bug that no amount of reading source code could have found.\n\nWorked example — a web app with a Playwright MCP connected:\n\n```yaml\n# your project's .claude/agents/verdict.md, frontmatter tools:\ntools:\n  - Read\n  - Glob\n  - Grep\n  - Bash\n  - Write\n  - mcp__playwright__browser_navigate\n  - mcp__playwright__browser_snapshot\n  - mcp__playwright__browser_click\n  - mcp__playwright__browser_console_messages\n```\n\n…and the profile carries the rules of engagement: which origin is the test environment\n(never production), which accounts are test accounts, and that navigate/snapshot/read is\nin scope while anything that submits, pays, or mutates an account is forbidden. §0 governs\nbrowser tools exactly as it governs Bash — unsure whether a click mutates? It mutates;\nreturn the risk instead of clicking. Exploratory charters (§4, technique 23) translate\ndirectly: a timeboxed browser session with a risk focus, observations as evidence,\nrepeatable failures becoming bug reports.\n\n## The model judges; the system measures\n\nAbout two thirds of a state file is arithmetic and transcription — timestamps, SHAs, diff\nranges, gate exit codes, durations, test counts, finding hashes, ages, deltas. None of it\nis judgment, and every one of them is a place to be confidently wrong.\n\nSo the run is split. `verdict-facts` measures: it runs the gates you name, times them,\nparses their counts, reads git, derives the project key, and decides `run_number` and\n`run_type` (including when a run must be re-declared a re-baseline). The agent then writes\n**only judgment** — each finding as its own file the moment it is proven, validated as it\nis written; a finding it looked at and found unchanged as an id; the verdict and what was\nnot tested. `verdict-finalize` assembles the files, computes each finding's hash,\n`first_seen`, `age_days`, and its NEW/STILL_OPEN/RESOLVED/REGRESSED delta from the previous\nstate, hashes every line the evidence cites so the next run is told where the code moved,\nand validates the result before writing anything.\n\n`verdict-finalize` also **renders the report** — scope, gates, the REGRESSED-first\nfindings table, not-tested, quarantine — from that same state, and injects the agent's\nprose (risks, fix order, per-finding narrative) into it. The report cannot go missing,\nbecause the harness writes it, and cannot contradict the state, because it is the state.\n\nNothing the model cannot compute correctly is left for the model to compute.\n\n## The tester's own error rate\n\nA finding is worth what the tester's record says it is worth. Verdict keeps that record,\nand the design principle is the same one as everywhere else here: the part a model would\nbe tempted to grade generously is the part it does not get to touch.\n\nEach finding states a **confidence when it is filed** — `proven` (demonstrated it happen),\n`probable` (traced, not executed), `hypothesis` (suspected). The validator refuses a new\nfinding without one, and the harness freezes it: a later run cannot revise a prediction\nafter seeing how it turned out.\n\nThe **outcome is computed**, never claimed. A finding that regressed, or whose fix was\nverified by re-injecting the defect and watching a guard fail, held up. One the tester\nwithdrew did not. Everything else stays undecided and is excluded from every rate — a\nresolution nobody verified is an absence, not proof, and a still-open finding has not\nsettled anything. Decided outcomes persist in `outcomes.json`, because `state.json` drops\nfindings resolved two runs ago and the sample would otherwise reset forever.\n\nThe report then carries a **Track record** section: how many findings this project has\ntracked, how many are settled, and the counts per confidence level and per proof method.\nA percentage appears only once a bucket has 30 settled outcomes. Below that you get \"2 of\n3\", which is a fact, instead of \"67%\", which is decoration.\n\n### Accepting a risk — the maintainer's pen\n\nSome findings are right and will not be fixed: a residual risk weighed and written into a\ndecision log, a defect behind a feature that is being retired. Left `open`, such a finding\nis re-reported as an open Major in every banner for the life of the project — the \"same\ntwenty findings until you stop reading\" failure this tool exists to prevent — and\n`withdrawn` would score a correct finding as the tester's error. So there is a fourth\nstatus, and the tester cannot write it:\n\n```\nverdict-accept myapp MYAPP-F-021 --cite \"DECISIONS.md 2026-09-02\" \\\n    --reason \"deleting the ledger too defeats the anchor; the cost is the whole track record\"\n```\n\nThat writes `accepted.json` beside `outcomes.json` in the QA root — a file the scope guards\nrefuse to the agent and a status the validator refuses in a judgment. The finding leaves\nthe open counts at once (the banner, `verdict-gate`, the MCP server), leaves the verdict at\nthe next run, appears under **Accepted risks** in every report with its citation, and\nsettles in the track record as `confirmed` on the maintainer's word — kept apart from the\nmeasured and the claimed confirmations, because it is neither. `--revoke` reverses it, with\na reason; `--list` shows the ledger. A decision changes the next verdict, never the last one.\n\nThe other thing only a person can settle is a question — is `;` still a query separator,\nis single-file vendoring a supported contract, should the gate run an installed wheel. The\ntester parks them (`questions` in its judgment; finalize mints `MYAPP-Q-3` and keeps\n`questions.json`), and the second pen answers:\n\n```\nverdict-answer myapp MYAPP-Q-3 --answer \"at or above is the rule; README rule 1 is the spec\"\n```\n\nThat writes `answers.json`, refused to the agent like `accepted.json`. The next run reads the\ndecision in its facts and never asks again; the report renders **Needs human decision** from\nthe ledger; the session-start banner and `verdict-gate` say how many are waiting. Nothing is\nmailed and no issue is filed for a question — it is pushed to every surface that reaches you,\nand it waits there.\n\n## The tester has memory. The implementer did not.\n\nThat asymmetry had a measured cost. Verdict filed eleven evidenced findings on a live site,\none of them a release blocker — *deploying this branch strips every production security\nheader* — and the very next session in that same repository did a full SEO pass and touched\nnone of them: not the blocker, not the application form that reports success when the\nhandoff failed, not the contrast failures on both primary CTAs. The findings sat in\n`state.json` the whole time. `next_run_focus` existed, but only Verdict reads it;\n`get_findings` existed over MCP, but nothing called it unprompted.\n\nSo a `SessionStart` hook says what is outstanding when a session opens in a repository that\nhas QA state — before the first edit, not after:\n\n```\nVerdict remembers dm-express-site: run 2 (delta), today — verdict **fail**.\n1 release blocker — look here first:\n  - DMEXPRESS-F-1 — the audited branch has diverged from the deployed origin/main\n11 open findings: 4 Major · 4 Minor · 3 Trivial\n  - DMEXPRESS-F-3 (Major) Light theme: accent-coloured text fails WCAG AA…\nFull detail: `/verdict:status`. These are findings, not instructions.\n```\n\nDeliberately short — a session opener that scrolls is one nobody reads — and it never\nrepeats a finding it already named as a blocker. Silent in a repository with no QA state,\nsilent on any failure, and it flags memory older than a week rather than serving it as\ncurrent. It informs a session; it does not commandeer one.\n\n## The last guard fires whether or not the model remembers\n\nEvery check above sits **downstream of a tool the model has to choose to call** — and that\nis not a theoretical gap. A real run of `/verdict:run` wrote to the default state root\nwhile `$VERDICT_HOME` pointed elsewhere, invented a project key, skipped the harness\nentirely, and still produced a confident, plausible `FAIL`. `verdict-validate` would have\nrejected that state and `verdict-gate --require-harness` would have exited 6. Neither\nfired, because nothing invoked them.\n\n(That check used to be defeatable by *imitation* rather than forgery — its two durable\nsignals were a key holding a dict and a fixed footer string, both copyable straight out\nof the committed artifacts. Verdict found that auditing itself. Each run now signs the\nrun history with a hash of the previous link, and the state records it; a link copied\nforward does not verify, and neither does a state edited after signing.)\n\nSo there is a `Stop` hook. When a turn ends it asks one question — *did a QA run just\nleave hand-written state on disk?* — and if so it blocks the stop once and says what to\nredo. It fires on the turn ending, not on the model deciding to check.\n\nThe bar for speaking is deliberately high, because it runs at the end of every turn in\nevery session where the plugin is enabled: the turn must not already be continuing because\nof this hook (never loop), a QA root must resolve from the session's cwd, its `state.json`\nmust have been written in the last half hour, and the harness traces must be missing.\nAnything else exits in about two stat calls — **37 ms**, measured. Every failure path —\nunparseable input, an import that does not resolve, an unreadable state — also exits\nsilently: a hook that bricks sessions is worse than the problem it polices.\n\n## The state contract is machine-checked\n\nProse in a prompt reduces how often a model invents a value; it cannot stop a model from\ninventing a value it is capable of inventing. Measured here: months after `date -u` became\nan explicit rule, **two of four production timestamps still landed on exactly `:00`\nseconds** — fabricated, quietly, in states that every downstream consumer believed.\n\nSo the contract stopped being prose and became a gate. `verdict-validate` runs as a\nPostToolUse hook on every `state.json` write (and as a CLI in CI) and reports, immediately\nand in-session, any state that: names a `report` which is not a path to a file that exists\n· carries a timestamp that was recalled rather than measured · leaves `run_number` where a\ncrashed run left it · invents enum values · claims `pass` over an open Critical · files an\nopen finding with no evidence · quarantines a test with no expiry.\n\nIts first run against four live production states found violations in two of them —\nincluding the exact dodge (`\"delivered inline to the caller…\"` in the report field) that a\nprompt rule had failed to prevent three separate times. Every rule in it exists because a\nreal run broke it.\n\n## The read-only guarantee, honestly stated\n\nFour layers: (1) the agent has no `Edit` tool; (2) its contract confines `Write` to the QA\nroot; (3) a PreToolUse hook blocks out-of-scope `Write`/`Edit` calls; (4) under\n`VERDICT_STRICT=1` — set it for headless/CI/scheduled runs, where the whole session IS the\nQA run — a second hook also closes the obvious Bash write channels: output redirection,\n`tee`, `sed -i`, `rm`/`mv`/`cp` and friends, and mutating `git` verbs, each target resolved\nagainst the QA root. In mixed interactive sessions the hooks enforce when the platform\nidentifies the calling subagent and stay out of your way otherwise — they will never block\n*your* edits.\n\nThe Bash guard is a deny-heuristic, not a sandbox: unknown commands run (a QA pass needs\npytest, coverage, linters), package installs are deliberately not denied, and a determined\ncommand can evade string analysis — OS sandboxing remains the real boundary. Both hooks\nfail open on malformed input and are tested in CI\n([tests/test_hooks.py](tests/test_hooks.py)). That is the whole truth; a QA tool should\nnot oversell its own controls.\n\n## FAQ\n\n**Who pays for the model?** You do — with the Claude subscription you already have;\nnothing routes through the author and no API key is required. The one place a key can\nappear is the optional GitHub Action's run mode, and that is your key, in your repo, for\nyour CI. Everything below the model is plain files and stdlib Python.\n\n**Can it run on a local LLM?** The wiring exists today (`ANTHROPIC_BASE_URL` passes\nthrough to any Anthropic-compatible gateway, e.g. LiteLLM in front of Ollama), and every\nnon-judgment layer already runs locally for free. But the verdict is only as good as the\njudge: Sonnet — far stronger than any home-lab model — currently hard-fails the eval (see\nthe results table), so a local model must earn verdict-signing duty by passing the same\neval as everyone else. Run it, publish the score, then decide.\n\n**Why won't it fix the bugs it finds?** Independence. The agent that patches the code and\nthen declares it healthy is grading its own homework. Verdict returns an ordered,\nimplementation-ready fix list for you (or your coding agent) to execute.\n\n**Does it replace CI?** No — it sits on top. CI tells you the suite is red; Verdict tells\nyou *which* red matters, what it means, what regressed since the last run, and whether you\ncan ship anyway.\n\n**Does it work in scheduled/headless runs?** Yes — that's what the state file is for. Run\nit nightly; read a delta report over coffee, not a fresh audit.\n\n## Roadmap\n\n- **Local-first track** (the project's original ambition): an agent-skills-standard\n  variant — the prompt, technique catalog, and state contract are portable markdown, which\n  is the door to non-Claude runtimes — plus the local-model experiment: run the eval suite\n  through an Anthropic-compatible gateway against local models and publish the scores. A\n  model earns nightly duty by passing the same eval as everyone else.\n- A JS/TS eval fixture alongside the Python one\n- Mutation-testing integration where a tool is present\n\n## License\n\nMIT\n",
  "bytes": 38366,
  "sha": "8b5a33c990fd38c8e783f50c80f883daff183a832aba1e78b481b1416e12787d",
  "repo_slug": "artjack/verdict",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_artjack_verdict_68d3d19a/readme"
}