{
  "markdown": "# dsh-verify\n\n[中文](README.zh.md) | English\n\n> **Witness** — The browser is the judge.\n> The quality gate for agent-built web apps. Agents say done; the browser proves it.\n> *(Witness is the product name; `dsh-verify` is the package name — same thing.)*\n\n[![ci](https://github.com/263311487-ux/dsh-verify/actions/workflows/ci.yml/badge.svg)](https://github.com/263311487-ux/dsh-verify/actions/workflows/ci.yml)\n[![npm](https://img.shields.io/npm/v/dsh-verify)](https://www.npmjs.com/package/dsh-verify)\n[![MCP server](https://glama.ai/mcp/servers/263311487-ux/dsh-verify/badges/score.svg)](https://glama.ai/mcp/servers/263311487-ux/dsh-verify)\n[![awesome-dsh-plugin](https://img.shields.io/badge/awesome--dsh--plugin-listed-brightgreen)](https://github.com/awesome-dsh-plugin/awesome-dsh-plugin)\n[![GitHub stars](https://img.shields.io/github/stars/263311487-ux/dsh-verify?style=social)](https://github.com/263311487-ux/dsh-verify/stargazers)\n[![self-acceptance](https://github.com/263311487-ux/dsh-verify/actions/workflows/self-acceptance.yml/badge.svg)](https://github.com/263311487-ux/dsh-verify/actions/workflows/self-acceptance.yml)\n\n*If Witness catches something for you, ⭐ [star the repo](https://github.com/263311487-ux/dsh-verify) — it's how this project stays alive.*\n\n**You asked an AI to build a web app. It said \"done.\" Does it actually work?**\n\n`dsh-verify` opens a real browser and checks — so you never have to take the agent's word for it.\n\n![dsh-verify — Agents say done. The browser proves it.](assets/social-card.png)\n\n![dsh-verify in action](assets/hero.gif)\n\nThe quality gate for **agent-built web apps**. Works with any agent — DeepSeek Harness (dsh), Claude Code, Cursor, Copilot, Codex — and with any CI. You write what a human would check in a browser; a real browser executes it and returns a `PASS`/`FAIL` verdict with receipts (screenshots + diff images).\n\nNo LLM judges the outcome. **The browser is the judge.**\n\n![Same task, same AI, two builds — only a real browser tells the difference](assets/wow-compare.png)\n\nSame task. Same AI. Two builds. One missing CSS rule — the agent's self-review passed, a real browser caught it.\n\n---\n\n## Why this exists\n\nWe ran a 4-agent web team (spec writer → frontend dev → QA → reviewer). Their own review said:\n\n> ✅ \"All requirements met. No issues found.\"\n\nIn a real browser, the dark-mode toggle **did nothing** — the `.dark` class was toggled, but the CSS rule was never written. Every agent self-test passed because there was nothing in the page for the agents to run. **No one opened a real browser.**\n\nThat's the gap: *agents verify against what they believe they built, not against what a user actually experiences.* Unit tests and static checks can't catch a missing CSS rule.\n\n| Build | What the agents said | What a real browser says |\n|---|---|---|\n| `demo/buggy` | \"No issues found\" | ❌ **FAIL** — background never changes |\n| `demo/fixed` | one CSS rule added | ✅ **PASS** — theme flips |\n\nSame page. Same JS. One missing CSS rule. Two different verdicts.\n\n## Why not just ...?\n\n| What you might reach for | Its blind spot | What dsh-verify adds |\n|---|---|---|\n| **Hand-rolled Playwright scripts** | Every agent project re-writes the same boilerplate; nothing is reviewable as a spec | A JSON spec is the whole contract — write once, reuse across agents and CI |\n| **LLM judges (promptfoo-style evals)** | An LLM says \"looks right\" — it doesn't run the app or see the pixels | A real browser executes clicks, inputs, styles, and returns screenshot receipts |\n| **Agent built-in browser tools** | They're the agent's *hands* — they share the same blind spots as the code they just wrote | dsh-verify is an independent witness, not part of the agent being tested |\n| **Screenshot-only visual tools** | They catch pixel drift, not \"button does nothing\" | Behavior checks: click, expect text/class/style change, console errors, network errors |\n\nThe agent graded its own homework. dsh-verify re-grades it in a real browser.\n\n## Use it three ways\n\n| Entry point | What it's for | One-liner |\n|---|---|---|\n| **MCP server** | Your AI agent verifies its own deliverable, mid-session | `claude mcp add dsh-verify -- npx -y -p dsh-verify dsh-verify-mcp` |\n| **CLI** | You or your CI verify a build/URL | `npx dsh-verify --spec demo/fixed.json` |\n| **GitHub Action** | Every push runs real-browser checks | `uses: 263311487-ux/dsh-verify/.github/actions/dsh-verify@main` |\n\n### From any AI agent (MCP)\n\n```bash\nclaude mcp add dsh-verify -- npx -y -p dsh-verify dsh-verify-mcp\n```\n\nThen tell your agent, in plain words:\n\n> Verify http://localhost:3000 — click `#dark-toggle`, then check `body` background-color changed. Screenshot it.\n\nTools exposed: `verify_spec` (run a spec JSON), `verify_url` (inline checks, no files), `generate_and_verify` (the AI drafts the checklist, real Chromium executes it), `health`.\n\n### In CI (GitHub Action)\n\n```yaml\n- uses: 263311487-ux/dsh-verify/.github/actions/dsh-verify@main\n  with:\n    spec: demo/fixed.json       # spec file or glob\n    # url: https://staging.example.com   # optional override\n    # out: dsh-verify-out               # report output dir (default)\n```\n\nThe repo dogfoods it: the [dogfood workflow](.github/workflows/dogfood.yml) asserts the fixed build **passes** and the buggy build **fails** on every push.\n\n### On the command line\n\n```bash\nnpm install -g dsh-verify          # or: npx dsh-verify\nnpx playwright install chromium    # one-time browser download\nnpx dsh-verify --spec 'specs/*.json'\n# [PASS] specs/home.json (5/5)\n# [FAIL] specs/cart.json (4/5)\n#   ❌ expect_text #total: got \"0\" want \"99\"\n```\n\n## What's in the box\n\n- **Deterministic judge** — a real headless Chromium (or Firefox / WebKit) executes human-style checks: click, fill, text, classes, **computed styles**, URLs, console errors, network errors, pixels.\n- **Receipts, not vibes** — every run emits a self-contained HTML report with screenshots and red-highlighted diff images; `--json` for machines; exit `0`/`1` for CI.\n- **Visual regression** — screenshot baselines, pixel-diff with thresholds (`expect_screenshot`), refresh with `--update-baselines`.\n- **AI-drafted checklists** — `dsh-verify gen --url ... --prompt \"...\"` learns the page in a real browser, has an LLM draft the checklist, then executes it deterministically. The AI drafts; it never judges.\n- **Multi-browser** — `chromium` | `firefox` | `webkit` per spec or `--browser`.\n- **Zero framework lock-in** — a JSON spec is all there is. No config language, no SDK, no vendor.\n\n## Example spec\n\n```json\n{\n  \"title\": \"my app\",\n  \"serve\": \"dist\",\n  \"browser\": \"chromium\",\n  \"steps\": [\n    { \"action\": \"goto\", \"path\": \"/index.html\" },\n    { \"action\": \"click\", \"selector\": \"#count-btn\", \"count\": 3 },\n    { \"action\": \"expect_text\", \"selector\": \"#count-btn\", \"text\": \"Clicked: 3\" },\n    { \"action\": \"capture_style\", \"selector\": \"#page\", \"prop\": \"backgroundColor\", \"var\": \"bg_before\" },\n    { \"action\": \"click\", \"selector\": \"#color-btn\" },\n    { \"action\": \"expect_class\", \"selector\": \"#page\", \"class\": \"dark\", \"present\": true },\n    { \"action\": \"expect_style_changed\", \"selector\": \"#page\", \"prop\": \"backgroundColor\", \"var\": \"bg_before\" },\n    { \"action\": \"screenshot\", \"name\": \"final-state\" }\n  ]\n}\n```\n\nTop-level fields: `title`, `serve` (static dir) or `base` (target URL), `browser`, `steps`. Run many at once with a glob; exit is `0` only if **all** pass.\n\n## The report\n\nA self-contained HTML report — every step with a pass/fail badge, selector, and detail, plus screenshots:\n\n![dsh-verify report](assets/report-screenshot.png)\n\n## Agent Arena — bring your agent\n\nReal-browser benchmark for agent-built web apps: same 3 tasks, same human\nchecks, **open entry**. Run your model on the board in ~10 minutes:\n\n```bash\ngit clone https://github.com/263311487-ux/dsh-verify && cd dsh-verify\nnpm install && npx playwright install chromium\nexport LLM_API_KEY=sk-...          # any OpenAI-compatible model\nnode arena/run.mjs --agent \"gpt-5/single\" --task all --repeat 1 --submitter yourname\n```\n\nYour setup appears on the live leaderboard next to DeepSeek v4-flash / v4-pro:\n[agent-arena](https://263311487-ux.github.io/dsh-verify/arena/).\nFull rules in [docs/ARENA.md](docs/ARENA.md).\n\n## Prove it (run it yourself)\n\n```bash\ngit clone https://github.com/263311487-ux/dsh-verify && cd dsh-verify\nnpm install && npx playwright install chromium\nnpm run demo:fixed    # → PASS (11/11)\nnpm run demo:buggy    # → FAIL (exit 1) — the missing .dark rule, caught\nnpm test              # engine self-tests\n```\n\nThe repo's own CI runs exactly that — engine self-tests, then asserts fixed **passes** and buggy **fails** — so the tool verifies itself on every push.\n\n## Agent Arena — can agents ship working web apps?\n\nSame task, same prompt, same human checks — different agents, graded by dsh-verify in a real browser. Latest run (2026-08-19): **44/48 runs passed** across 2 models × 2 strategies × 3 tasks, 4 runs per cell. Two counterintuitive findings: the pricier **v4-pro single-shot scored below the cheaper v4-flash single-shot** (10/12 vs 11/12), and a real-browser self-check loop lifted v4-pro to **12/12** — while v4-flash's self-check crashed once when its own verification report came back as corrupt JSON. Every failure is reproducible and invisible to an LLM judge.\n\n[![Agent Arena](https://img.shields.io/badge/Agent%20Arena-live-3fb950)](https://263311487-ux.github.io/dsh-verify/arena/)\n\nSee [docs/ARENA.md](docs/ARENA.md) — methodology, the tasks, and how to run your own agent.\n\n## Badge your agent-built app\n\nBuilt something with an AI agent? Prove it in a real browser and show the world:\n\n```markdown\n[![agent deliverable: browser-verified](https://img.shields.io/badge/agent_deliverable-browser_verified-brightgreen?logo=playwright&logoColor=white)](https://github.com/263311487-ux/dsh-verify)\n```\n\nAdd a spec, wire the GitHub Action, and the badge is earned, not claimed. See [docs/verified-badge.md](docs/verified-badge.md).\n\n## Roadmap\n\n- [x] MCP server · AI-drafted checklists · visual regression · multi-browser · GitHub Action · dsh plugin\n- [ ] **Agent arena** — a public benchmark: give the same task to different agent setups, grade them in real browsers, publish the leaderboard\n- [ ] Spec recorder (browser extension: click through once → spec generated)\n- [ ] Cloud runs + shareable report links + PR comments\n\n## Related\n\n- [dsh-doublecheck](https://github.com/PerryLink/dsh-doublecheck) — delivery quality gate for DeepSeek Harness (/gate): requirements grill + evidence discipline. Complementary pair: /gate keeps the evidence honest, dsh-verify keeps the browser honest.\n- [falsify](https://github.com/263311487-ux/falsify) — the scientific thinking protocol for AI agents (hypothesis → falsify → evidence → calibrated conclusion). The pair: falsify catches the wrong conclusion, dsh-verify catches the broken output.\n- Featured in the DeepSeek Harness community — [Show Your Plugins: dsh-verify](https://github.com/deepseek-ai/deepseek-harness/discussions/2806) (48-run Agent Arena results in-thread)\n\n## License\n\nMIT\n",
  "bytes": 11109,
  "sha": "7b9fd654b118735ce8ce06ead0be52c3ef29d3eb28f4e95a586bc4485fcf1519",
  "repo_slug": "263311487-ux/dsh-verify",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_263311487_ux_dsh_verify_7958ad0f/readme"
}