\n\n\n \n \n \n\n\n**Everyone is vibecoding. Nobody is verifying. Umbra scores it.**\n\nUmbra is a deterministic Trust Score (0–100) for AI-generated code: the vibe\ncoding security scanner that verifies what your agent shipped, not what it\nclaimed. One command, fully local, evidence for every finding.\n\n**→ [umbra-badge.umbrabadge.workers.dev](https://umbra-badge.umbrabadge.workers.dev) — the landing, live badges and report pages**\n\n[](https://www.npmjs.com/package/@elberacasa/umbra)\n[](https://www.npmjs.com/package/@elberacasa/umbra)\n[](https://github.com/elberacasa/umbra/stargazers)\n[](https://umbra-badge.umbrabadge.workers.dev)\n[](https://glama.ai/mcp/servers/elberacasa/umbra)\n[](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.elberacasa/umbra)\n[](./LICENSE)\n[](https://github.com/elberacasa/umbra/actions/workflows/ci.yml)\n[](./RUBRIC.md)\n[](https://www.npmjs.com/package/@elberacasa/umbra)\n\n[Quickstart](#quickstart) · [Demo](#demo) · [The Audit](#the-audit-61-vibe-coded-repos-scanned) · [How it works](#how-it-works) · [The Four Axes](#the-four-axes) · [FAQ](#faq) · [Roadmap](#roadmap) · [Contributing](#contributing) · [Website](https://umbra-badge.umbrabadge.workers.dev)\n\n
\n\n\n\n\n\n\n\n## Why Umbra exists\n\nStudies put exploitable vulnerabilities in 40 to 60 percent of AI-generated\ncode, and coding agents routinely claim \"all tests pass\" when three do. The\ntooling for *writing* code with AI is a year ahead of the tooling for\n*trusting* it. Umbra closes that gap: SAST rebuilt for how software gets\nwritten now, plus sandboxed verification that catches what static rules\ncannot.\n\nOne command scans any repo an agent produced (Claude Code, Cursor, Copilot,\nWindsurf, Lovable) and returns a score with file:line evidence for every\nfinding. With `--deep` it goes further: Umbra builds and boots the repo in a\nlocked-down Docker sandbox, then replays the agent's own claims against\nreality. If the agent is lying about tests, the score is capped below\npassing, with receipts.\n\n## The audit: 61 vibe-coded repos, scanned\n\nWe ran Umbra over 61 public, actively-maintained AI-built repos and\npublished everything. [The Vibe-Coding Security Audit](./docs/vibe-coding-audit-2026-08.md):\n\n| Finding | Repos hit |\n|---|---:|\n| Hardcoded-secret findings (committed `.env`, service keys in source) | **25%** |\n| API routes with no auth check | **26%** |\n| Injection sinks (SQL interpolation, unsafe HTML injection) | **49%** |\n| Entire databases / SQL dumps committed to git | **13%** |\n| At least one critical finding | **10%** |\n| Zero scored findings (genuinely clean) | 7 of 61 |\n\nMean trust score: **74/100**. One in five repos fails outright. The full\nreport has per-class deep dives with representative snippets and fixes, the\ncomplete per-repo table, and an honest methodology section — including the\nfalse positives we found in our own rules while running it, and fixed\n(rubric v4).\n\n## Quickstart\n\n```bash\nnpx umbra-scan # check — scans the directory you're standing in\nnpx umbra-scan --fix # heal — applies provably-safe fixes, shows the score climbing\nnpx umbra-scan --setup # protect — pre-commit gate, PR checks, agent guardrails\n```\n\nThat's the whole interface. Three verbs: check, heal, protect.\n\n**Using an AI coding agent?** Umbra is built to be driven by agents, not\njust run by humans:\n\n- **Any agent** — it reads this repo's [AGENTS.md](./AGENTS.md) / [llms.txt](./llms.txt) and knows what to do. Or tell yours: \"check this repo with umbra.\"\n- **Claude Code / Kimi Code** — `--setup` installs PreToolUse hooks so every file the agent writes is guarded before it lands.\n- **Claude Code, Cursor, Copilot, Windsurf** — the [trust-review skill](./skills/README.md) makes the agent scan its own work before declaring done.\n- **MCP-native agents** — add `umbra-mcp` (`npx --yes -p @elberacasa/umbra umbra-mcp`) and the agent gets `scan_repo`, `guard_content`, and `get_score` as tools.\n\nReal output, scanning a typical vibe-coded Next.js app\n([fixtures/bad-app](./fixtures/bad-app) in this repo, Trust Score **30/100**):\n\n```\n$ npx umbra-scan ./fixtures/bad-app\n\nUMBRA TRUST SCORE: 30/100 🔴\n\nSAFE 🔴 5/100 — 14 findings\nCLEAN ✅ 87/100 — 10 findings\nRUNS — not measured — run with --deep\nHONEST — not measured — run with --deep\n\nScore computed over measured axes only (full rubric: SAFE 35%, RUNS 25%, HONEST 25%, CLEAN 15%). Rubric v4.\n…plus 7 further findings beyond the per-rule cap (see report)\n\nTop findings:\n [safe/hardcoded-secrets] Hardcoded Supabase service_role JWT — bypasses all row level security — .env:2\n [safe/hardcoded-secrets] Hardcoded Supabase service_role JWT — bypasses all row level security — lib/supabase.ts:5\n [safe/supabase-antipatterns] Supabase service_role key reachable from client-side code — full database bypass for anyone who opens the bundle — .env:2\n [safe/supabase-antipatterns] Supabase service_role key reachable from client-side code — full database bypass for anyone who opens the bundle — app/components/UserList.tsx:10\n [safe/hardcoded-secrets] Committed environment file with secret values: .env — .env:1\n\nNotes (low confidence — not scored):\n [safe/missing-rate-limit] Auth endpoint with no rate-limiting signal in the repo — brute-force / credential-stuffing exposure (heuristic) — app/api/login/route.ts:3\n\nBadge: [](https://github.com/elberacasa/umbra)\n```\n\nThe exit code is **1** when the score is below 50, so CI can gate on it.\n\n\nAll commands and flags (the expert layer — most users never need these)\n\n```bash\numbra [path] # path defaults to the current directory\numbra [path] --json # machine-readable output\numbra [path] --offline # skip npm registry checks, fully local\numbra [path] --deep # verify RUNS and HONEST in a Docker sandbox\numbra [path] --report # write UMBRA.md: an agent-actionable task list\numbra [path] --fix # apply provably-safe fixes and re-scan (score before → after)\numbra [path] --dry-run # preview --fix without writing anything\numbra [path] --baseline-write # write .umbra-baseline.json: grandfather current findings, gate only on new ones\numbra [path] --baseline # use an explicit baseline file (\"write\" is shorthand for --baseline-write)\numbra [path] --publish # self-report the score to the hosted badge service (live README badge)\numbra setup # install everything (hooks + Action + agent guards)\numbra init # only the pre-commit hook + GitHub Action\numbra protect # only the agent PreToolUse hooks (--remove uninstalls)\numbra guard --stdin # hook entrypoint (agents call this, not humans)\numbra mcp # run the MCP server (bin: umbra-mcp)\n```\n\nThe canonical package is `@elberacasa/umbra`; `umbra-scan` is the short\nalias. Same engine either way.\n\n\n\n## How it works\n\n```\nrepo in\n │\n ▼ Layer 0 · static rules (17 SAFE + CLEAN rules, 0 tokens, <1s)\n ▼ Layer 1 · evidence gating (confidence-scored, low never moves the score)\n ▼ Layer 2 · --deep sandbox (Docker: build, boot, HTTP probe, claim replay)\n │\n ▼ deterministic Trust Score + verdict + badge\n```\n\nEvery finding carries a confidence level and file:line evidence. Only high\nand medium confidence findings move the score; hunches go to a notes section.\nThe rubric is versioned (currently v3), so the same repo always gets the same\nscore. Full math in [RUBRIC.md](./RUBRIC.md).\n\n## The immune layer: guard the write, not just the repo\n\nScanning finds problems after they land. The immune layer checks every file\nyour agent writes **before** it lands. `umbra protect` installs PreToolUse\nhooks into Claude Code and Kimi Code (auto-detected, one command); the same\nengine backs the `umbra-mcp` server for MCP-native agents.\n\n\n\n```mermaid\nflowchart LR\n CC[Claude Code hook] --> E\n KC[Kimi Code hook] --> E\n MCP[\"umbra-mcp: guard_content\"] --> E\n E{\"guardContent(file, content) file rules + path guard\"} -->|allow / warn| W[write lands]\n E -->|\"block (exit 2)\"| B[\"reason fed back: agent fixes the root cause\"]\n```\n\n```bash\nnpx umbra-scan protect # install the hooks; --remove uninstalls cleanly\n```\n\nA leaked Stripe key or an `alg: none` JWT never reaches the file. The path\nguard hard-blocks agent writes into `.git/hooks` and `.git/config`\n([CVE-2026-26268](https://anomity.ai/blog/cursor-git-hooks-sandbox-escape-rce-cve-2026-26268/),\nthe agent-planted git hook escape), and live credentials going into `.env`.\nBlocking is reserved for high-confidence critical/high findings; everything\nelse warns, and every failure fails open. Verdicts land in ~0.2 ms, so the\nguard never slows the agent down. Full story:\n[docs/immune-layer.md](./docs/immune-layer.md).\n\n## `--deep`: verify AI code, don't trust it\n\nThe fast scan is static. `--deep` is LLM code verification with evidence.\nUmbra copies the repo into a throwaway Docker container (no network at\nruntime, 512 MB / 1 CPU hard limits, 120-second kill switch), builds it,\nboots it, HTTP-probes its endpoints, and replays every claim found in\nREADMEs and agent artifacts against what actually happens. Slower (minutes,\nnot seconds) and needs a running Docker daemon. Without Docker the sandboxed\naxes are skipped and left out of the score; unverifiable is never punished.\n\nReal output, deep-scanning a repo whose README lies\n([fixtures/claims-app](./fixtures/claims-app), capped at **49/100** by the\nliar cap):\n\n```\n$ npx @elberacasa/umbra ./fixtures/claims-app --deep\n\nUMBRA TRUST SCORE: 49/100 🔴\n\nSAFE ✅ 100/100 — 0 findings\nCLEAN ✅ 100/100 — 2 findings\nRUNS — not measured — No detectable run path (no Dockerfile, no package.json start script or main entry)\nHONEST ⚠️ 50/100 — 2 claims failed, 2 verified, 1 unverifiable\n\nScore computed over measured axes only (full rubric: SAFE 35%, RUNS 25%, HONEST 25%, CLEAN 15%). Rubric v4.\nScore capped below passing: a documented claim was verified false. Trust is the product.\n\nClaim receipts:\n CLAIM FAILED: \"14 tests pass\" — README.md:7 — actually 3 tests pass, 0 fail\n CLAIM FAILED: \"build passes\" — README.md:9 — actually build exits 1\n CLAIM VERIFIED: \"All tests pass\" — CLAUDE.md:3 — 3 tests pass\n CLAIM VERIFIED: \"All tests are passing\" — README.md:8 — 3 tests pass\n```\n\nAny claim verified false caps the total at 49: a repo caught lying does not\nget a passing trust score. For contrast, a genuinely working app\n([fixtures/runnable-app](./fixtures/runnable-app)) scores **100/100** under\n`--deep`.\n\n## The Four Axes\n\n| Axis | Question | How it's measured |\n|------|----------|-------------------|\n| **SAFE** (35%) | Is it vulnerable? | 13 deterministic static rules, every scan, fully offline. |\n| **RUNS** (25%) | Does it actually build and boot? | Docker sandbox: install, build, start, HTTP probe. *(`--deep`)* |\n| **HONEST** (25%) | Is the agent lying about tests or the build? | Claims extracted from READMEs and agent files, replayed against sandbox reality, receipts emitted. *(`--deep`)* |\n| **CLEAN** (15%) | How much is slop? | Static rules: dead exports, unused deps, mega-files, duplication. |\n\nThe SAFE rules cover the failures AI-generated code security actually ships:\nhardcoded secrets (Stripe keys, JWTs, connection strings), Supabase\nservice-role keys exposed client-side and missing **Supabase RLS**, missing\nauth on API routes, injection sinks, rate-limit hints, hallucinated and\ntyposquatted dependencies, CORS wildcard with credentials, JWT misconfig\n(`alg: none`, no expiry, decode-as-authorization), debug flags and\nstack-trace leaks, committed sensitive files (`.pem`, `id_rsa`, SQL dumps),\nand default credentials.\n\nIt also lints the agent's own setup — the surface nobody else covers:\nprompt-injection payloads in instruction files (`CLAUDE.md`, `.cursor/rules`,\nskills: zero-width Unicode, override phrases in HTML comments) and dangerous\nMCP configs (literal API keys in `.mcp.json`, unpinned `npx -y` servers,\n`curl | sh` installers). These run in the guard too, so an agent editing its\nown config gets checked mid-write.\n\n## Umbra vs. existing tools\n\n| | Umbra | Traditional SAST (Semgrep, Snyk Code) | Secret scanners (trufflehog, Gitleaks) | Agent review bots |\n|---|---|---|---|---|\n| Built for AI-generated code | ✅ | generic rulesets | secrets only | ✅ |\n| Verifies the app builds, boots, and answers HTTP | ✅ (sandbox) | — | — | — |\n| Replays agent claims, caps liars below passing | ✅ | — | — | — |\n| Deterministic score, versioned rubric | ✅ | findings list | findings list | prose review |\n| Agent-native surfaces (skill, Action, MCP) | ✅ | — | — | partial |\n\nExisting tools answer \"is this code pattern dangerous?\" Umbra answers the\nquestion vibe coding actually raises: \"the AI wrote this, can I trust it?\"\n\n## The badge\n\nEvery scan prints badge markdown. Paste it in your README and your repo\nadvertises its own trust score:\n\n```markdown\n[](https://github.com/elberacasa/umbra)\n```\n\n[](https://github.com/elberacasa/umbra)\n\n**Live badges** are one flag away: run with `--publish` (or the Action's\n`publish: true`) and your score reports to the hosted badge service, so your\nREADME always shows the current number with a full report page behind the\nclick — self-reported by your CI, labeled as such:\n\n```markdown\n[](https://umbra-badge.umbrabadge.workers.dev/OWNER/REPO)\n```\n\n## One engine, every surface\n\n- **CLI** (`npx @elberacasa/umbra`): the core, available today. Short alias:\n `npx umbra-scan`.\n- **Agent skill**: a [trust-review skill](./skills/README.md) installable\n into Claude Code, Cursor, Copilot, and Windsurf, so the agent checks its\n own work before you do. Claude Code / Cursor / Copilot security, from\n inside the agent.\n- **GitHub Action**: [`uses: elberacasa/umbra@v1`](./action.yml) comments the\n Trust Score on every PR. Trust gating in CI, zero local setup.\n- **`umbra setup`**: the one-word installer — pre-commit gate, PR score\n comments, and PreToolUse guard hooks for detected agents, all idempotent\n and clobber-free. (`init` and `protect` remain for piecemeal installs.)\n- **`umbra protect`**: installs PreToolUse hooks into Claude Code and Kimi\n Code (auto-detected, idempotent, `--remove` to uninstall) so Umbra reviews\n every agent write mid-stream and blocks dangerous ones before they land.\n- **MCP server** (`umbra-mcp`): agents call Umbra mid-stream and catch their\n own mistakes before the code lands. Add it with\n `npx --yes -p @elberacasa/umbra umbra-mcp`.\n\nDay-to-day recipes (CI gating, JSON parsing, hooks): [docs/daily-use.md](./docs/daily-use.md).\n\n## Roadmap\n\n- **v0.1** *(shipped)*: CLI, SAFE + CLEAN static axes, deterministic score, verdict output, badge markdown.\n- **v0.2** *(shipped)*: the surfaces. Agent skill, GitHub Action, `umbra init`.\n- **v0.3** *(shipped, current)*: RUNS axis (sandbox build, boot, HTTP probe) and HONEST axis (claim receipts plus the liar cap).\n- **v1.0** *(shipped)*: the immune layer. Umbra sits between the agent and your codebase, intercepting writes mid-stream and scoring them before they land. Full story in [docs/immune-layer.md](./docs/immune-layer.md).\n- **Beyond**: attack graphs across your dependency tree, a security twin of your app that gets probed so production doesn't, hosted report permalinks behind every badge.\n\nThe wedge is a score. The destination is the verification layer every\nAI-built repo runs through.\n\n## FAQ\n\n**How do I adopt Umbra in a repo that already has findings?**\nRun `npx umbra-scan --baseline-write` once. Umbra writes `.umbra-baseline.json`\ninto the repo root, and from then on the gate only blocks **new** issues —\nexisting findings are grandfathered (the verdict shows\n`baseline: N existing findings grandfathered (M new)`), so you fix forward\ninstead of boiling the ocean. Commit the baseline file so the whole team and\nCI share it.\n\n**How is Umbra different from Semgrep, Snyk, or trufflehog?**\nThey scan code patterns; Umbra verifies outcomes. Static rules are one input\nto the SAFE axis. Umbra additionally boots the app in a sandbox to prove it\nruns, and replays the agent's documented claims to prove it isn't lying.\n\"README says 14 tests pass, actually 3 do\" costs the repo a passing grade.\n\n**Does Umbra send my code anywhere?**\nNo. Scanning is fully local; `--offline` skips even the npm registry checks.\n`--deep` runs your repo in a local Docker container with no network at\nruntime. Nothing leaves your machine.\n\n**Does it need Docker?**\nOnly for `--deep` (RUNS and HONEST). The default fast scan is pure static\nanalysis. Without Docker the sandboxed axes are skipped and excluded from the\nscore, never punished.\n\n**What languages does it support?**\nJavaScript and TypeScript (including Next.js and Supabase apps) have the\ndeepest coverage today, which is where most vibe-coded repos live. The rule\nengine is extensible; new rules need a fixture and a test.\n\n**Is the score reproducible?**\nYes. Same repo, same rubric version, same score, every time. The rubric is\nversioned (v2) and printed in every report, and low-confidence findings never\naffect it. Skipped axes are excluded and renormalized over, never punished.\n\n**What does it catch that my AI agent won't mention?**\nThe classics of AI-generated code: a Supabase `service_role` JWT shipped to\nthe browser (bypasses all row level security), live Stripe keys in `.env`,\nAPI routes with no auth check, `alg: none` JWTs, CORS `*` with credentials,\nhallucinated dependencies that don't exist on npm, and whether its own claims\nabout tests and builds are true.\n\n**Can Umbra stop my agent mid-write?**\nYes, via hooks. Run `npx @elberacasa/umbra protect` and Umbra installs a\nPreToolUse hook into Claude Code and/or Kimi Code that reviews every\n`Write`/`Edit`/`MultiEdit` before it lands. Only high-confidence critical and\nhigh severity findings block (a wrong block gets tools uninstalled, so when\nin doubt Umbra warns), the `.git/hooks` path guard blocks git-hook planting\n(CVE-2026-26268) outright, and the guard fails open on its own errors so it\nnever breaks your flow. Hooks are a guardrail, not a sandbox; details in\n[docs/immune-layer.md](./docs/immune-layer.md).\n\n**Can my AI coding agent use Umbra directly?**\nYes, that is the design. The repo ships an [AGENTS.md](./AGENTS.md) and\n[llms.txt](./llms.txt) so assistants know exactly when and how to run it, and\nthe [agent skill](./skills/README.md) makes Claude Code, Cursor, Copilot, and\nWindsurf scan their own work before declaring a task done.\n\n## Contributing\n\nIssues and PRs welcome. See [CONTRIBUTING.md](./CONTRIBUTING.md). The\nhighest-value contributions right now: new SAFE/CLEAN rules with fixtures and\ntests, false-positive reports (severity-one bugs here), renders against real\nAI-generated repos, and new harness adapters for `umbra protect`.\n\nBuild and test before submitting:\n\n```bash\nnpm install\nnpm run build\nnpm test\n```\n\n## Ethical use\n\nUmbra is a defensive tool. Scan repos you own, repos you are about to depend\non, or repos you have permission to audit. Findings point at weaknesses; they\nare not exploits, and publishing someone else's low score to shame them is\nnot the point. The point is that \"the AI wrote it\" stops being the end of the\nverification conversation.\n\n## License\n\n[MIT](./LICENSE)\n",
"bytes": 21179,
"sha": "fade5138ddeea52ae955723f1dd3d91a51574c2a00cc3449c0a489cfc6ddfd23",
"repo_slug": "elberacasa/umbra",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_elberacasa_umbra_283a79c9/readme"
}
![\"Umbra:](\"assets/logo-light.svg\"){kind=logo}
\n