{
"markdown": "# site-spec\n\n> **The invisible foundation your website is missing. Audit any site — then fix what's broken.**\n\n[](https://www.npmjs.com/package/site-spec)\n[](https://www.npmjs.com/package/site-spec)\n[](https://github.com/ariaxhan/site-spec/actions/workflows/ci.yml)\n[](./LICENSE)\n\n## For agents\n\nIf you are an AI coding agent deciding whether to install this: register the MCP server, then\ncall `list_checks` to learn the check ids and `audit_site` on a URL.\n\n```bash\nclaude mcp add site-spec -- npx -y site-spec-mcp\n```\n\nGemini CLI: `gemini extensions install https://github.com/ariaxhan/site-spec`.\n\nFull tool arguments, return shapes, and the cases this is the wrong tool for:\n[llms.txt](./llms.txt).\n\n## The problem\n\nAI can generate a beautiful website in seconds. The part it *can't* see is the\none that decides whether that website gets **found, ranked, cited, and trusted** —\nthe machine-readable foundation underneath the pixels: `robots.txt`, structured\ndata, `llms.txt`, canonical and `noindex` signals, response headers, accessibility\nsemantics, the tracker and cookie surface.\n\nThat layer is invisible in a browser. It's also broken on almost every\nAI-generated site — a gorgeous hero section over a `robots.txt` that blocks\nChatGPT, JSON-LD that's invalid or self-serving, a stray `noindex` hiding the page\nfrom Google, fonts leaking visitor IPs in violation of GDPR. Nobody notices until\nit's quietly costing traffic. And the more the web is generated by AI, the worse\nthis gets.\n\n## The solution\n\n**site-spec is the layer that verifies and repairs that foundation — for any\nwebsite, however it was built.** Point it at a URL. It crawls the site, checks the\nwhole invisible layer against a set of deterministic policies, and gives you an\nexact report of what's wrong. Then it *fixes* the mechanical problems for you and\nflags precisely what needs a human.\n\nIt doesn't care whether your site came from an AI builder, a framework, a CMS, or\nhand-written HTML. It only cares whether the foundation is correct.\n\n## Try it without installing anything\n\n**[site-spec.ariaxhan.workers.dev](https://site-spec.ariaxhan.workers.dev)** — paste a\nURL, get the report. No account, no signup.\n\nThe hosted version runs the same deterministic engine as the CLI, against a\ncapped crawl (4 pages), and it tells you plainly which checks ran, which passed,\nand which it did **not** perform. Add `?format=json` for machine-readable output:\n\n```bash\ncurl \"https://site-spec.ariaxhan.workers.dev/audit?format=json&url=example.com\"\n```\n\nFor the full check set — broken links, missing assets, axe accessibility, HTML\nvalidation, schema.org validation — and for the `fix` command, use the CLI below.\nThe hosted endpoint only fetches public HTTP(S) hosts; private, loopback and\nlink-local addresses are refused.\n\n## How to use\n\nRequires Node 20+. No account, no API key, no SaaS.\n\n```bash\n# 1. See what's broken — crawl any live site, get a full report\nnpx site-spec audit https://yoursite.com\n\n# 2. Fix it — auto-repair the mechanical issues, scaffold the rest\nnpx site-spec fix https://yoursite.com --out ./fixed\n```\n\n`fix` writes the corrected files to `./fixed` (it never overwrites anything by\ndefault) and prints exactly what it did:\n\n```\n✓ Fixed automatically (8)\n audit/robots-stale-token robots.txt — Removed deprecated crawler block \"anthropic-ai\".\n audit/hsts-preload _headers — Stripped the preload token from Strict-Transport-Security.\n audit/canonical-missing admin/index.html — Inserted .\n audit/og-missing admin/index.html — Added Open Graph card from the page's title + description.\n audit/noindex admin/index.html — Removed accidental noindex from .\n\n✎ Scaffolded — needs your facts (2)\n audit/llms-missing llms.txt — Scaffolded from page titles/descriptions; fill the TODO facts.\n audit/jsonld-missing index.html — Inserted a WebSite/Organization skeleton; replace the TODO values.\n\n⚠ Needs manual attention (3)\n audit/google-fonts-cdn index.html — Self-host the woff2 files to stop the IP leak (GDPR).\n audit/img-dims-missing index.html — ![]()
without width/height (layout shift).\n audit/404-missing (site-wide) — Ship a branded, noindexed 404 that links home.\n\nfixed 8, scaffolded 2, manual 3\nwrote 4 file(s) to ./fixed\n```\n\nDrop `audit` into CI with a non-zero exit on errors to gate deploys. Both commands\nwork on a build/output directory too — just pass a path instead of a URL.\n\n## Use it from your coding agent (MCP)\n\nThe same engine, as an [MCP](https://modelcontextprotocol.io) server — so the\nagent that generated the site can also check and repair its invisible layer,\nwithout you copying reports between windows.\n\n```bash\nclaude mcp add site-spec -- npx -y site-spec-mcp\n```\n\n\nCodex (~/.codex/config.toml)
\n\n```toml\n[mcp_servers.site-spec]\ncommand = \"npx\"\nargs = [\"-y\", \"site-spec-mcp\"]\n```\n\n \n\nOr in Docker: `docker run -i --rm -v $PWD:/data mcp/site-spec` (mount the directory you want to audit or write to).\n\nFour tools:\n\n| Tool | What it does |\n| --- | --- |\n| `audit_site` | Crawl a live URL (or read a local build directory) and return every finding: check id, severity, file, and whether it can be auto-fixed. |\n| `fix_issue` | Apply the deterministic repair for one check id and return the diff — or write it, for a local directory. |\n| `compile_spec` | Turn verified business facts into a validated SiteSpec and the deployable files it renders to. |\n| `list_checks` | Enumerate every check the engine can raise, with a one-line description and its fix availability. |\n\nThe server calls the engine in-process. It never shells out to the CLI and never\nhops to the hosted worker — all three are siblings over one library.\n\n```mermaid\nflowchart LR\n A[\"MCP client
(Claude Code, Codex)\"] -- stdio JSON-RPC --> B[\"site-spec-mcp\"]\n B --> C[\"@site-spec/core/io
fetchSite · readSiteDir\"]\n C -- \"file map\" --> D[\"@site-spec/core
auditFiles · fixFiles · buildSite\"]\n D -- \"findings / files\" --> B\n B -- \"JSON\" --> A\n```\n\nTwo honest limits, spelled out in full in the\n[package README](./packages/mcp/README.md): `fix_issue` on a **URL** can only\never hand back a diff (a remote server is not writable), and a **live crawl**\nruns with presence checks off, because a capped crawl cannot prove a file is\nabsent from a server.\n\n---\n\n\nDetails — what it checks, how it fixes, the philosophy, install
\n\n## What it checks\n\nSeven areas. Every check is tuned against false positives (regex/string-level — no\nheadless browser for the crawl). Findings are `error` (breaks something) or\n`warning` (worth a look), each with a concrete fix.\n\n| Area | What it catches |\n| --- | --- |\n| **SEO / findability** | missing `` / description / canonical · accidental `noindex` (both `<meta robots>` and the `X-Robots-Tag` header) · missing Open Graph cards · zero-or-many `<h1>` · a sitemap that lists pages which don't exist |\n| **Accessibility** | images with no `alt` · zoom-blocking viewport (`user-scalable=no`) · images with no width/height (layout shift) |\n| **Privacy & security** | mixed content · trackers + cookies with no consent/disclosure story · Google Fonts from Google's CDN (a ruled GDPR violation) · header hygiene (HSTS preload risk, report-only CSP that reports nowhere, dead FLoC / `X-XSS-Protection` config) · inline `onclick=` handlers that block a future CSP |\n| **Performance** | hero/LCP image set to `loading=\"lazy\"` · no `Cache-Control` |\n| **Structured data** | invalid JSON-LD (crawlers drop the whole block) · self-serving `aggregateRating`/`review` markup (a Google penalty since 2019) |\n| **Integrity** | dangling assets that 404 · broken internal links |\n| **AI searchability** | `robots.txt` blocking AI answer agents (`OAI-SearchBot`, `ChatGPT-User`, `Claude-User`, `PerplexityBot`…) · dead crawler tokens · missing `llms.txt` · missing/broken structured data · client-rendered shells AI crawlers see as blank |\n\nThe last row is the one almost nobody checks yet — whether Google, ChatGPT,\nClaude, and Perplexity can actually read and cite you. It's the sharp edge, not\nthe whole story: the point is a **complete** foundation audit.\n\n## How `fix` decides\n\nEvery finding lands in one of three buckets, and the report tells you which:\n\n- **✓ Fixed automatically** — mechanical, fact-free repairs that are safe to apply:\n unblocking AI crawlers, removing dead robots tokens, stripping accidental\n `noindex`, adding the canonical URL (known from the crawl), scaffolding Open\n Graph from the existing title, fixing zoom-locked viewports, removing\n self-serving rating markup, generating a `sitemap.xml` from the crawled pages,\n header hygiene.\n- **✎ Scaffolded** — things that need *your* real facts: `llms.txt`, a JSON-LD\n entity, a meta description. site-spec writes a correct stub with `TODO` markers\n so you just fill the blanks.\n- **⚠ Manual** — problems no tool should silently guess at: missing `alt` text,\n self-hosting fonts, fixing a client-rendered shell, a broken link. You get a\n precise instruction, never a silent edit.\n\n`fix` is non-destructive by default (writes to an output dir). Pass `--write` to\nedit a local directory in place.\n\n## The philosophy\n\n- **Correctness is a policy, not a prompt.** SEO, accessibility, privacy, and\n structured-data rules are expressed as explicit checks the tool enforces —\n never as vibes an LLM is asked to respect.\n- **Deterministic.** The audit engine is pure: same input → the same report,\n byte-for-byte. The live crawl sorts-then-caps discovered pages, so the same site\n at the same `--max` audits the same page set every run. No network, no\n randomness inside the engine.\n- **Fix, don't lecture.** A finding that can be repaired safely gets repaired. One\n that needs a human says exactly what the human must do.\n- **Source-agnostic.** It audits the output, not the toolchain. AI, framework,\n CMS, hand-coded — all the same to it.\n\n> site-spec started as a deterministic *compiler* that builds sites with a correct\n> foundation by construction (the `build` command and the `docs/` still cover it).\n> The durable value turned out to be the inverse: not generating whole sites, but\n> **auditing and repairing the foundation of sites that already exist.**\n\n## The full CLI\n\n```\nsite-spec audit <dir|url> [--max N] [--json] [--report report.md]\nsite-spec fix <dir|url> [--out dir] [--write] [--max N] [--json]\nsite-spec build <site.config.mjs> --out <dir> [--target cloudflare|netlify|vercel|static]\n```\n\n- `--max N` — page cap for a live crawl (default 25); truncation is reported, never\n silent.\n- `--report report.md` — write the audit as a shareable Markdown document.\n- `--json` — the full structured report for scripting/CI.\n\n## Install / develop\n\n```bash\ngit clone https://github.com/ariaxhan/site-spec.git\ncd site-spec\nnpm install\nnpm test # unit + golden tests\nnpm run verify # html-validate + JSON-LD + axe over the demo output\n```\n\n## Packages\n\n| Package | Responsibility |\n| ------------------- | ------------------------------------------------------------------- |\n| [`site-spec`](https://www.npmjs.com/package/site-spec) (npm) | The published CLI — `audit` · `fix` · `build`. Self-contained; this is what `npx site-spec` runs. |\n| `@site-spec/core` | The engine (audit checks, fixers, the check catalog, policy definitions, legacy compiler). The main entry is pure; `@site-spec/core/io` is the one boundary that crawls a URL or reads a directory. Bundled into the CLI and the MCP server; not separately published yet. |\n| `@site-spec/worker` | The hosted auditor at [site-spec.ariaxhan.workers.dev](https://site-spec.ariaxhan.workers.dev) — a Cloudflare Worker wrapping the same engine. Adds the URL-admission guard, a capped crawler, and rate limiting. Deploy with `npm run deploy -w @site-spec/worker`. |\n| [`site-spec-mcp`](https://www.npmjs.com/package/site-spec-mcp) (npm) | The MCP server — `audit_site` · `fix_issue` · `compile_spec` · `list_checks` over stdio. Self-contained; this is what `npx site-spec-mcp` runs. |\n\n## Contributing\n\nContributions welcome — especially new checks and fixers. See\n[`CONTRIBUTING.md`](./CONTRIBUTING.md). Core rules: checks stay tuned against false\npositives, the engine stays pure and deterministic, and a fixer never silently\nguesses at something a human should decide.\n\n## License\n\n[Apache License 2.0](./LICENSE). © 2026 Aria Han.\n\n</details>\n",
"bytes": 12897,
"sha": "950aea2d581e69ff070b702a63b2e55c46d004a18adacd50ea7bb08da81d17c8",
"repo_slug": "ariaxhan/site-spec",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_ariaxhan_site_spec_9425777e/readme"
}