{
  "markdown": "# nlpm\n\n[![Validated by NLPM](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/xiaolai/nlpm/main/nlpm-badge.json)](https://github.com/xiaolai/nlpm/blob/main/nlpm-badge.json)\n\nNatural-Language Programming Manager — score, check, fix, and test NL artifacts across **Claude Code, Codex CLI, and Antigravity**. Tier-aware scoring with per-tool overlays.\n\nPart of the [xiaolai plugin marketplace](https://github.com/xiaolai/claude-plugin-marketplace).\n\nNLPM is the only multi-tool NL artifact validator that systematically checks **manifest-vs-disk consistency** — the bug class where a SKILL.md exists on disk but is silently missing from `plugin.json` (and therefore invisible after `claude plugin install`). Verified across 8+ tools including Anthropic's official `plugin-validator` and the Linux Foundation's `skills-ref`. See [`analysis/ecosystem-gap.md`](analysis/ecosystem-gap.md) for the research.\n\n## What it does\n\nNLPM treats natural language artifacts as **programs that can be linted**. Just as ESLint scores JavaScript and ruff scores Python, NLPM scores the markdown files that drive AI behavior: skills, agents, commands, rules, hooks, prompts, CLAUDE.md, and memory files.\n\nEight commands, each doing one thing:\n\n| Command | What it does |\n|---------|-------------|\n| `/nlpm:ls` | Discover and inventory all NL artifacts in a repo |\n| `/nlpm:score` | Score artifact quality (100-point scale) |\n| `/nlpm:check` | Cross-component consistency checks |\n| `/nlpm:fix` | Auto-fix fixable issues |\n| `/nlpm:trend` | Track quality score trends over time |\n| `/nlpm:test` | Run NL artifact tests against spec files (TDD) |\n| `/nlpm:init` | Initialize NLPM for a project |\n| `/nlpm:security-scan` | Scan plugins for security risks in executable artifacts |\n\nSlash commands ship as a Claude Code plugin. The scoring rubric covers three ecosystems (Claude Code, Codex CLI, Antigravity) via tier-aware overlays — see [`analysis/multi-tool-design-2026-05.md`](analysis/multi-tool-design-2026-05.md). The standalone Python 3.11+ validator (`bin/nlpm-check`) has no Claude Code dependency and runs in pre-commit hooks or CI on any tool's artifacts.\n\n### Beyond linting: the learning loop\n\nNLPM also runs as a self-evolving GitHub Actions pipeline that audits real plugin repos, contributes fix PRs, harvests teaching examples from clean ones, and feeds learnings back into its own rule catalog:\n\n- **Exemplar pipeline** (v0.8.17+): repos that audit clean at score ≥ 90 produce a teaching artifact under `auditor/exemplars/` — 62 published so far, covering 31 of the 50 Rules with real-world positive references. See [the gallery](auditor/exemplars/README.md).\n- **Rule-citation auto-PR** (v0.8.18+): `auditor-cite-exemplars.yml` runs weekly and opens a human-gated PR adding `> Real-world example: [<repo>]` links to `skills/nlpm/rules/SKILL.md`, so each rule documents both the bad case (in the rule body) and the good case (in a real repo).\n- **Two-stage drift detector** (v0.8.15–v0.8.16): `auditor/scripts/validate-rule-ids.py` re-validates every audit's `rule_id` against the rubric (type drift) and the rule's title keywords (semantic drift). The 2026-05-13 sweep found 990 mislabeled `rule_id`s across 128 historical audits; the validator is now wired as a soft-warn telemetry step in every new audit so future drift is caught immediately.\n- **Drift-filtered rule health**: `auditor/scripts/rule-health.py` reports `validated_hits` per rule (raw hits minus drift hits) and `exemplars_count` per rule, so the \"needs attention\" view is calibrated against actual rule violations rather than scorer noise.\n\n## Installation\n\nTwo install paths — both reach the same code. Pick one:\n\n**Via Anthropic's official community marketplace** (curated; updates lag the maintainer's marketplace by up to ~24h):\n\n```bash\nclaude plugin marketplace add anthropics/claude-plugins-community\nclaude plugin install nlpm@claude-community --scope project   # or --scope user\n```\n\n**Via the xiaolai marketplace** (latest version lands here first):\n\n```bash\nclaude plugin marketplace add xiaolai/claude-plugin-marketplace\n\n# Project scope (recommended)\nclaude plugin install nlpm@xiaolai --scope project\n\n# Global (all projects)\nclaude plugin install nlpm@xiaolai --scope user\n```\n\n> **Install fails with \"Plugin not found in marketplace 'xiaolai'\"?** Your local marketplace clone is stale. Run `claude plugin marketplace update xiaolai` and retry — `plugin install` does not auto-refresh. (The community marketplace doesn't have this caveat.)\n\n## Quick Start\n\nIn Claude Code:\n\n```\n/nlpm:ls                    # see what NL artifacts you have\n/nlpm:score                 # score them all\n/nlpm:score agents/         # score just agents\n/nlpm:score --changed       # score only git-changed files\n/nlpm:check                 # check cross-component consistency\n/nlpm:fix                   # auto-fix what's fixable\n/nlpm:trend                 # track score history over time\n/nlpm:test                  # run NL-TDD specs\n```\n\nFrom CI or a pre-commit hook (no Claude Code required):\n\n```bash\ncurl -fsSL -o /usr/local/bin/nlpm-check \\\n  https://raw.githubusercontent.com/xiaolai/nlpm/main/bin/nlpm-check\nchmod +x /usr/local/bin/nlpm-check\nnlpm-check .               # exit 1 on high-confidence findings\n```\n\n## For plugin/skill authors — standalone validator\n\nIf you author a plugin and want NLPM in your **pre-commit hook, CI, or pre-publish gate**, use the standalone binary at [`bin/nlpm-check`](bin/nlpm-check). It's a single Python 3.11+ file with no external dependencies. It runs the deterministic subset of `/nlpm:check` — including the manifest-vs-disk consistency check that no other validator (Anthropic's official `plugin-validator`, Linux Foundation's `skills-ref`, third-party tools) currently covers.\n\n```bash\n# One-line install\ncurl -fsSL -o /usr/local/bin/nlpm-check \\\n  https://raw.githubusercontent.com/xiaolai/nlpm/main/bin/nlpm-check\nchmod +x /usr/local/bin/nlpm-check\n\n# Run in your plugin repo\nnlpm-check .\n```\n\nTemplates ship in [`templates/`](templates/):\n- `pre-commit-nlpm.sh` — drop-in git pre-commit hook\n- `workflows/nlpm-check.yml` — drop-in GitHub Actions workflow\n\nSee [`docs/for-authors.md`](docs/for-authors.md) for the full author guide. See [`analysis/ecosystem-gap.md`](analysis/ecosystem-gap.md) for the research on why this check exists and which other validators do (and don't) cover it.\n\n## Scoring System\n\nScores start at 100 and go down. Every issue has a fixed penalty. The score is deterministic: same artifact, same penalties, same number.\n\n| Score | Band | Meaning |\n|-------|------|---------|\n| 90-100 | Excellent | Production-ready |\n| 80-89 | Good | Minor gaps |\n| 70-79 | Adequate | Meets threshold, should improve |\n| 60-69 | Weak | Below threshold |\n| <60 | Rewrite | Fundamental problems |\n\nDefault pass threshold: 70. Configure in `.claude/nlpm.local.md`.\n\nSee `skills/nlpm/scoring/SKILL.md` for the full penalty tables. See `skills/nlpm/rules/SKILL.md` for the 50 Rules of Natural Language Programming.\n\n## What it scores\n\n20+ artifact types across one universal floor and three per-tool overlays. The scorer auto-classifies each artifact by its path (see `agents/scorer.md` step 3 for the tier classifier) and applies the matching rules.\n\n| Tier | Artifacts |\n|------|-----------|\n| Universal (Tier 1, open spec at agentskills.io) | `SKILL.md`, `AGENTS.md` (canonical universal memory file, per nlpm decision) |\n| Claude Code (Tier 2-Claude) | `commands/`, `shared partials`, `agents/`, `skills/`, `hooks/hooks.json`, `.claude-plugin/plugin.json`, `.claude-plugin/marketplace.json`, `.mcp.json`, `CLAUDE.md`, `.claude/rules/`, `.claude/settings.json`, `.lsp.json`, `monitors/monitors.json`, `~/.claude/projects/*/memory/*.md` |\n| Codex CLI (Tier 2-Codex) | `.agents/skills/<n>/SKILL.md`, `.codex-plugin/plugin.json`, `.agents/plugins/marketplace.json`, `.codex/config.toml` (TOML — `[mcp_servers.*]`, `[hooks.*]`, `[agents]` global settings), `.codex/agents/*.toml` (subagent definitions), `.codex/hooks.json`, `agents/openai.yaml` sidecar, root `AGENTS.md` (hierarchical) |\n| Antigravity (Tier 2-Antigravity, advisory) | `.gemini/skills/`, `.agent/skills/`, `.gemini/commands/<n>.toml`, `.gemini/settings.json` (with embedded `mcpServers` + `hooks`), `gemini-extension.json`, `GEMINI.md` |\n\n## NL-TDD\n\nWrite test specs BEFORE writing artifacts:\n\n```\n1. Write spec:    .nlpm-test/my-agent.spec.md\n2. /nlpm:test     -> RED (artifact doesn't exist)\n3. Write artifact: agents/my-agent.md\n4. /nlpm:test     -> check trigger accuracy, output format, score\n5. /nlpm:score    -> verify quality score\n6. Iterate        -> fix until GREEN\n```\n\nSee `skills/nlpm/testing/SKILL.md` for the full spec format.\n\n## Configuration\n\nCreate `.claude/nlpm.local.md` (or run `/nlpm:init`):\n\n```yaml\n---\nstrictness: standard\nscore_threshold: 70\nrule_overrides:\n  R09: { min_examples: 1 }      # require only 1 example block\n  R05: { threshold: 600 }       # allow skills up to 600 lines\n  R23: { budget: 800 }          # increase rules budget\n---\n```\n\n| Level | Threshold | Effect |\n|-------|-----------|--------|\n| Relaxed | 60 | Only flag seriously broken artifacts |\n| Standard | 70 | Flag artifacts that need improvement |\n| Strict | 80 | Flag anything below good quality |\n\n## Continuous Enforcement\n\nNLPM ships a `PostToolUse` hook that fires when you write or edit files. A shell script (`scripts/check-artifact.sh`) classifies the file -- if it's an NL artifact, Claude reminds you to run `/nlpm:score`. Non-NL files produce no output.\n\nThis is advisory -- it does not block writes. For blocking enforcement, use a `PreToolUse` hook (see tdd-guardian for an example).\n\n## Architecture\n\n```\ncommands/           User-facing commands (8 + 3 shared partials)\n  ls.md             Discover artifacts -> dispatches scanner\n  score.md          Score quality -> dispatches scorer + vague-scanner in parallel\n  check.md          Cross-component checks -> dispatches checker\n  fix.md            Auto-fix issues -> dispatches scorer\n  trend.md          Track score history -> dispatches scorer + vague-scanner\n  test.md           Run NL-TDD specs -> dispatches tester\n  init.md           Configure project\n  security-scan.md  Scan plugins for security risks -> dispatches security-scanner\n  shared/\n    discover.md         Artifact path patterns (not user-invocable)\n    classify.md         Type classification rules (not user-invocable)\n    append-history.md   Persist scoring snapshot to .claude/nlpm-history.json with scope marker (not user-invocable)\n\nagents/             Dispatched by commands (6 agents)\n  scanner.md        haiku -- fast artifact discovery\n  scorer.md         sonnet -- 100-point quality scoring\n  checker.md        sonnet -- cross-component consistency\n  vague-scanner.md  haiku -- mechanical vague-word counting\n  tester.md         sonnet -- evaluates artifacts against test specs\n  security-scanner.md sonnet -- security risk detection in executable artifacts\n\nskills/nlpm/        Knowledge base (17 skills)\n\n  Core (loaded by agents):\n  conventions/              Universal NL floor: SKILL.md open spec, AGENTS.md, vague-quantifier list, naming\n  conventions-claude/       Claude Code overlay: .claude/* paths, plugin.json, hook events, CLAUDE.md, tool catalog\n  conventions-codex/        Codex CLI overlay: .codex/config.toml, .codex-plugin/plugin.json, .agents/skills/, openai.yaml sidecar\n  conventions-antigravity/  Antigravity overlay: .gemini/* + .agent/*, GEMINI.md (advisory)\n  patterns/                 NL programming best practices + anti-patterns\n  scoring/                  Penalty tables with rule number cross-references\n  rules/                    The 50 Rules of Natural Language Programming (R01-R50)\n  vocabulary/               Canonical noun/verb registry for R51 drift detection (opt-in)\n  testing/                  NL-TDD spec format, test patterns\n  security/                 Security pattern database for executable artifact scanning\n\n  Writing Reference (loaded on demand):\n  writing-skills/   How to write SKILL.md files\n  writing-agents/   How to write agent definitions\n  writing-rules/    How to write .claude/rules/ files\n  writing-prompts/  Universal prompt engineering guide\n  writing-hooks/    How to write Claude Code hooks\n  writing-plugins/  How to design and build plugins\n  orchestration/    Multi-agent workflow patterns\n\nhooks/\n  hooks.json        PostToolUse advisory (command type + check-artifact.sh)\n\nscripts/\n  check-artifact.sh NL artifact classifier for the PostToolUse hook\n\n.nlpm-test/         Self-test specs (dogfooding NL-TDD)\n\nbin/                Standalone author surface (v0.8.0+)\n  nlpm-check        Pure-Python validator for pre-commit / CI / pre-publish\n  nlpm-badge        shields.io endpoint generator + optional attestation sidecar\n\ntests/              Python unittest suite (81 tests total)\n  test_nlpm_check.py                       bin/nlpm-check\n  test_nlpm_badge.py                       bin/nlpm-badge\n  test_validate_rule_ids.py                auditor/scripts/validate-rule-ids.py\n  test_exemplar_helpers.py                 batch-process.py + rule-health.py exemplar paths\n  test_exemplar_gallery_and_citations.py   build-exemplar-gallery.py + propose-rule-citations.py\n\ntemplates/          Drop-in author templates\n  pre-commit-nlpm.sh             git pre-commit hook\n  workflows/nlpm-check.yml       GitHub Actions workflow\n\ndocs/\n  for-authors.md    Full guide for plugin/skill authors\n\nanalysis/\n  ecosystem-gap.md                  Why this validator exists (stable ref)\n  scope-expansion-2026-05.md        Author-surface plan\n  2026-05-11-why-obvious-bugs-persist.md   Original research snapshot\n\nauditor/            Self-evolution pipeline (GitHub Actions + data)\n  audits/           Per-repo audit reports and findings sidecars\n  exemplars/        Teaching artifacts from clean audits + auto-generated gallery (v0.8.17+)\n  case-studies/     Narrative articles from post-merge re-audits\n  disclosures-pending/  Security disclosures queued for manual filing\n  feedback/         Rolling rule-health summary\n  findings.jsonl    Append-only audit findings (joined by fingerprint)\n  disagreements.jsonl  self_false_positive + maintainer_rejected + pr_comments_snapshot\n  logs/events.jsonl Lifecycle events + outcome signals + drift telemetry\n  registry/         Repo tracking database\n  scripts/          25+ pipeline helpers — see \"Auditor — Self-Evolution Pipeline\"\n  prompts/          Shared rubric and exemplar-writer prompts\n  reports/          Daily pipeline reports\n```\n\n## Tips\n\n- **Score early, score often.** Run `/nlpm:score` after writing any new artifact.\n- **Use `--changed` for speed.** `score --changed` only scores git-modified files.\n- **Use `/nlpm:trend` before releases.** Catches regressions that individual scoring misses.\n- **Do not chase 100.** 85+ is excellent. The last 5-10 points are diminishing returns.\n- **R01 is the most common penalty.** \"appropriate\", \"relevant\", \"as needed\" each cost -2. Replace with measurable criteria.\n- **Auto-fix handles the mechanical stuff.** Focus your energy on descriptions, examples, and scope notes.\n- **Pre-commit + slash commands together.** Run `nlpm-check` in your pre-commit hook for the deterministic checks; let `/nlpm:score` handle the judgment calls inside Claude Code.\n\n## Troubleshooting\n\n**\"Score seems too low\"** -- Check which penalties hit. Scoring is deterministic. Vague quantifiers stack up fast.\n\n**\"Writing skill didn't load\"** -- Use keywords from the skill's description: \"write an agent definition\", \"create a new agent\".\n\n**\"Check found orphans that aren't really orphans\"** -- Writing skills are on-demand (loaded by Claude, not referenced by agents). This is expected.\n\n**\"Trend shows no history\"** -- Run `/nlpm:score` first to create the baseline snapshot.\n\n## Case Studies\n\n25+ case studies in [`case-studies/`](case-studies/) from the auditor pipeline. A few representative ones:\n\n- [The frontmatter tax: 19 silent registration failures in a 33,000-star plugin collection](case-studies/2026-04-24-wshobson-agents.md) — `wshobson/agents`, 100 artifacts sampled of 509, 5 PRs batched and agentically merged in 13 seconds. (Companion [learnings debrief](case-studies/2026-04-18-wshobson-agents-learnings.md).)\n- [Four bytes of quoting, approved by two OpenAI engineers](case-studies/2026-04-07-openai-codex-plugin-cc.md) — `openai/codex-plugin-cc`, 93/100 Gold tier, two shell-injection fixes merged by OpenAI contributors in 39 hours.\n- [Auditing kubesphere/kubesphere](case-studies/2026-05-07-kubesphere-kubesphere.md) — 16k-star repo, 18 findings including duplicate sections and broken YAML, surfaced by manifest-vs-disk and cross-component checks the other validators don't run.\n- [When the Linter Met Its Match](case-studies/2026-04-06-how-we-helped-gsd.md) — `gsd-build/get-shit-done`, 80 files scored, 5 PRs accepted, plus the false-positive that improved NLPM itself.\n\n## Effectiveness\n\nAs of 2026-05-19 the auditor pipeline has filed 278 PRs across 44 distinct accepting repos, with a 71% acceptance rate (98 merged + 20 applied-separately, 49 rejected, 111 still open). The following data points are the highest-signal:\n\n- **`google-gemini/gemini-skills`** and **`googleworkspace/cli`** — both Google orgs that originally CLA-blocked the pipeline — ended up accepting work: 2 merged and 4 applied-separately respectively, once the CLA gate was satisfied.\n- **`openai/codex-plugin-cc`** has 2 merges — first-party OpenAI org acceptance.\n- **`kubesphere/kubesphere`** (24k+ stars) accepted 5 PRs — the highest-profile downstream.\n- 8 repos (`zubair-trabzada/geo-seo-claude`, `wshobson/agents`, `sickn33/antigravity-awesome-skills`, `kubesphere/kubesphere`, `jeremylongshore/claude-code-plugins-plus-skills`, `Jeffallan/claude-skills`, `hesreallyhim/awesome-claude-code`, `caliber-ai-org/ai-setup`) each hit the per-repo PR cap of 5 — more PRs could ship if the cap were raised.\n- 2 repos have crossed into **rule-adoption** (maintainer credited NLPM in CHANGELOG or systemically backfilled siblings): [`jeremylongshore/claude-code-plugins-plus-skills`](https://github.com/jeremylongshore/claude-code-plugins-plus-skills/blob/main/CHANGELOG.md) and [`sickn33/antigravity-awesome-skills`](https://github.com/sickn33/antigravity-awesome-skills/blob/main/CHANGELOG.md).\n\n## Auditor — Self-Evolution Pipeline\n\nThe `auditor/` directory contains a GitHub Actions pipeline that systematically discovers, audits, and contributes to Claude Code repos across GitHub. Two branches run in parallel: bugs become contribute PRs, clean repos become teaching exemplars. Both branches feed back into NLPM's rules.\n\n```\ndiscover (weekly) → audit\n                      ├─ has bugs ─→ contribute PRs ─→ track merges ─→ write case study\n                      │                                                       ↓\n                      │                                              feedback/log.json\n                      │                                                       ↓\n                      └─ clean (≥90) ─→ write exemplar ─→ gallery ──→ rule-citation PR (weekly, human-gated)\n                                                                              ↓\n                                                                    update NLPM rules → audit better\n```\n\n15 workflows in [`.github/workflows/auditor-*.yml`](.github/workflows/): discover, batch-processor, audit, contribute, track, case-study, **exemplar** (v0.8.17+), **cite-exemplars** (v0.8.18+), classify, daily-report, suppressions, refine-rules, docs-diff, rule-review, integration-test. Human-in-the-loop via issue labels at the audit, contribute, exemplar, and rule-refinement decision points.\n\nSee [auditor/README.md](auditor/README.md) for the full pipeline documentation and [auditor/SCHEMAS.md](auditor/SCHEMAS.md) for the data contracts.\n\n## Prerequisites\n\n- **Slash commands (`/nlpm:*`)**: none. Pure markdown — no Python, no Node.js.\n- **Standalone `bin/nlpm-check`**: Python 3.11+ (stdlib only; no pip install).\n- **Auditor workflows**: `CLAUDE_CODE_OAUTH_TOKEN`, `PAT_TOKEN`, and `OPENAI_API_KEY` GitHub repo secrets.\n\n## License\n\nISC\n",
  "bytes": 20223,
  "sha": "3ad0aa31a79d89183b1bd4579b34bd4c7d218e81096b112acdae0a847157e2f8",
  "repo_slug": "xiaolai/nlpm-for-claude",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_xiaolai_nlpm_for_claude_nlpm_2c1ca560/readme"
}