{
  "markdown": "# skill-optimizer\n\nDocker workbench and Agent Skill for running deterministic evals against agent skills.\n\nUse this repo in two ways:\n\n- Install the `skill-optimizer` skill/plugin into your agent so it can author and debug eval suites.\n- Run the local CLI to execute cases and suites in Docker against OpenRouter models.\n\n## Installation\n\nInstallation differs by agent. The canonical skill is `skills/skill-optimizer/SKILL.md`; every plugin manifest points at that same file.\n\n### Claude Code\n\nRegister this repository as a Claude Code plugin marketplace:\n\n```text\n/plugin marketplace add fastxyz/skill-optimizer\n```\n\nThen install the plugin:\n\n```text\n/plugin install skill-optimizer@skill-optimizer\n```\n\n### OpenAI Codex CLI\n\nRegister this repository as a Codex plugin marketplace:\n\n```bash\ncodex plugin marketplace add fastxyz/skill-optimizer\n```\n\nThen open the plugin search interface:\n\n```text\n/plugins\n```\n\nSelect `skill-optimizer` and install it.\n\n### OpenAI Codex App\n\nIn the Codex app, open Plugins from the sidebar, search for `skill-optimizer`, and install it from the Coding section.\n\nIf it is not listed, install it from Codex CLI first:\n\n```bash\ncodex plugin marketplace add fastxyz/skill-optimizer\n```\n\n### Cursor\n\nInstall the skill with the open skills CLI:\n\n```bash\nnpx skills add fastxyz/skill-optimizer --skill skill-optimizer -a cursor -y\n```\n\nCursor can also import the skill from GitHub via Settings -> Rules -> Project Rules -> Add Rule -> Remote Rule (Github). The Cursor plugin metadata lives at `.cursor-plugin/plugin.json`.\n\n### OpenCode\n\nTell OpenCode:\n\n```text\nFetch and follow instructions from https://raw.githubusercontent.com/fastxyz/skill-optimizer/refs/heads/main/.opencode/INSTALL.md\n```\n\nOr add the plugin to `opencode.json` at user or project scope:\n\n```json\n{\n  \"plugin\": [\"skill-optimizer@git+https://github.com/fastxyz/skill-optimizer.git\"]\n}\n```\n\nRestart OpenCode. See `docs/README.opencode.md` for details.\n\n### Gemini CLI\n\nInstall the Gemini extension from GitHub:\n\n```bash\ngemini extensions install https://github.com/fastxyz/skill-optimizer\n```\n\nTo update:\n\n```bash\ngemini extensions update skill-optimizer\n```\n\n### Skill-Only Install\n\nIf you only want the skill files without plugin metadata, use the open skills CLI:\n\n```bash\nnpx skills add fastxyz/skill-optimizer --skill skill-optimizer -a claude-code -a opencode -a codex -a cursor -y\n```\n\n## Local CLI Setup\n\nRequirements:\n\n- Node.js 20+\n- Docker\n- `OPENROUTER_API_KEY` for real model runs\n\nInstall and build:\n\n```bash\nnpm install\nnpm run build\n```\n\nOnly `openrouter/...` model refs are supported.\n\n## Quick Start\n\nRun the suite against the models listed in `suite.yml`:\n\n```bash\nnpx tsx src/cli.ts run-suite examples/workbench/pdf/suite.yml --trials 1\n```\n\nRun one case directly:\n\n```bash\nnpx tsx src/cli.ts run-case ./case.yml --model openrouter/google/gemini-2.5-flash\n```\n\nCLI help:\n\n```bash\nnpx tsx src/cli.ts --help\nnpx tsx src/cli.ts run-case --help\nnpx tsx src/cli.ts run-suite --help\n```\n\n## How The Workbench Works\n\nThe workbench gives an agent a skill/reference folder, an isolated `/work` directory, and deterministic graders. It is designed for evals where success can be verified from files, command logs, SQL, generated artifacts, or other local state.\n\nCore concepts:\n\n- A case is one user-like task plus one or more graders.\n- A suite is a matrix of cases and OpenRouter models.\n- `references/` is copied into `/work`; this is where the skill under test lives.\n- The agent phase sees only `/work`, not graders, hidden answers, `/case`, or `/results`.\n- Graders run after the agent with `$CASE`, `$WORK`, and `$RESULTS` available.\n- Graders are the acceptance contract. They can inspect workspace files and artifacts, `answer.json`, `trace.jsonl`, and result state under `$RESULTS`.\n\nRead `docs/workbench.md` for the full model: directory layout, Docker phases, graders, outputs, and debugging.\n\n## Examples\n\nTracked examples live under `examples/workbench/`. The PDF example includes positive PDF extraction/splitting/creation cases and a negative case that checks the agent did not read the PDF skill file for a non-PDF task. The MCP example shows a local calculator server started as a hidden Docker service and exposed through the workbench `mcp` command.\n\n```bash\nnpx tsx src/cli.ts run-suite examples/workbench/pdf/suite.yml --trials 1\nnpx tsx src/cli.ts run-suite examples/workbench/mcp/suite.yml --trials 1\n```\n\n## Development\n\n```bash\nnpm run typecheck\nnpm test\nnpm run build\nnpx tsx src/cli.ts --help\n```\n\nFor Docker runner or image changes:\n\n```bash\ndocker build -t skill-optimizer-workbench:local -f docker/workbench-runner.Dockerfile .\n```\n\nDo not commit `.skill-eval/`, `.results/`, `.env`, or credentials.\n",
  "bytes": 4738,
  "sha": "a87b967ade3c71d8845cf6dd59ac4cfd86b70214893abfeca50e94a8f07feeb7",
  "repo_slug": "fastxyz/skill-optimizer",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_fastxyz_skill_optimizer_5dc32759/readme"
}