{ "markdown": "

\n\n

autoresearch-skill

\n

\n Define a goal. Let the agent research, experiment, and iterate -- autonomously.\n

\n

\n When to Use · Quick Start · Features · Usage · 한국어\n

\n

\n \n \n \n \n \n \n \n \n

\n\n---\n\n### autoresearch-skill in Action\n\n| | Example | Result | Iterations | Evaluator |\n|:---:|:---|:---|:---:|:---:|\n| 1 | **Code Optimization** — Sort 1M integers faster | 2.12s → 0.15s (−93%) | 8 | `benchmark.py` |\n| 2 | **Function Fitting** — Discover hidden math function | RMSE 2.11 → 0.030 (−99%) | 8 | `evaluate.py` |\n| 3 | **Skill Elaboration** — Improve P&ID analysis skill | 0.28 → 0.98 composite (+255%) | 2 | `evaluate.py` |\n| 4 | **Literature Review** — Exercise timing papers | 1/8 → 8/8 categories, 19 papers | 4 | Agent (Tier 2) |\n\n> [!NOTE]\n> An LLM skill that turns natural-language research goals into autonomous experiment-evaluate-iterate loops -- inspired by [Karpathy's autoresearch](https://github.com/karpathy/autoresearch). Write a `research.md`, and the agent handles hypothesis generation, experimentation, evaluation, and iteration. Works with Claude Code, Codex CLI, OpenCode, and Gemini CLI.\n\n## Expected Outputs: Visual Result Gallery\n\nEach run leaves behind human-readable reports, machine-readable logs, and visual evidence. These examples are checked into the repo so you can see the shape of a completed autoresearch loop before running your own.\n\n| Example | Goal | Metric | Before → After | Iterations | Visual preview | Artifacts |\n|---|---|---:|---:|---:|---|---|\n| [Code Optimization](./examples/code-optimization/README.md) | Sort 1M integers faster | median runtime ↓ | 2.12s → 0.15s | 8 | [results.png](./examples/code-optimization/results.png) | `research.md`, `autoresearch-results.tsv`, `final_report.md` |\n| [Function Fitting](./examples/function-fitting/README.md) | Recover an unknown function from data | RMSE ↓ | 2.11 → 0.030 | 8 | [results.png](./examples/function-fitting/results.png) | `train_data.csv`, `test_data.csv`, `evaluate.py`, `final_report.md` |\n| [Skill Elaboration](./examples/skill-elaboration/README.md) | Improve a PDF/P&ID analysis skill | structural score ↑ | 0.28 → 0.98 | 2 | [results.png](./examples/skill-elaboration/results.png) | original/improved `SKILL.md`, `evaluate.py`, `final_report.md` |\n| [Literature Review](./examples/literature-review/README.md) | Fill exercise-timing literature coverage gaps | categories covered ↑ | 1/8 → 8/8 | 4 | [results.png](./examples/literature-review/results.png) | `research_log.md`, `autoresearch-results.tsv`, `final_report.md` |\n\nTypical final directory shape:\n\n```text\nmy-research/\n├── research.md # living state + iteration history\n├── research_log.md # append-only reasoning and evidence log\n├── autoresearch-results.tsv # machine-readable metric table\n├── progress.png # convergence plot refreshed during runs\n└── final_report.md # final result, failures, and next steps\n```\n\n## Features\n\n- **Karpathy-Inspired Loop** -- Autonomous experiment -> evaluate -> keep/revert cycle, generalized beyond ML training\n- **Natural Language Programming** -- `research.md` is your program: define goals, metrics, and constraints in plain English\n- **Zero Dependencies** -- Python stdlib only. No pip packages required for core functionality\n- **Multi-Agent Compatible** -- Works with Claude Code, Codex CLI, OpenCode, and Gemini CLI out of the box\n- **Automatic Rollback** -- Failed experiments are reverted automatically; only improvements are kept\n- **Full Audit Trail** -- Every iteration logged to `research_log.md` with timestamps, changes, and results\n- **3 Tier Environment Detection** -- Adapts to your runtime: full experimentation (Tier 1), research-only (Tier 2), or analysis-only (Tier 3)\n- **Safety Built In** -- Max iterations, pause-for-review intervals, forbidden-change boundaries, and time budgets\n\n## Command Inventory\n\n| Command | Purpose |\n|---------|---------|\n| `/autoresearch` | Core 5-stage loop — understand, hypothesize, experiment, evaluate, log & iterate |\n| `/autoresearch:plan` | 7-step setup wizard that produces a ready-to-run `research.md` |\n| `/autoresearch:debug` | Scientific bug hunting with falsifiable hypotheses and evidence tables |\n| `/autoresearch:fix` | Iterative error crusher — runs until error count reaches zero |\n| `/autoresearch:predict` | Multi-persona deliberation with anti-herd-bias detection |\n| `/autoresearch:security` | STRIDE + OWASP iterative security audit |\n| `/autoresearch:scenario` | 12-dimension scenario exploration for decision analysis |\n| `/autoresearch:reason` | Adversarial refinement with blind-judge scoring panel |\n| `/autoresearch:ship` | Universal shipping workflow supporting 9 ship types |\n| `/autoresearch:learn` | Feedback-to-eval loop for improving the skill itself |\n\n## Quick Decision Guide\n\n**What do you want to do?**\n\n| Goal | Use |\n|------|-----|\n| Optimize something iteratively toward a numeric target | `/autoresearch` |\n| Set up a new research project from scratch | `/autoresearch:plan` |\n| Hunt down a hard-to-reproduce bug | `/autoresearch:debug` |\n| Crush all errors in a codebase to zero | `/autoresearch:fix` |\n| Forecast outcomes or predict what will happen | `/autoresearch:predict` |\n| Audit a system for security vulnerabilities | `/autoresearch:security` |\n| Explore \"what if\" scenarios before committing to a path | `/autoresearch:scenario` |\n| Think through a complex decision rigorously | `/autoresearch:reason` |\n| Release a feature, library, or artifact | `/autoresearch:ship` |\n| Turn a failed/confusing skill run into an improvement plan | `/autoresearch:learn` |\n\n## Why This Skill?\n\nOther autoresearch implementations provide the loop concept. This repo provides the **complete toolkit**:\n\n- **4 worked examples** with real measured data -- not templates, not placeholders\n- **Visual evidence** -- before/after charts, optimization trajectories, error heatmaps\n- **Multi-agent compatible** -- works with Claude Code, Codex CLI, OpenCode, and Gemini CLI\n- **Copy-paste install** -- one block, paste into your LLM chat, done\n- **Scaffolding tool** -- `init_research.py` creates a ready-to-run research project in seconds\n- **Core principles** -- 8 formalized Karpathy principles with practical mapping to `research.md`\n- **Stuck detection** -- automatic strategy shifts when the loop plateaus\n- **Endgame strategy** -- switches from explore to exploit when iterations are running out\n- **TSV logging** -- machine-readable `autoresearch-results.tsv` for CI integration and analysis\n\n## When to Use\n\nMost LLM CLI tools already ship with iterative execution modes -- `ralph`, `autopilot`, `/loop`, cron-based scheduling, etc. Those work well for code-centric tasks tied to git, test runners, and build systems. autoresearch-skill targets a different problem shape: **anything with a numeric metric and a search space to explore**, whether or not it involves code.\n\n### autoresearch-skill vs built-in iterative modes\n\n| | Manual prompting | Built-in modes (ralph, autopilot, team) | autoresearch-skill |\n|:---|:---|:---|:---|\n| **Domain** | Anything, but you drive each cycle | Code projects (git + tests + build) | Any domain with a measurable metric |\n| **Evaluation** | LLM self-reports results | Acceptance criteria, often subjective | Mechanical evaluator: `{\"pass\": true, \"score\": 0.94}` |\n| **On plateau** | You decide what to try next | Retry or terminate | 3-level pivot -- switch strategy, then paradigm, then finalize |\n| **Autonomy** | One cycle per human turn | High, but verification gates can pause | Uses full iteration budget without asking |\n| **Overnight runs** | Not practical | Platform-specific (`/loop`, `CronCreate`) | Cross-platform bash script (Claude Code, Codex, Gemini CLI) |\n| **Environment** | Depends on tool | Assumes shell access | 3-tier auto-detection (shell / web-only / text-only) |\n| **Dependencies** | Varies | git, pytest, etc. | Python 3.8+ stdlib only |\n\n### Pick autoresearch-skill when\n\n- You have a **numeric metric** and a script that outputs `{\"pass\": bool, \"score\": number}` -- the mechanical evaluator removes LLM judgment from keep/revert decisions\n- The problem is **not a code project** -- simulation parameter sweeps, literature coverage gaps, prompt tuning against test cases, function fitting from data\n- You need **overnight runs across CLI platforms**, not just Claude Code\n- Progress will **plateau**, and you want the agent to pivot strategy instead of stopping\n- You want a **machine-readable audit trail** (TSV + append-only log) of every iteration\n\n### Stick with built-in modes when\n\n- The task is **bug fixes or feature implementation** -- ralph and autopilot understand PRDs, acceptance criteria, and code review workflows\n- Multiple agents need to **work on different subtasks in parallel** -- that's what team mode does\n- There is **no measurable metric** -- autoresearch needs a target to iterate toward\n- **One attempt is enough** -- no iteration loop needed\n\n## Quick Start\n\n### Copy-Paste Install\n\n> [!TIP]\n> Works with any LLM CLI that supports skills (Claude Code, Codex, Gemini CLI). Just paste the block below into your chat.\n\n```\nI want to install the autoresearch-skill. Do these steps:\n1. git clone https://github.com/wjgoarxiv/autoresearch-skill.git /tmp/autoresearch-skill\n2. mkdir -p ~/.claude/skills/autoresearch-skill && cp -r /tmp/autoresearch-skill/SKILL.md /tmp/autoresearch-skill/scripts /tmp/autoresearch-skill/assets ~/.claude/skills/autoresearch-skill/\n3. Test: python ~/.claude/skills/autoresearch-skill/scripts/init_research.py --goal \"test\" --metric \"score\" --direction maximize --output /tmp/test-research && echo \"OK: autoresearch-skill installed\"\n4. Say \"autoresearch-skill installed successfully\"\n```\n\n### Plugin Marketplace Install\n\nIf your LLM CLI supports a plugin marketplace (`.claude-plugin/` discovery), paste this single block into your chat:\n\n```\nInstall the autoresearch-skill plugin:\n1. git clone https://github.com/wjgoarxiv/autoresearch-skill.git /tmp/autoresearch-skill\n2. mkdir -p ~/.claude/plugins && cp -r /tmp/autoresearch-skill/.claude-plugin ~/.claude/plugins/autoresearch-skill\n3. Reload plugins and confirm: \"autoresearch-skill plugin installed\"\n```\n\n### Manual Install\n\n```bash\n# Clone the repo\ngit clone https://github.com/wjgoarxiv/autoresearch-skill.git\ncd autoresearch-skill\n\n# Symlink into your skills directory\nmkdir -p ~/.claude/skills\nln -s \"$(pwd)\" ~/.claude/skills/autoresearch-skill\n\n# No pip dependencies needed!\n```\n\n### Other Tools\n\n| Tool | Skills Path | Install Command |\n|------|-------------|-----------------|\n| **Claude Code** | `~/.claude/skills/autoresearch-skill/` | See above |\n| **Codex CLI** | `~/.codex/skills/autoresearch-skill/` | `mkdir -p ~/.codex/skills && ln -s \"$(pwd)\" ~/.codex/skills/autoresearch-skill` |\n| **OpenCode** | `~/.config/opencode/skills/autoresearch-skill/` | `mkdir -p ~/.config/opencode/skills && ln -s \"$(pwd)\" ~/.config/opencode/skills/autoresearch-skill` |\n| **Gemini CLI** | `~/.gemini/skills/autoresearch-skill/` | `mkdir -p ~/.gemini/skills && ln -s \"$(pwd)\" ~/.gemini/skills/autoresearch-skill` |\n\n## Installation\n\nCopy this skill into your CLI tool's skills directory:\n\n| Platform | Command |\n|----------|---------|\n| **Claude Code** | `cp -r autoresearch-skill/ ~/.claude/skills/autoresearch-skill/` |\n| **Codex CLI** | `cp -r autoresearch-skill/ ~/.codex/skills/autoresearch-skill/` |\n| **OpenCode** | `cp -r autoresearch-skill/ ~/.config/opencode/skills/autoresearch-skill/` |\n| **Gemini CLI** | `cp -r autoresearch-skill/ ~/.gemini/skills/autoresearch-skill/` |\n\nOr clone directly:\n```bash\ngit clone https://github.com/wjgoarxiv/autoresearch-skill.git\ncp -r autoresearch-skill/ ~/.claude/skills/ # adjust path for your platform\n```\n\nThe skill is automatically discovered when you mention \"autoresearch\" or \"research.md\" in your prompt.\n\n## Usage\n\n### 1. Literature Review\n\n```\nResearch the latest advances in \"LLM agents for scientific discovery\".\nFind and synthesize at least 15 papers from 2024-2026.\n```\n\n### 2. Code Optimization\n\n```\nMy sort function takes 2.3s on 1M items. Use auto-research to make it faster.\nTarget: under 0.5 seconds. Pure Python only, no C extensions.\n```\n\n### 3. Function Fitting\n\n```\nI have data points from an unknown function in train_data.csv.\nUse autoresearch to discover the function. Minimize RMSE below 0.05.\nHere's my evaluate.py that outputs {\"pass\": true, \"score\": -0.034}.\n```\n\n### 4. Scaffold a New Research Project\n\n```bash\npython scripts/init_research.py \\\n --goal \"Optimize database query performance\" \\\n --metric \"query_time_ms\" \\\n --direction minimize \\\n --target \"< 50\" \\\n --output ./db-research/\n```\n\n## Overnight Runs\n\nTo run autoresearch overnight (or for days), use the universal loop script:\n\n```bash\n# 1. Set up your research project\npython scripts/init_research.py --goal \"...\" --metric \"...\" --direction maximize --output ./my-research/\n\n# 2. Start the overnight loop (pick one)\n\n# Option A: Keep terminal open (simplest)\nbash scripts/autoresearch-loop.sh ./my-research/\n\n# Option B: Background without tmux\nnohup bash scripts/autoresearch-loop.sh ./my-research/ > autoresearch.log 2>&1 &\n\n# Option C: Background with tmux (best experience)\ntmux new-session -d -s research 'bash scripts/autoresearch-loop.sh ./my-research/'\n\n# 3. Check progress anytime\nbash scripts/check_progress.sh ./my-research/\n```\n\nThe script auto-detects your CLI tool and handles session restarts, completion detection, and safety limits. Works with Claude Code, Codex CLI, OpenCode, and Gemini CLI. No dependencies beyond bash.\n\n## How It Works\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│ research.md │\n│ (Goal, Metric, Constraints, Search Space, History) │\n└─────────────────────┬───────────────────────────────────────┘\n │\n v\n ┌─────────────────┐\n │ 1. UNDERSTAND │ Read research.md + history\n └────────┬────────┘\n v\n ┌─────────────────┐\n │ 2. HYPOTHESIZE │ Propose a testable change\n └────────┬────────┘\n v\n ┌─────────────────┐\n │ 3. EXPERIMENT │ Execute: run code / search / analyze\n └────────┬────────┘\n v\n ┌─────────────────┐\n ┌──│ 4. EVALUATE │──┐\n │ └─────────────────┘ │\n improved? not improved?\n │ │\n ┌────v────┐ ┌─────v─────┐\n │ KEEP │ │ REVERT │\n └────┬────┘ └─────┬─────┘\n │ │\n └──────────┬────────────┘\n v\n ┌─────────────────┐\n │ 5. LOG & ITERATE │──→ Back to step 1\n └─────────────────┘ (or STOP if done)\n```\n\n## Output Format\n\nThe skill produces a small, predictable artifact bundle:\n\n| File | Purpose | Updated |\n|------|---------|---------|\n| `research.md` | Living research document with goal, constraints, search space, and history | Every iteration |\n| `research_log.md` | Detailed append-only experiment log: hypothesis, command output, evaluator result, keep/revert decision | Every iteration |\n| `autoresearch-results.tsv` | Machine-readable 8-column metric table for plotting, CI, and later analysis | Every iteration |\n| `progress.png` | Lightweight convergence plot showing metric trajectory and best-so-far envelope | Every iteration when plotting is available |\n| `results.png` / `results.pdf` | Example-specific final visualization, if the run produces one | End of run |\n| `final_report.md` | Structured summary with best result, failed attempts, reproducibility commands, and next steps | End only |\n\n## Environment Tiers\n\nThe skill automatically detects your runtime capabilities:\n\n| Tier | Environment | Capabilities | Use Case |\n|------|-------------|--------------|----------|\n| **Tier 1** | Claude Code, Codex CLI, terminal | Bash + Python + full tools | Run experiments, benchmark, modify files |\n| **Tier 2** | Claude App with web access | WebFetch + WebSearch | Literature review, web research |\n| **Tier 3** | Restricted (no shell, no network) | Text generation | Analyze user data, propose hypotheses |\n\n## Requirements\n\n| Requirement | Details |\n|-------------|---------|\n| **Python** | 3.8+ (stdlib only) |\n| **LLM CLI** | Claude Code, Codex CLI, OpenCode, or Gemini CLI |\n| **Domain tools** | Varies by use case (e.g., Python for code optimization, web access for lit review) |\n\n## Inspired By\n\n[Karpathy's autoresearch](https://github.com/karpathy/autoresearch) -- a 630-line framework where an AI agent autonomously runs ML experiments overnight. This skill generalizes that loop to any domain where you have a measurable goal and a search space to explore.\n\n## Contributing\n\nContributions welcome! Ideas for new example `research.md` templates are especially appreciated.\n\n1. Fork the repo\n2. Create your feature branch (`git checkout -b feature/amazing-example`)\n3. Commit your changes (`git commit -m 'Add amazing example'`)\n4. Push to the branch (`git push origin feature/amazing-example`)\n5. Open a Pull Request\n\n## License\n\nMIT -- see [LICENSE](./LICENSE) for details.\n", "bytes": 18269, "sha": "070bb84a6d6aee5402204e76abfb10dba7b450b18038812c6477a373b91fee02", "repo_slug": "wjgoarxiv/autoresearch-skill", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/plg_wjgoarxiv_autoresearch_skill_de6ff33e/readme" }