{
  "markdown": "# ResearchOps Toolkit\n\nResearchOps Toolkit is a modular **Research and Development workflow plugin** for Codex, Claude Code, Gemini CLI, and optional third-party model workers. It combines progressively loaded Skills, deterministic runtime services, task-aware behavior policies, evidence/state management, and project-scoped model evaluation and routing.\n\n> It does not try to make one Agent “write a paper automatically.” It gives questions, work units, artifacts, evidence, model assignments, failures, costs, decisions, risks, and human approvals explicit owners and durable records.\n\nVersion 2.1 contains 13 top-level Skills, eight Behavior Packs, seven internal components, one SQLite-backed Model Intelligence core, native hook adapters, and filtered release Presets.\n\n## Send this to your Agent\n\n> Clone `git@github.com:hungryDodo/researchops-toolkit.git`, enter the repository, run `python3 -m rops inspect /path/to/project`, then `python3 -m rops bootstrap /path/to/project --title \"My Project\" --mode auto --upgrade`, install the `research-routed` Preset for your Harness, and use `python3 -m rops up --root /path/to/project --open` to review the adopted project state and Dashboard.\n\nReplace `codex` with `claude`, `gemini`, `portable`, or `all` where appropriate.\n\n## Quick start\n\n```bash\ngit clone git@github.com:hungryDodo/researchops-toolkit.git\ncd researchops-toolkit\n\npython3 -m rops inspect /path/to/project\n\npython3 -m rops bootstrap /path/to/project \\\n  --title \"My Project\" \\\n  --mode auto \\\n  --upgrade\n\npython3 -m rops install \\\n  --target codex \\\n  --scope project \\\n  --project /path/to/project \\\n  --mode link \\\n  --preset research-routed \\\n  --with-agents \\\n  --with-behavior \\\n  --behavior-mode guide\n\npython3 -m rops doctor --target codex --project /path/to/project\npython3 -m rops up --root /path/to/project --open\n```\n\nThe installer records the selected Preset beside the harness Skills, so `doctor` checks the intended\ninventory instead of reporting intentionally omitted Skills as missing.\n\nOn Codex, project installation preserves existing `.codex/hooks.json` groups while adding the\nselected Behavior handlers. Codex still requires its normal user trust decision for new or changed\nhooks.\n\nThe project receives one hidden root:\n\n```text\n.researchops/\n├── state/          project designs, runs, evidence, decisions, dashboard state\n├── governance/     project policies, registries, routing configuration\n├── intelligence/   authoritative SQLite database and generated projections\n├── runtime/        replaceable hooks, Behavior Runtime, and local runtime copy\n├── artifacts/      large or generated local artifacts\n├── cache/\n└── logs/\n```\n\nThere is no second `.research/` state directory in v2. Legacy projects are migrated into this single root.\n\nResearchOps does not assume that installing the plugin means starting from zero. `rops inspect` runs before writes. A non-empty repository is adopted without overwriting its files or adding root policy files by default; the deterministic inventory is then interpreted by the Research Program Orchestrator at a light, standard, or deep adoption depth. The inferred phase is visible immediately but remains explicitly reviewable.\n\n## The important architectural distinction\n\nResearch, Development, and Routing are **not three equivalent workflow domains**:\n\n- **Research-led R&D** and **Development-led R&D** are user-facing workflow orientations with different objective functions and acceptance rules.\n- **Model Intelligence and Routing** are cross-cutting services that can support either workflow, Visual Communication, Hardware work, or a custom external task family.\n- Communication/Visual, Hardware, Hygiene, and Skill-system maintenance remain separate domains where their artifacts, permissions, or risk boundaries differ materially.\n\n```text\n                         Installation Preset / native plugin\n                                      │\n                 ┌────────────────────┴────────────────────┐\n                 │                                         │\n        User-facing workflow Skills                 Behavior Runtime\n Research-led R&D · Development-led R&D       scope · evidence · risk · approval\n Communication · Hardware · Hygiene                         │\n                 │                                         │\n                 └────────────────────┬────────────────────┘\n                                      │\n                         Deterministic ROPS runtime\n       Model Gateway · Model Intelligence · Engineering Assurance · Evidence\n                                      │\n                         canonical `.researchops/` state\n                                      │\n        routing · dossier · dashboard · benchmark · audit projections\n                                      │\n                      optional local/external recall adapters\n```\n\n## Why SQLite from the first event\n\nModel evaluation records are relational, time-dependent, frequently updated, and queried by project, work-unit operation, model execution configuration, endpoint, Judge, failure pattern, and mitigation. ResearchOps therefore uses:\n\n```text\n.researchops/intelligence/state.sqlite\n```\n\nas the authoritative source from the first Evaluation Event. JSONL is supported only for import, export, audit, and reproducible research exchange:\n\n```bash\npython3 -m rops intelligence --root /path/to/project export-jsonl \\\n  --out /tmp/evaluation-events.jsonl\n\npython3 -m rops intelligence --root /path/to/project import-jsonl \\\n  /tmp/evaluation-events.jsonl\n```\n\nOne deterministic profile engine reads canonical events and regenerates all projections. Routing profiles and model dossiers no longer maintain separate aggregation logic.\n\n## Model Intelligence at a glance\n\nThe independently installable Model Intelligence module provides:\n\n- canonical live, shadow, and anchor Evaluation Events;\n- finite task-conditioned profile slices rather than a high-dimensional Cartesian product;\n- joint model × reasoning-effort arms, effort-demand fit, hard effort bounds, and executable Codex dispatch fields;\n- route-driven isolated worker sessions across Codex-native and profiled third-party providers, with bounded eligible-arm fallback and acceptance-event closure;\n- task-conditioned single/Lead-worker/centralized-fanout topology instead of fixed company-style model personas;\n- posterior success estimates, verified progress, quality, cost, latency, human correction, and uncertainty;\n- separate endpoint-health, pricing, and declared/observed model-identity signals;\n- project warmup, soft transfer, zero-start comparison, and negative-transfer rejection;\n- aggregated failure patterns and scoped mitigation revisions;\n- blind/pairwise Judge observations with task-family-conditioned calibration;\n- black-box behavior-drift alerts and deployment epochs;\n- read-only routing, dossier, dashboard, benchmark, and audit views;\n- lifecycle-aware four-layer local Memory with scoped deduplication, temporal validity, supersession, provenance, project synchronization, and optional external adapters that never replace authoritative facts.\n\nConnectivity probes and smoke tests update endpoint/identity telemetry only. They never teach the router that a model is competent.\n\n## Project dashboard\n\n`python3 -m rops up --root /path/to/project --open` is the one-command adoption/status/Dashboard path. The project-progress dashboard includes compact Intake, Memory, Routing, and Model Intelligence panels. It shows only information useful to a human operator:\n\n- the current model preference and concise reason;\n- recent model assignments and completed work;\n- verified observations, success trend, cost, and service health;\n- project adaptation/warmup progress;\n- active behavior-drift warnings;\n- selected failure patterns and mitigations when actionable.\n\nLow-level score factors, posterior parameters, policy internals, and complete audit evidence remain available in generated projections rather than crowding the main view.\n\n## Installation Presets\n\nA Preset is an installation and packaging recipe. It is not a Git bundle, Git submodule, or code-ownership boundary.\n\n```bash\npython3 -m rops presets\npython3 -m rops presets routing-core --format json\n```\n\n| Preset | Purpose |\n|---|---|\n| `routing-core` | Model gateway, evaluation, routing, drift, Judge calibration, warmup, and dashboard support |\n| `development-core` | Development-led technical investigation, implementation, debugging, review, and release assurance |\n| `research-base` | Research-led workflow without requiring dynamic multi-model routing |\n| `research-routed` | `research-base` plus `routing-core`; default source-tree preset |\n| `communication-visual` | Academic communication and optional visual-reference intake |\n| `hardware` | Physical experiment workflow and hardware safety |\n| `hygiene` | Archive-first repository/data lifecycle |\n| `platform-dev` | Maintaining Skills, hooks, manifests, and the plugin system itself |\n| `full` | All supported capabilities |\n\n`rops bundles` and `--bundle` remain compatibility aliases.\n\n## User-facing Skills and internal task codes\n\nUsers describe the goal in ordinary language or invoke a stable top-level Skill. Internal codes such as `discover`, `design`, `implement`, `debug`, `validate`, and `communicate` are machine-facing work-unit descriptors used by routing, evaluation, and Benchmark Packs. Users do not need to memorize them.\n\n| Skill | Primary owner |\n|---|---|\n| `research-program-orchestrator` | lifecycle, gates, next owner, project progress |\n| `research-discovery` | traceable survey, closest work, Related Work synthesis |\n| `research-route-evaluator` | fatal-flaw checks and a bounded set of falsifiable routes |\n| `experimental-research` | experiment contract, execution, analysis, evidence |\n| `research-engineering` | research-led code whose behavior can affect claims or measurements |\n| `software-development` | development-led investigation, implementation, debugging, review, release |\n| `adaptive-agent-orchestration` | bounded delegation, model routing, independent acceptance |\n| `research-validation` | reproduction, artifact audit, manuscript red-team review |\n| `research-writing` | evidence-gated drafting and LaTeX revision |\n| `research-communication` | publication figures, result plots, research presentations |\n| `hardware-experiment-loop` | physical topology, calibration, leases, restoration |\n| `project-hygiene` | archive, restore, quarantine, approved purge, worktrees |\n| `skill-system-engineering` | Skill/Pack boundaries, triggers, hooks, provenance, release |\n\n## Research-led and Development-led R&D\n\nBoth orientations share the same engineering skeleton:\n\n```text\nFrame → Investigate → Decide → Implement → Verify → Learn\n```\n\nResearch-led work optimizes for valid knowledge and claim-to-evidence linkage; novel, expensive, or negative-result routes can remain useful. Development-led work optimizes for a reliable, maintainable, deployable deliverable and rejects complexity whose system-level value does not justify its cost. The shared `engineering-assurance` component provides task contracts, RED/baseline evidence, diff analysis, and risk-scaled verification; each Skill adds its own acceptance rules.\n\n## Visual reference intake is optional\n\nA user may send a reference image to any capable external vision model and ask it to produce the schema in `components/visual-contracts/visual-reference.schema.json` using `templates/visual-reference-analysis.md`. ResearchOps consumes the resulting design brief; it does not require the main Harness to have vision capability and does not copy brand-specific assets.\n\n## Model and provider identity\n\nAn execution arm records model family/revision, endpoint, quantization or reasoning configuration, adapter/tool schema, Harness/prompt revision, mitigation bundle, and a local deployment epoch. When a closed provider exposes no immutable revision, ResearchOps cannot prove that hidden weights changed. It can detect sustained changes in observable behavior, format/tool compliance, latency, token use, anchor outcomes, and returned identity metadata, then isolate future evidence in a new deployment epoch.\n\n## Memory boundary\n\nROPS owns authoritative state. Built-in Memory v2.1 adds episodic, semantic, procedural, and preference layers; candidate/active/superseded/retired lifecycle; source-aware deduplication; temporal validity; relations; project synchronization; and bounded provenance-bearing context bundles. Harness memory, vector stores, or temporal graph systems remain optional Recall Adapters. Retrieved memory cannot directly change a routing profile, approve a mitigation, replace current prices, or authorize a high-risk operation.\n\n## Product evaluation and baselines\n\nResearchOps now evaluates the tool as well as the models it routes:\n\n```bash\npython3 -m rops evaluate \\\n  --baseline-root /path/to/older/researchops \\\n  --out /tmp/researchops-product-benchmark\n```\n\nThe bundled deterministic suite measures non-destructive adoption, actual HTTP Dashboard startup, SQLite authority, and Memory lifecycle behavior. External products can be ingested through the standardized report contract, but a feature matrix is not treated as proof of superior performance. Paired Skill/Workflow studies and longitudinal project outcomes remain necessary for broader claims.\n\n## Packaging and releases\n\nGenerate a filtered native artifact from the monorepo:\n\n```bash\npython3 -m rops package \\\n  --out /tmp/researchops-release \\\n  --preset routing-core \\\n  --target codex\n\npython3 -m rops package \\\n  --out /tmp/researchops-release \\\n  --preset full \\\n  --target portable\n```\n\nEach archive contains only the selected Skills, components, and Behavior Packs; its own default Preset, catalog, native manifest, validation report, and SHA-256 manifest are regenerated inside the package.\n\n## Documentation\n\n| Document | Purpose |\n|---|---|\n| [Documentation index](docs/README.md) | Recommended reading paths |\n| [Getting started](docs/getting-started.md) | Inspect/adopt, install, quick-start Dashboard, upgrade |\n| [Architecture and state](docs/architecture.md) | Horizontal layers, vertical capability slices, authority |\n| [Model Intelligence](docs/model-intelligence.md) | events, aggregation, projections, routing, drift, warmup, Judge |\n| [Research agenda](docs/research-agenda.md) | research questions, baselines, metrics, ablations, and empirical boundaries |\n| [Research and Development](docs/research-and-development.md) | research-led vs development-led objective functions |\n| [Presets and distribution](docs/presets-and-distribution.md) | monorepo composition and filtered native artifacts |\n| [Product landscape](docs/product-landscape.md) | Adjacent observability, research-agent, coding-agent, and memory systems |\n| [Evaluation and baselines](docs/evaluation-and-baselines.md) | Tool regression, external adapters, paired Skill tests, routing and user studies |\n| [State and memory](docs/state-and-memory.md) | `.researchops/`, SQLite authority, lifecycle-aware Memory and optional adapters |\n| [Skills and progressive loading](docs/skills-and-bundles.md) | top-level ownership, internal modes, triggers |\n| [Agents and model routing](docs/agents-and-model-routing.md) | provider setup, execution arms, dispatch and evaluation |\n| [External provider configuration](docs/provider-configuration.md) | DeepSeek, GLM/LiteLLM bridge, MiMo, MiniMax endpoints, credentials, modes, and Codex compatibility |\n| [Safety and hygiene](docs/safety-and-hygiene.md) | lifecycle hooks, approvals, archive-first operations |\n| [Migration to v2](docs/migration-v2.md) | legacy state and JSONL migration |\n| [Development and release](docs/development.md) | tests, provenance, packaging, contribution |\n\nAgents modifying this repository should read [`AGENTS.md`](AGENTS.md).\n\n## Validation\n\n```bash\npython3 -m rops validate\npython3 tests/smoke.py\npython3 tests/intelligence_smoke.py\npython3 tests/behavior_smoke.py\npython3 tests/model_control_plane_smoke.py\npython3 tests/model_effort_routing_smoke.py\npython3 tests/worker_dispatch_smoke.py\n# After installing the pinned LiteLLM proxy dependency:\npython3 tests/litellm_glm_bridge_smoke.py\npython3 tests/adoption_memory_smoke.py\npython3 -m rops evaluate --out /tmp/researchops-product-benchmark\npython3 -m rops package --out /tmp/researchops-release --preset full --target portable\n```\n\nThe tests validate deterministic structure and behavior. They do not prove that every future model/provider/Harness will route perfectly, that a closed provider did or did not change hidden weights, or that a research project will reach a particular venue.\n\n## Provenance and license\n\nResearchOps Toolkit is released under the [MIT License](LICENSE). External projects and platform documentation informed design analysis, but no third-party Skill, prompt, script, template, handbook, or visual asset is vendored by default. See [`PROVENANCE.json`](PROVENANCE.json) for the machine-readable declaration.\n",
  "bytes": 17044,
  "sha": "f0b5208f29ae1fbe24ce98b4040556377c73e2ebf84d4663c4bdf8163fe555a7",
  "repo_slug": "hungrydodo/researchops-toolkit",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_hungrydodo_researchops_toolkit_9a69d55a/readme"
}