methods-mcp
MCP server for structured methods extraction + reproducibility heuristics on academic papers.
Open source Open in the app JSON README (API)
About
MCP server for structured methods extraction + reproducibility heuristics on academic papers.
Details
- Kind
- MCP servers
- Topic
- Government & public data
- Publisher
- flynnlachendro
- Origin
- official
- Category
- ferramentas
- Transport
- local
- Version
- 0.1.7
- Open pull requests
- 1
- Last push
- 2026-07-01T16:37:48Z
- Repository state
- ativo
- Language
- Python
- License
- MIT
- Added
- 2026-08-29 03:01:55
- Updated
- 2026-08-29 03:01:55
- Origin id
io.github.FlynnLachendro/methods-mcp
README
# methods-mcp
<!-- mcp-name: io.github.FlynnLachendro/methods-mcp -->
[](https://pypi.org/project/methods-mcp/)
[](https://pypi.org/project/methods-mcp/)
[](LICENSE)
> Lightweight, on-demand MCP server for **structured methods extraction** + **reproducibility heuristics** on academic papers. Built for the [Worldwide AI Science Fellowship](https://www.aisciencesummit.com/) build challenge.
> ⚠️ **Status: alpha (0.1.x).** The tool surface and output shapes may shift between minor versions. **Pin to an exact version** in production. Bug reports very welcome via [GitHub Issues](https://github.com/FlynnLachendro/methods-mcp/issues).
## Quick demo
```text
$ uvx --from methods-mcp methods-mcp --version
methods-mcp 0.1.6
# In a Claude Code session:
> /mcp add methods-mcp methods-mcp
> Run methods_repro_review on https://arxiv.org/abs/2509.06917
→ tool: methods_repro_review({"input_str":"https://arxiv.org/abs/2509.06917"})
# Returns a MethodsReproReview object. Read `narrative` first — it explains
# everything else in plain English, so no tool-learning is required:
{
"status": "ok",
"narrative":
"Resolved the paper: 'Paper2Agent' by Miao et al. (arxiv 2509.06917, "
"2025-09-08). Extracted 11 methods steps at moderate self-reported "
"confidence (0.72) — the procedure is clearly described but hyperparameters "
"and software versions are absent. Detected the associated code repository "
"https://github.com/jmiao24/Paper2Agent from an inline link in the paper "
"text (detection confidence 0.94). The repo scored 0.90/1.00 on the "
"reproducibility heuristic — verdict: likely reproducible. Present signals: "
"substantive README, dependencies file, notebooks, figure-plotting script, "
"recent activity, permissive license. Missing: data/fixtures directory. "
"Suggested entrypoint: `python make_figures.py`.",
"metadata": { ... }, # PaperMetadata
"methods": { ... }, # MethodsStructured (null if extraction failed)
"code_repo": { ... }, # CodeRepo (null only if input unresolvable)
"repro_assessment": { ... }, # ReproAssessment (null if no repo detected)
"errors": [] # [{step, error_type, message, hint}] on partial
}
```
---
`methods-mcp` is a small, sharply-scoped [Model Context Protocol](https://modelcontextprotocol.io) server. It gives any AI agent (Claude Code, Claude Desktop, your Agent SDK script, etc.) eight tools that turn an academic paper URL into:
- canonical metadata,
- best-effort full text + section split,
- a **Pydantic-validated structured methods object** (steps / reagents / equipment / analyses),
- the paper's associated **code repository** (best-effort discovery),
- a **no-execution-required reproducibility verdict** for that repo, and
- a multi-mode summary.
The wedge: heavyweight pipelines like [Paper2Agent](https://arxiv.org/abs/2509.06917) (Stanford) take 30 minutes to hours to digest a paper into agent-ready tools. `methods-mcp` is the **agent-callable, on-demand** complement — every tool returns in seconds, no clone, no execution.
---
## Install
```bash
uv add methods-mcp
# or, install globally:
uv tool install methods-mcp
# or, classic pip:
pip install methods-mcp
```
### API keys
For best performance, set **both**:
| Variable | Required? | What you get without it |
|---|---|---|
| `ANTHROPIC_API_KEY` | **Required** for `extract_methods`, `summarize_paper`, `methods_repro_review` | Those tools raise `RuntimeError: ANTHROPIC_API_KEY not set`. Non-LLM tools (`fetch_paper_text`, `find_code_repo`, `assess_repo_reproducibility`) still work fine. |
| `GITHUB_TOKEN` | **Optional but recommended** for `assess_repo_reproducibility` / `methods_repro_review` | You're capped at the GitHub unauthenticated rate limit (**60 req/hr per IP**). Each repo assessment is ~3 calls, so you'll hit the ceiling after ~15–20 repos/hr. With a token: **5,000 req/hr** (effectively unlimited). |
```bash
export ANTHROPIC_API_KEY=sk-ant-...
export GITHUB_TOKEN=ghp_... # optional but recommended
```
Neither key is logged or persisted — they're sent only to `api.anthropic.com` and `api.github.com` respectively. See [`SECURITY.md`](SECURITY.md).
## Use it from Claude Code
```
/mcp add methods-mcp methods-mcp
```
Then in any Claude Code chat:
> Take https://arxiv.org/abs/2509.06917 and run `methods_repro_review`. Summarise what the paper does, the methods steps, and how reproducible the repo looks.
## Use it from the Claude Agent SDK
```python
from claude_agent_sdk import ClaudeAgentOptions, ClaudeSDKClient
options = ClaudeAgentOptions(
mcp_servers={
"methods-mcp": {
"type": "stdio",
"command": "methods-mcp",
"args": [],
}
},
allowed_tools=["mcp__methods-mcp__methods_repro_review"],
)
async with ClaudeSDKClient(options=options) as client:
await client.query(
"Run methods_repro_review on https://arxiv.org/abs/2509.06917 "
"and tell me whether the repo looks reproducible."
)
async for msg in client.receive_response():
print(msg)
```
## Tools
| Tool | What it does |
|---|---|
| `health` | Server liveness + config check. |
| `get_paper_metadata(input_str)` | Resolve URL / arXiv ID / DOI to canonical metadata. arXiv inputs hit the arXiv export API for title/authors/abstract. |
| `fetch_paper_text(input_str, prefer="auto"\|"html"\|"pdf")` | Full text + section split. Defaults to ar5iv HTML for arXiv papers (cheap, structured), PDF fallback otherwise. |
| `extract_methods(input_str, model=None)` | LLM-driven, Pydantic-validated structured methods extraction. Returns `{steps, reagents, equipment, analyses, confidence}`. |
| `find_code_repo(input_str)` | Discover the paper's code repo via paper text → abstract → Papers With Code. |
| `assess_repo_reproducibility(repo_url, paper_id=None)` | Heuristic, no-clone reproducibility assessment via the GitHub REST API. Weighted signals (README, deps, fixtures, notebooks, figure scripts, recent maintenance, license) → `{verdict, score, recommended_entrypoint}`. |
| `summarize_paper(input_str, mode="tldr"\|"abstract"\|"exec")` | LLM summary in three depths. |
| `methods_repro_review(input_str)` | Composite — metadata + methods + repo + repro in one call. |
All tools return Pydantic v2 models (validated, JSON-serialisable). See [`src/methods_mcp/schemas.py`](src/methods_mcp/schemas.py) for the full type surface.
## Design notes
- **`extract_methods` uses Anthropic tool-use to coerce the model into emitting an instance of the `MethodsStructured` Pydantic schema.** On validation failure we send one repair message with the validation error and try again before raising.
- **`assess_repo_reproducibility` does not clone or execute anything.** It scores the repo from publicly-readable GitHub metadata + the recursive tree listing. This is the deliberate wedge against batch tools that try to actually rerun the paper.
- **`fetch_paper_text` prefers ar5iv HTML over PDF parsing for arXiv papers.** Falls back to `pypdf` for non-arXiv inputs.
- **The default model is `claude-sonnet-4-6`.** Override via `METHODS_MCP_MODEL` env var or per-call `model=` arg.
- **`methods_repro_review` returns a self-describing response.** Every call sets a top-level `status` (`"ok"` / `"partial"` / `"empty"`) and a `narrative` string that summarises everything retrieved in plain English — including every numeric score in context. A reader who reads only `narrative` + `status` gets the full picture without needing to learn the sub-object shapes. Sub-objects can be `null` when unavailable (e.g. `repro_assessment: null` on a paper with no detected repo — `status` stays `"ok"` because "no repo" isn't a failure). Failed sub-steps contribute a structured entry to `errors` with `{step, error_type, message, hint}`, where `hint` is an actionable plain-English suggestion for recognised patterns (missing API keys, rate-limits, 404s, timeouts, etc.) and `null` otherwise.
## Scores & verdicts explained
Tool outputs contain three numeric fields that look similar but mean very different things. They are **triage signals for an agent deciding whether a paper is worth digging into**, not calibrated claims about correctness.
| Field | Range | How it's computed | How to read it |
|---|---|---|---|
| `methods.confidence` | 0–1 | **LLM self-report.** The extractor model sets it per instructions in the system prompt: `≥0.8` only if the paper gives explicit reagents/volumes/equipment, `~0.3` if the methods section is sparse. Uncalibrated. | Soft signal for *"is this a wet-lab paper with concrete procedure, or a sparse systems paper?"* Useful as a flag; don't treat as a trust percentage. |
| `code_repo.confidence` | 0–1 | **Varies by `detection_method`.** `papers-with-code`: fixed `0.95` (authoritative paper→repo API). `paper-text`: computed as `0.6 + 0.2·(strong-phrase-present) + 0.015·score_margin`, capped at `0.95`. `abstract-link`: fixed `0.85`. `none`: `0.0`. | Tells you **how** the repo was found and how decisively. High score + `paper-text` means a strong phrase like *"code is available at …"* sat next to the URL. |
| `repro_assessment.overall_score` | 0–1 | **Weighted sum of 8 binary signals**, all computed from the GitHub REST API (no clone, no execution): `has_readme` (0.10), `readme_substantial` (0.15), `has_dependencies_file` (0.20), `has_data_or_fixtures` (0.10), `has_notebook` (0.10), `has_figure_script` (0.20), `actively_maintained` (0.10), `permissive_license` (0.05). Each present signal contributes its weight. | The only fully-deterministic score of the three. Still a heuristic, not a proof — a high score means the repo **looks** well-structured for reproduction. For actual validation see [Paper2Agent](https://arxiv.org/abs/2509.06917). |
**Verdict buckets** (`repro_assessment.verdict`) are thresholds on `overall_score`:
| Verdict | Score | Meaning |
|---|---|---|
| `likely-reproducible` | ≥ 0.70 | Most repro-friendly signals present. Worth trying to run. |
| `partial` | ≥ 0.45 | Some infrastructure, likely gaps. Expect to fill in missing pieces. |
| `unlikely` | ≥ 0.20 | Minimal signal. Possible code dump without the scaffolding to rerun it. |
| `insufficient-info` | < 0.20 *or* repo unreachable | Not enough to tell. Don't draw conclusions either way. |
**Enum values you'll see in outputs:**
- `code_repo.detection_method`: `paper-text` | `abstract-link` | `papers-with-code` | `metadata` | `none`
- `metadata.source`: `arxiv` | `biorxiv` | `doi` | `url` | `unknown`
## Security & limitations
What this server *actually* does when you install and run it:
- **Network calls only to**: `export.arxiv.org`, `ar5iv.labs.arxiv.org`, `arxiv.org` (PDFs), `api.github.com`, `paperswithcode.com`, `api.anthropic.com`. No telemetry, no analytics, no phone-home.
- **Reads** `ANTHROPIC_API_KEY` (required for LLM tools) and optionally `GITHUB_TOKEN` from environment variables. These are sent only to Anthropic / GitHub respectively. Never logged, never persisted to disk.
- **Writes** nothing to your filesystem. No cache directories, no downloaded PDFs, no temp files.
- **Executes** no user-supplied code. No `eval`, `exec`, `subprocess`, `pickle.loads`, or shell-outs. The reproducibility tool deliberately *does not* clone or run repositories — it scores from the GitHub REST API only.
**Limitations to be aware of:**
- **Adversarial papers may produce misleading structured output.** The `extract_methods` tool sends paper text to Claude. A paper containing prompt-injection content could yield wrong (but schema-valid) structured methods. Treat the output as a research aid, not ground truth.
- **The reproducibility verdict is a heuristic, not a proof.** A high score means the repo *looks* well-structured for reproduction; it does not guarantee that running the code reproduces the paper. For full validation see [Paper2Agent](https://arxiv.org/abs/2509.06917).
- **Intended for local stdio use.** The HTTP/SSE transports are provided for development convenience but should only be exposed on trusted networks (no SSRF protection beyond what httpx provides).
**Reporting issues:**
Security issues: please email flynnlachendro@hotmail.co.uk (also see [`SECURITY.md`](SECURITY.md)). Functional bugs: open a [GitHub issue](https://github.com/FlynnLachendro/methods-mcp/issues).
## Pair with `paper-mcp`
For broader paper search / citation graph tooling, run [`paper-mcp` (Bhvaik)](https://pypi.org/project/paper-mcp/) alongside in the same Claude Code session. `paper-mcp` does title-keyed search, full-text fetch, citations, and references; `methods-mcp` adds the structured-methods + reproducibility layer on top. The two were intentionally designed to compose.
## Develop locally
```bash
git clone https://github.com/FlynnLachendro/methods-mcp
cd methods-mcp
uv sync --extra dev --extra agent
uv run pytest # 49 tests, offline (respx-mocked httpx + unittest.mock for Anthropic)
uv run ruff format .
uv run ruff check . --fix
uv run mypy src
uv run methods-mcp --help
```
## License
MIT — see [`LICENSE`](LICENSE).
## Acknowledgements
Built for the [Worldwide AI Science Fellowship](https://www.aisciencesummit.com/) inaugural cohort. Thanks to Michael Raspuzzi for the open-ended brief.
Built on:
- [FastMCP 3.x](https://github.com/jlowin/fastmcp) — the MCP server scaffold.
- [Claude Agent SDK](https://github.com/anthropics/claude-agent-sdk-python) — the agent loop in the demo.
- [ar5iv.labs.arxiv.org](https://ar5iv.labs.arxiv.org/) — clean HTML for arXiv papers.
- [Anthropic Claude](https://platform.claude.com) — the LLM behind structured extraction.