{
"markdown": "# NeuronScope\n\n\n\n\n[](https://github.com/RudrenduPaul/NeuronScope/actions/workflows/ci.yml)\n[](https://pypi.org/project/neuronscope-cli/)\n[](https://github.com/RudrenduPaul/NeuronScope/blob/main/LICENSE)\n\n![\"NeuronScope](\"https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1222875&theme=light&t=1786882007537\")
\n\nAsk a language model \"why did you say that\" and get back the actual attention heads and\nneurons responsible, as JSON, from the command line or from an agent over MCP.\n\n\n\n## Install\n\n```bash\npip install neuronscope-cli\n```\n\nThat gets you the `neuronscope` command. To install from source instead (for development or to\ntrack `main`):\n\n```bash\ngit clone https://github.com/RudrenduPaul/NeuronScope\ncd NeuronScope\npip install -e .\n```\n\n> [!NOTE]\n> The first run of any command downloads the requested model from the HuggingFace Hub\n> (`gpt2` is about 500MB) and prints two lines to stderr that are expected, not errors: a\n> CPU-fallback notice if you don't have a CUDA GPU, and an unauthenticated-HF-Hub\n> rate-limit notice. Neither one means anything broke.\n\n## Quickstart\n\n```bash\nneuronscope trace gpt2 \"The capital of France is Paris. The capital of Japan is\" --top-k 5\n```\n\nReal output from this exact command (stderr trimmed to the two expected warnings mentioned\nabove):\n\n```\nPrompt: The capital of France is Paris. The capital of Japan is\nPredicted next token: ' Tokyo'\nTop attention heads (by direct logit\n attribution)\n┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━┓\n┃ Layer ┃ Head ┃ Logit attribution ┃\n┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━┩\n│ 9 │ 8 │ 4.0679 │\n│ 8 │ 11 │ 2.9028 │\n│ 10 │ 7 │ -1.4782 │\n│ 8 │ 10 │ -1.3999 │\n│ 10 │ 0 │ 1.1424 │\n└───────┴──────┴───────────────────┘\nTop MLP neurons (by activation\n magnitude)\n┏━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┓\n┃ Layer ┃ Neuron ┃ Activation ┃\n┡━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━┩\n│ 10 │ 97 │ 7.8394 │\n│ 11 │ 611 │ 4.6954 │\n│ 11 │ 2997 │ 4.6468 │\n│ 10 │ 1793 │ 4.5443 │\n│ 9 │ 1460 │ 4.4196 │\n└───────┴────────┴────────────┘\n```\n\ngpt2 predicts `Tokyo` correctly, and head `L9H8` is the single biggest contributor to that\nprediction. Add `--json` to get the machine-readable version of the same result:\n\n```bash\nneuronscope trace gpt2 \"The capital of France is Paris. The capital of Japan is\" --top-k 3 --json\n```\n\n```json\n{\n \"schema_version\": 1,\n \"operation\": \"trace\",\n \"model\": {\n \"requested_name\": \"gpt2\",\n \"resolved_name\": \"gpt2\",\n \"backend\": \"transformer_lens\",\n \"device\": \"cpu\",\n \"n_layers\": 12,\n \"n_heads\": 12,\n \"d_model\": 768,\n \"d_mlp\": 3072\n },\n \"prompt\": \"The capital of France is Paris. The capital of Japan is\",\n \"predicted_token\": \" Tokyo\",\n \"predicted_token_id\": 11790,\n \"top_neurons\": [\n { \"layer\": 10, \"neuron_index\": 97, \"activation\": 7.839381217956543 },\n { \"layer\": 11, \"neuron_index\": 611, \"activation\": 4.695372581481934 },\n { \"layer\": 11, \"neuron_index\": 2997, \"activation\": 4.646785736083984 }\n ],\n \"top_heads\": [\n { \"layer\": 9, \"head_index\": 8, \"logit_attribution\": 4.067923545837402 },\n { \"layer\": 8, \"head_index\": 11, \"logit_attribution\": 2.9028172492980957 },\n { \"layer\": 10, \"head_index\": 7, \"logit_attribution\": -1.4781968593597412 }\n ]\n}\n```\n\n## What it does\n\nNeuronScope is a CLI and [MCP](https://modelcontextprotocol.io) server built on top of\n[TransformerLens](https://github.com/TransformerLensOrg/TransformerLens). TransformerLens\ndoes the actual model loading, hooking, and activation math; NeuronScope adds a stable CLI,\na versioned JSON schema, and an MCP server around it, so a script or an agent can ask \"which\ncomponents drove this output\" without writing TransformerLens code directly.\n\n- **`trace`**: runs a prompt through the model and ranks attention heads by direct logit\n attribution to the predicted token, and MLP neurons by activation magnitude at the final\n prompt position.\n- **`activations`**: dumps shape, mean, std, min/max, and the max-activating sequence\n position for every layer's residual stream, MLP neuron activations, and attention pattern.\n- **`patch`**: zero-ablates one component (`resid_pre`, `resid_mid`, `resid_post`,\n `attn_out`, `mlp_out`, or `mlp_post`) at a given layer and reports how the predicted token\n and its logit changed.\n- **`circuit`**: a best-effort automated circuit sketch. Ranks candidate heads/neurons by\n logit attribution, then measures each one's individual causal effect via single-component\n ablation. This is not full path-patching with clean/corrupted prompt pairs and does not\n capture interaction effects between components. The `--json` output says so explicitly in\n its `method` field.\n- Every command supports `--json` for a `schema_version`-stamped document instead of a\n table, and the same four operations are exposed as MCP tools returning the identical\n shape via `.model_dump()`, so a CLI call and an MCP tool call produce the same document\n for the same input.\n- Model support is whatever `transformer_lens.HookedTransformer.from_pretrained` supports.\n Installing `neuronscope-cli` today pulls TransformerLens 3.6.0, which supports 249\n pretrained checkpoints and aliases (`OFFICIAL_MODEL_NAMES`), covering GPT-2, Pythia,\n Llama, Gemma, Qwen, and more. Small models like `gpt2` run comfortably on CPU.\n\nNeuronScope does not replace TransformerLens, [nnsight](https://nnsight.net/),\n[SAELens](https://github.com/jbloomAus/SAELens), Anthropic's\n[circuit-tracer](https://github.com/decoderesearch/circuit-tracer), or\n[Neuronpedia](https://www.neuronpedia.org/). It wraps TransformerLens for one narrower job:\nfast, scriptable, agent-callable component tracing on a single prompt. It leaves deeper\nmechanistic work (SAE training, transcoder-based circuit graphs, hosted feature browsing) to\nthose tools.\n\n## CLI reference\n\nEvery command takes `MODEL` (any name `HookedTransformer.from_pretrained` accepts, for\nexample `gpt2` or `EleutherAI/pythia-70m`) and `PROMPT` as positional arguments.\n\n| Command | Extra flags | What it does |\n|---|---|---|\n| `neuronscope trace MODEL PROMPT` | `--top-k INTEGER` (default 10), `--json` | Ranks top attention heads (logit attribution) and MLP neurons (activation magnitude) for the predicted next token |\n| `neuronscope activations MODEL PROMPT` | `--json` | Dumps per-layer activation summary stats (residual stream, MLP, attention pattern) |\n| `neuronscope patch MODEL PROMPT` | `--layer INTEGER` (required), `--component [resid_pre\\|resid_mid\\|resid_post\\|attn_out\\|mlp_out\\|mlp_post]` (required), `--json` | Zero-ablates one component and reports the logit/prediction delta |\n| `neuronscope circuit MODEL PROMPT` | `--top-k INTEGER` (default 10), `--json` | Best-effort circuit sketch via ranked single-component ablation |\n| `neuronscope mcp-server` | none | Starts the MCP server over stdio |\n\nGlobal: `neuronscope --version`, `neuronscope --help`. Exit codes: `0` success,\n`1` a runtime error (prompt too long for the model's context window, `--layer` out of range,\netc.), `2` a Click usage error (bad flags), `3` an unsupported model name.\n\n\n\n\n\n## MCP Server\n\nNeuronScope ships a [Model Context Protocol](https://modelcontextprotocol.io) server so an AI\nagent (Claude, Cursor, or any MCP-compatible client) can trace, inspect, ablate, and sketch\ncircuits directly, without a human invoking the CLI by hand.\n\nInstall the extra:\n\n```bash\npip install \"neuronscope-cli[mcp]\"\n```\n\nAdd it to your MCP client's config (for Claude Desktop, `claude_desktop_config.json`):\n\n```json\n{\n \"mcpServers\": {\n \"neuronscope\": {\n \"command\": \"uvx\",\n \"args\": [\"--from\", \"neuronscope-cli\", \"neuronscope-mcp\"]\n }\n }\n}\n```\n\nThe server exposes four tools, `trace`, `activations`, `patch`, and `circuit`, each returning\nthe identical pydantic-model-shaped JSON the CLI's `--json` flag prints, via `.model_dump()`, so\nan agent calling this server and a script calling the CLI get the same document for the same\ninput. A real `trace` call and its response:\n\n```\ntrace(model=\"gpt2\", prompt=\"The capital of France is Paris. The capital of Japan is\", top_k=3)\n\n{\n \"schema_version\": 1,\n \"operation\": \"trace\",\n \"predicted_token\": \" Tokyo\",\n \"predicted_token_id\": 11790,\n \"top_neurons\": [\n { \"layer\": 10, \"neuron_index\": 97, \"activation\": 7.839381217956543 }\n ],\n \"top_heads\": [\n { \"layer\": 9, \"head_index\": 8, \"logit_attribution\": 4.067923545837402 }\n ]\n}\n```\n\nErrors never raise across the tool boundary: every handler catches its exceptions and returns a\nstructured `ErrorResponse` dict instead, so a calling agent always gets a parseable result.\n\n> [!WARNING]\n> NeuronScope caps model size (2B parameters by default, `NEURONSCOPE_MAX_MODEL_PARAMS`)\n> and how many models can load at once (1 by default, `NEURONSCOPE_MAX_CONCURRENT_LOADS`),\n> but puts no timeout on model loading or forward passes. If you expose this MCP server\n> somewhere an untrusted agent can call it, still put a resource limit around the process\n> (a cgroup, `ulimit`, or a container memory/CPU cap) as defense in depth rather than\n> relying on these in-process caps alone.\n\nTransport is stdio, so there is nothing to host: the MCP client spawns the server as a local\nsubprocess. Source: [`neuronscope/mcp_server.py`](neuronscope/mcp_server.py).\n\n- **Claude Code** reads this from a project-level `.mcp.json` in your repo root, or you can\n add it with `claude mcp add neuronscope -- neuronscope-mcp`.\n- **Claude Desktop** reads this from its `claude_desktop_config.json`\n (`~/Library/Application Support/Claude/claude_desktop_config.json` on macOS,\n `%APPDATA%\\Claude\\claude_desktop_config.json` on Windows), under the same\n `\"mcpServers\"` key.\n- The `neuronscope mcp-server` CLI subcommand still works as a local, non-`uvx` alternative\n that runs the same server over stdio from an existing install.\n\n## How it compares\n\nAll five of these are real, actively maintained projects doing different jobs. This table\ncompares CLI/JSON-agent-output surface and model coverage, not depth of interpretability\nresearch, where TransformerLens, nnsight, SAELens, circuit-tracer, and Neuronpedia are all\nmore mature than NeuronScope. Star counts, release info, and last-push dates below were\npulled from each project's GitHub API on 2026-08-03 and will drift over time; check the repos\ndirectly for current numbers.\n\n| Project | Stars | Last activity | CLI | Agent-callable structured output | Model coverage |\n|---|---|---|---|---|---|\n| [TransformerLens](https://github.com/TransformerLensOrg/TransformerLens) | 3,750 | v3.6.0 released 2026-07-28, pushed 2026-08-03 | No (Python library) | No | 249 pretrained checkpoints/aliases (its own official list) |\n| [nnsight](https://github.com/ndif-team/nnsight) | 1,014 | v0.7.0 released 2026-05-05, pushed 2026-07-30 | No (Python library) | No (returns tensors/Python objects) | Any HuggingFace or PyTorch model generically, no fixed list |\n| [circuit-tracer](https://github.com/decoderesearch/circuit-tracer) (Anthropic-authored, moved from `safety-research/circuit-tracer`) | 2,882 | v0.5.2 released 2026-07-18, pushed 2026-07-18 | Yes | JSON attribution-graph export; no MCP server | Fixed transcoder allowlist: Gemma-2 (2B), Gemma-3 (270M-27B), Llama-3.2 (1B), Llama-3.1 (8B Instruct), Qwen-3 (0.6B-14B), GPT-OSS (20B) |\n| [SAELens](https://github.com/jbloomAus/SAELens) | 1,492 | v6.47.0 released 2026-07-28, pushed 2026-07-28 | No (Python library) | No | Any PyTorch model generically; deepest integration is with TransformerLens |\n| [Neuronpedia](https://github.com/hijohnnylin/neuronpedia) | 1,093 | continuously deployed, tag v1.0.795 | No (hosted web app + REST API) | REST API returns JSON; MCP access exists only via an unofficial third-party wrapper, not the official repo | Models loadable through TransformerLens's model table (GPT-2, Gemma-2, Llama, DeepSeek, etc.) |\n| **NeuronScope** (this project) | 1 | this commit | Yes | Yes: `--json` on every command, plus a native MCP server returning the same schema | Whatever TransformerLens's `HookedTransformer.from_pretrained` supports: 249 checkpoints/aliases |\n\nThe honest differentiation is narrow: NeuronScope is the only one of these with a CLI, a\nnative MCP server, and a versioned JSON schema together in one package, and it's\nmodel-agnostic across whatever TransformerLens supports rather than pinned to a fixed\ntranscoder allowlist like circuit-tracer. It is not more capable, more mature, or more\nwidely used than any of these projects.\n\n## What is NeuronScope and why does it exist\n\nTransformerLens gives you a Python API for loading a model and running hooked forward\npasses. That's the right interface for a research notebook. It's the wrong interface for a\nscript that needs a subprocess call and a JSON document back, or for an agent that needs a\ntool it can call over MCP. NeuronScope exists to be that second interface: the same\nunderlying computation, wrapped so a CLI invocation or an MCP tool call gets back a\nschema-versioned document instead of a Python object graph.\n\n## FAQ\n\n**Is this a replacement for TransformerLens, nnsight, SAELens, circuit-tracer, or\nNeuronpedia?**\nNo. NeuronScope is built directly on TransformerLens and does not do anything TransformerLens\nitself can't already do at a lower level. It doesn't train SAEs (SAELens), do full\npath-patching circuit discovery with transcoders (circuit-tracer), give you a Python-native\ntracing context manager for arbitrary PyTorch models (nnsight), or host a browsable feature\ndatabase (Neuronpedia). It's a CLI and MCP wrapper around one slice of TransformerLens's\nfunctionality.\n\n**What models are supported?**\nAnything `transformer_lens.HookedTransformer.from_pretrained` supports, which today is 249\ncheckpoints and aliases spanning GPT-2, Pythia, Llama, Gemma, Qwen, and others. Run\n`python -c \"from transformer_lens.loading_from_pretrained import OFFICIAL_MODEL_NAMES; print(len(OFFICIAL_MODEL_NAMES))\"`\nin your own environment to get the exact count for your installed version, since\nTransformerLens adds models over time.\n\n**Does it need a GPU?**\nNo. Small models like `gpt2` run fine on CPU; that's what the test suite and the quickstart\nabove run on. Larger models will be slow on CPU. NeuronScope does not auto-select Apple\nSilicon's MPS backend even when available, because PyTorch's MPS backend can silently\nproduce incorrect values for some ops that this project's activation-patching math depends\non being exact. Pass `device=\"mps\"` explicitly in your own code if you want it anyway.\n\n**Is it safe to expose the MCP server to an untrusted agent?**\nOnly with resource limits in place. See Known limitations below.\n\n**How is NeuronScope different from circuit-tracer, the other CLI tool in this list?**\ncircuit-tracer does deeper circuit analysis (full attribution graphs from trained\ntranscoders) but only for a fixed allowlist of models: Gemma-2, Gemma-3, Llama-3.1/3.2,\nQwen-3, and GPT-OSS. NeuronScope trades that depth for breadth: it works with any of\nTransformerLens's 249 supported checkpoints with no transcoder training step, and ships an\nMCP server so an agent can call it directly. The cost is that NeuronScope does\nsingle-component logit attribution and zero-ablation, not transcoder-based path patching.\n\n**Does the installed version always match what's on PyPI?**\nRun `neuronscope --version` after installing to check. `pip install neuronscope-cli` pulls\nwhatever release PyPI has published most recently; the code on this repo's `main` branch can\nbe ahead of that between releases. Installing from source (`pip install -e .`) always tracks\n`main` exactly, including whatever hasn't been released yet.\n\n**What license is NeuronScope under, and can I use it commercially?**\nMIT. You can use, modify, and redistribute it in commercial and closed-source projects,\nwith attribution and the license notice kept intact. The dependencies it pulls in\n(TransformerLens, PyTorch, the `mcp` package) carry their own licenses; check those\nseparately if you're redistributing a bundled product rather than just calling\n`neuronscope-cli` as a dependency.\n\n## Known limitations\n\n- **`circuit` is an approximation.** It ranks components by logit attribution and measures\n each one's individual causal effect via single-component zero-ablation on one prompt. It\n does not do full path-patching with clean/corrupted prompt pairs, and it will not catch\n interaction effects between components. The `--json` output states this in its `method`\n field so a caller doesn't have to trust prose to know the caveat.\n- **No timeout on model loading or forward passes.** Once a request passes the resource\n caps below, NeuronScope runs the load and the forward pass to completion with no\n built-in wall-clock limit. If you run the MCP server somewhere an untrusted agent can\n call it, put a resource limit around the process (a cgroup, `ulimit`, or a container\n memory/CPU cap) as defense in depth.\n- **Model size and load concurrency are capped, but only in-process.** `neuronscope/core/limits.py`\n rejects a model over `NEURONSCOPE_MAX_MODEL_PARAMS` (2B parameters by default) before any\n weights are downloaded, and rejects a load once `NEURONSCOPE_MAX_CONCURRENT_LOADS` (1 by\n default) other loads are already in flight, both with a structured error rather than a\n hang or a crash. The size check is best-effort: if a model's parameter count can't be\n determined (for example, fully offline with nothing cached yet), it fails open rather\n than blocking a legitimate request, so it's not a hard guarantee on its own -- pair it\n with a process-level resource limit for untrusted deployments.\n- **`HookedTransformer.from_pretrained` is deprecated upstream.** TransformerLens 3.6.0\n emits a `DeprecationWarning` pointing at `TransformerBridge.boot_transformers` as the\n replacement. It still works today, and every command shown in this README ran on it, but\n NeuronScope's backend hasn't migrated yet. Tracked as an open item; migrating would be a\n change inside `neuronscope/backends/transformer_lens.py`, not a change to any CLI command\n or MCP tool signature.\n\n## Contributing\n\nIssues and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for dev setup,\nwhere the code lives, and what a PR needs before it merges. Quick version:\n\n```bash\npip install -e \".[dev,mcp]\"\npytest -v\n```\n\nCI runs the same suite on Python 3.10, 3.11, and 3.12 on every push and pull request against\n`main`. The suite covers 87% of `neuronscope/` (`pytest --cov=neuronscope`), with the MCP\nserver's less-exercised paths (specific error branches) the main gap.\n\n## License\n\nMIT. See [LICENSE](LICENSE).\n",
"bytes": 20040,
"sha": "5e4565e5c0ded707bb1912614315a7a0305ea57c6539b41cab803446e570da49",
"repo_slug": "rudrendupaul/neuronscope",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_rudrendupaul_neuronscope_d55b5589/readme"
}