{
  "markdown": "# BGI - Big Indexer\n\n[![ahmedxuhri/bigindexer MCP server](https://glama.ai/mcp/servers/ahmedxuhri/bigindexer/badges/score.svg)](https://glama.ai/mcp/servers/ahmedxuhri/bigindexer)\n[![PyPI version](https://img.shields.io/pypi/v/bigindexer.svg)](https://pypi.org/project/bigindexer/)\n[![License](https://img.shields.io/github/license/ahmedxuhri/bigindexer.svg)](https://github.com/ahmedxuhri/bigindexer/blob/master/LICENSE)\n[![Tests](https://img.shields.io/badge/Tests-Passing-brightgreen)](https://bigindexer.com/validation)\n[![Interactive Demo](https://img.shields.io/badge/Demo-Interactive%20Sandbox-blueviolet)](https://bigindexer.com/demo)\n[![MCP Registry](https://img.shields.io/badge/MCP-Registry-blue)](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.ahmedxuhri/bigindexer)\n[![PR Risk Bot Action](https://img.shields.io/badge/PR%20Risk%20Bot-Action-blueviolet)](https://github.com/ahmedxuhri/bigindexer-pr-risk-bot)\n\n<!-- mcp-name: io.github.ahmedxuhri/bigindexer -->\n\nBGI is a static architecture analysis tool for large codebases.\nIt groups code units by **behavioral role** and emits explicit architectural boundaries.\nProject domain: `bigindexer.com`\n\n## Use via MCP Registry\n\nBig Indexer is published in the MCP Registry as `io.github.ahmedxuhri/bigindexer`.\n\n```bash\npip install bigindexer==0.1.3\nbgi mcp --graph bgi-graph.json --fuse-graph fuse-graph.json\n```\n\nValidation: https://bigindexer.com/validation\n\n## What problem this solves\n\nMost architecture graphs fail at scale in two ways:\n\n- too many noisy edges\n- giant clusters that collapse unrelated components together\n\nBGI is built to keep both under control, so the output remains usable on large repos.\n\n## What you can do with it\n\n1. **\"Where should this boundary be before we refactor?\"**  \n   BGI groups units by behavioral role (COV tokens + DRS clustering) so likely component boundaries are visible.\n2. **\"Which subsystem coupling is risky?\"**  \n   BGI surfaces high-coupling seams and fuse-boundary signals between clusters so integration risk is easier to spot.\n3. **\"How do we plug architecture data into automation?\"**  \n   BGI emits machine-readable artifacts (`bgi-graph.json`, `fuse-graph.json`) plus optional human context (`bigindexer.md`).\n4. **\"How do we make AI changes less random?\"**  \n   MCP tools (`task_fingerprint`, `behavioral_twins`, `twin_context`) ground prompts in in-repo behavior patterns.\n5. **\"Can I run this automatically on PRs as a live example?\"**  \n   Yes — use the dedicated action repo [`ahmedxuhri/bigindexer-pr-risk-bot`](https://github.com/ahmedxuhri/bigindexer-pr-risk-bot) to auto-comment PRs with blast radius, seams, and risk hints.\n\n---\n\n## 30-second demo\n\nRun BGI on the included fixture repo:\n\n```bash\ngit clone https://github.com/ahmedxuhri/bigindexer\ncd bigindexer\npip install -e .\nbgi scan tests/fixtures --lang python --out /tmp/bgi-example.json\nhead -50 /tmp/bgi-example.json\n```\n\nObserved result on this repository:\n\n- units: `12`\n- edges: `14`\n- clusters: `2`\n- max cluster in sample: `6` units\n\nOne produced edge looks like:\n\n```json\n{\n  \"source\": \"auth_module.py::AuthService::__init__\",\n  \"target\": \"auth_module.py::AuthService::__del__\",\n  \"key\": \"COV.INIT\",\n  \"lock\": \"COV.TEARDOWN\",\n  \"type\": \"HARD\"\n}\n```\n\nWhy this matters: instead of raw syntax references only, you get behavioral relationships plus cluster structure that can drive architecture decisions.\n\n---\n\n## Plain-English glossary\n\n| BGI term | Plain meaning |\n|---|---|\n| **COV token** | A behavior label for a unit (for example: `FETCH`, `PERSIST`, `AUTHENTICATE`) |\n| **Key-Lock edge** | A behavioral connection between two units with complementary roles |\n| **DRS cluster** | A unit-level grouping by behavioral role. Mostly intra-file in practice. File-level architectural components are better expressed via the BGI edge graph or the fuse-graph boundary signal — see [external benchmark](docs/VALIDATION_EVIDENCE.md#external-benchmark-vs-louvain) |\n| **Fuse edge / fuse event** | A refused merge because cluster growth hit the cap; treated as boundary signal |\n| **Spectral masks** | Scope rules that limit where matching is allowed (global, directory, file) |\n\n---\n\n## Architecture in one view\n\n```text\nSource files\n   ->\nGate 1: fingerprint unit behavior (COV tokens)\n   ->\nGate 2: create behavioral edges with scoped matching\n   ->\nGate 3: cluster with hard size cap + boundary emission\n   ->\nArtifacts: bgi-graph.json, fuse-graph.json, bigindexer.md, optional routes/graphml/html\n```\n\nCore approach:\n\n1. **TOKEN-CENSUS** - classify token frequency per repo.\n2. **SPECTRAL-MASKS** - restrict match scope by token frequency.\n3. **FUSE-MAP** - cap cluster growth and record refused merges.\n4. **MASK-4-GATE-3** - use import proximity as clustering signal.\n5. **WATER-CLOCK + `.scm`** - single-pass query extraction path in Gate 1.\n\n---\n\n## Why BGI is different from common alternatives\n\n| Capability | LSP / SCIP index | Call-graph + generic community detection | BGI |\n|---|---|---|---|\n| Fast symbol lookup | Strong | Medium | Available (Phase 6 index) |\n| Behavioral token model | No | Usually no | **Yes** |\n| Hard-bounded clustering | No | Usually no | **Yes** (unit-level) |\n| First-class boundary artifact | No | Usually no | **Yes (`fuse-graph.json`)** |\n| Scope-constrained edge generation | Limited | Rare | **Yes (spectral masks)** |\n\nExternal head-to-head benchmark (Louvain on BGI's edges vs Louvain on raw imports, scored against package layout): BGI's edges win on Python (django F1 0.38 vs 0.29, MoJoFM 0.45 vs 0.34) and currently tie/lose on Go due to lower cross-file edge density on tier-2 scanners. Full results and methodology in [docs/VALIDATION_EVIDENCE.md](docs/VALIDATION_EVIDENCE.md#external-benchmark-vs-louvain).\n\n---\n\n## Evidence (current, verifiable)\n\n### Large-repo scale evidence\n\nComparable kubernetes sample (`go` comparable mode, 162,917 units):\n\n- Gate 1: `141.964s`\n- Gate 2: `67.261s` (historical comparable baseline: `138.869s`)\n- Gate 3: `9.359s`\n- Total: `218.584s`\n- Max cluster: `1.113%`\n- Fuse events: `0`\n\nArtifact: `output/validation/kubernetes-optionb-controlled-median-v21.json`\n\n### Quality guard evidence (beyond raw speed)\n\n- Gate 2 scope safety tests block invalid cross-scope merges (see `tests/test_gate2.py`).\n- Gate 3 tests verify no legacy namespace over-merge without import evidence (see `tests/test_gate3.py`).\n- Current full suite status: `python3 -m pytest tests/ -x -q` (project baseline target remains passing).\n\n### Evidence summary\n\n- Current published validation set: **100 scored runs** across 5 repos and 3 models.\n- Full 20-run post-shipment benchmark refresh for BGI-TWIN context (`task → COV → top-3 twins + seam + rubric`) is complete: actionability **4.75/5** (p04 slice: **4.8/5**), boundary **1.0**, hallucinations **0**.\n- Independent-model replication is now complete on **azure/gpt-4o** (20 runs) and **gemini/auto** (20 runs): GPT-4o actionability **4.85/5**, Gemini actionability **4.25/5**, both with zero hallucinations; Gemini boundary **0.95** reflects one genuine `django/p02` miss.\n- Still missing: labeled precision/recall benchmark on an external corpus and head-to-head quantitative benchmark vs external tools on the same labeled dataset.\n\n---\n\n## Language support tiers (explicit)\n\nBGI does not treat all languages equally; support is tiered:\n\n1. **Query-backed (`.scm`)**: `python`, `typescript`, `tsx`, `javascript`, `go`, `rust`, `java`, `csharp`, `php`, `ruby`, `kotlin`, `scala`\n2. **Tree-sitter scanner + rule path**: `c`, `lua`, `elixir`\n3. **Generic regex fallback by extension**: `swift`, `r`, `dart`, `bash`, `nim`, `zig`, `haskell`, `ocaml`, `fsharp`, `clojure`, `erlang`, `matlab`, `vb`, `crystal`, `cobol`, `groovy`\n\nUse this as a reliability signal: query-backed and dedicated scanner tiers are stronger than generic fallback.\n\n**Cross-file edge density caveat:** the language tiers above describe parser quality. A separate axis is *cross-file behavioral edge density* — how many key-lock pairs the scanner produces that link units in different files. Tier-1 (`.scm`-backed) languages produce dense cross-file edges. Tier-2 scanner-backed languages currently produce sparser cross-file edges because their token mix is dominated by structural tokens (INTAKE/OUTPUT/CONDITIONAL/LOOP) that gate-2 deliberately scopes to same-file to prevent O(N²) noise. The user-visible MCP product (boundary detection, twin retrieval, AI-assistant context) still works on tier-2 languages — see the validation evidence — but cluster-recovery benchmarks against import-graph baselines reflect this density gap. Concrete numbers in [docs/VALIDATION_EVIDENCE.md](docs/VALIDATION_EVIDENCE.md#external-benchmark-vs-louvain).\n\n---\n\n## Limitations and non-goals\n\n1. BGI is **static analysis**; it does not ingest runtime traces.\n2. Cross-file semantic resolution is heuristic and language-dependent.\n3. Cluster-size health is measured; full external precision/recall is not yet published.\n4. Shared-host benchmarking introduces variance; decisions should use controlled medians.\n\n---\n\n## Install\n\n```bash\npip install -e .\n```\n\n## Quickstart commands\n\n```bash\n# scan\nbgi scan /path/to/repo --lang auto --out bgi-graph.json\n\n# optional outputs\nbgi scan /path/to/repo --lang auto \\\n  --fuse-graph fuse-graph.json \\\n  --routes routes.json \\\n  --graphml graph.graphml \\\n  --html\n\n# incremental\nbgi scan /path/to/repo --lang auto --incremental --cache .bgi-cache.json\n\n# diff\nbgi diff /path/before /path/after --lang auto --out diff.json\n\n# run MCP server over generated artifacts\nbgi mcp --graph bgi-graph.json --fuse-graph fuse-graph.json\n```\n\nExample MCP usage pattern (from your client prompt):\n\n```text\nUse MCP tool twin_context for:\n\"Add endpoint that validates input and persists data.\"\nReturn top twin candidate, seam suggestion, and rubric checklist.\n```\n\n---\n\n## Telemetry\n\nBGI ships with **opt-in, off-by-default** anonymous telemetry. To enable:\n\n```bash\nexport BGI_TELEMETRY=1\nbgi mcp --graph bgi-graph.json --fuse-graph fuse-graph.json\n```\n\nWhat's collected when enabled: BGI version, OS, repo size bucket, and a 12-char hash of your repo's git remote (so we can deduplicate \"same repo seen twice\" without ever knowing which repo). What's never collected: file paths, source code, repo names, user identity, or IP addresses. Full schema and disable instructions in [`docs/TELEMETRY.md`](docs/TELEMETRY.md).\n\n---\n\n## Documentation map\n\n- `MEMORANDUM.md` - design contracts and invariants\n- `docs/LANGUAGE_SUPPORT.md` - language implementation details\n- `docs/CONTRIBUTING_LANGUAGES.md` - language contribution guide\n- `docs/INDEX_SCHEMA.md` - interactive index schema\n- `docs/QUERY_PLANNER.md` - query planner scoring\n- `docs/MCP_SETUP.md` - MCP server setup and usage\n- `docs/MCP_WITH_CONTINUE.md` - 5-minute Continue + BGI walkthrough\n- `docs/TELEMETRY.md` - opt-in telemetry: what we collect and how to disable\n- `https://bigindexer.com/validation` - public validation evidence\n- `docs/MCP_QUICKSTART_DEMO.md` - 5-minute demo walkthrough\n- `docs/MCP_EXAMPLE_TRANSCRIPTS.md` - real-world MCP tool invocation examples\n- `docs/MCP_REAL_TRANSCRIPT.md` - unedited transcript from FastAPI analysis\n- `scripts/mcp-demo.sh` - automated demo script for multiple CLIs and repositories\n\n## License and Copyright\n\n- License: Apache License 2.0 (`LICENSE`)\n- Contributor terms: Developer Certificate of Origin (`DCO`) enforced on pull requests\n\n## Legal\n\nBig Indexer is a product of **SUDO SOFTWARES LTD**, registered in England and Wales (Company No. 16976948).\nContact: [ahmed@bigindexer.com](mailto:ahmed@bigindexer.com) · https://bigindexer.com\n",
  "bytes": 11653,
  "sha": "896dcf683dfa716ae0d29f08716717b843224bc81f5b388309e22d7238fac79e",
  "repo_slug": "ahmedxuhri/bigindexer",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_ahmedxuhri_bigindexer_1a67e58b/readme"
}