{
  "markdown": "# fittok\n\n**Retrieve only the relevant source code for a question — instead of the model\nreading whole files — so an LLM answers codebase questions on a small, focused\nslice of context.** Less input = fewer tokens, lower cost, faster answers.\n\nWorks three ways from one install: an **MCP server**, a **CLI**, and a **Python\nlibrary** — plus a **Claude Code plugin** that injects context automatically.\n\n📖 **[Full command reference → docs/HANDBOOK.md](https://github.com/likhithreddy/fittok/blob/main/docs/HANDBOOK.md)**\n\n<!-- MCP Registry: PyPI ownership verification — do not remove this line -->\nmcp-name: io.github.likhithreddy/fittok\n\n---\n\n## How it works\n\n```\ncodebase ──▶ graphify ──▶ slurp ──▶ readable slice ──▶ LLM answers\n             (parse)      (select)   (trim to budget)\n```\n\n1. **graphify** — parses the repo with tree-sitter into a knowledge graph of\n   functions / classes / methods (Python, JS, JSX, TS, TSX, Java, Go, Rust).\n   Supports multi-language call/import/reference edges.\n2. **slurp** — scores every node against the question using a **4-signal hybrid**:\n   - **Semantic embeddings** (all-MiniLM-L6-v2) — meaning-based matching\n   - **Content-BM25** (with camelCase/snake_case splitting) — keyword matching\n   - **Summary-BM25** (node name + file + callers + callees) — structural matching\n   - **PageRank** — graph centrality / hub detection\n\n   Signals are fused via **Reciprocal Rank Fusion (RRF)** — rank-based, no score\n   calibration issues. Nodes are selected via **round-robin directory interleaving**\n   (guarantees facet coverage on multi-aspect queries — one node from each code\n   area before any gets a second) with a **per-node token cap** (25% of budget, so\n   large components don't crowd out smaller functions). A **relevance cliff**\n   (semantic OR BM25 OR summary-BM25 threshold) excludes noise.\n3. **readable output** — returns the **actual source code** of selected nodes,\n   plus a **codebase map** (table of contents with docstrings, inspired by\n   Karpathy's LLM Wiki / Google's OKF) so the model can route follow-up calls\n   precisely. The model answers directly from it — no file reads needed.\n\nAs you edit, a file watcher (auto-started on first query) updates the graph\n**incrementally** — only changed files are re-parsed and merged, and only\nchanged functions re-embed. Graphs and embeddings are cached on disk\n(`~/.cache/fittok`). Set `FITTOK_AUTOWATCH=false` to disable the watcher, in\nwhich case an edit triggers a full re-parse on the next query.\n\n---\n\n## Getting the best results (and known limitations)\n\n### Ask focused questions\n\nfittok ranks code against your question using a **4-signal hybrid** (semantic +\nBM25 + structural + PageRank, fused via RRF) with **round-robin directory\ndiversity** — so multi-facet questions surface code from multiple areas (UI,\nserver, database) instead of clustering in one dominant area. It's most accurate\nwith **focused, specific questions** — ideally one concern each, and naming the\nfunction/component/route when you can. Multi-facet questions are supported via\n**decomposition** (the tool description tells the model to call once per aspect)\nand the **codebase map** (a table of contents prepended to every response).\n\n- ✅ *\"How does `runSandboxQuery` execute and isolate a SQL query?\"* → surfaces the exact function + its isolation code.\n- ✅ *\"How does the querydle client submit a query and render results?\"* → surfaces the UI component.\n- ✅ *\"Trace the full lifecycle: UI submission, sandbox execution, data isolation\"* → decomposition + round-robin diversity covers multiple facets; the codebase map routes the model to any missed files.\n\n**Rule of thumb:** one concern per question (or 2–3 facets max). For \"explain\nthe whole feature,\" split it into a few focused questions instead of one mega-query.\n\n### Known limitations\n\n- **GitHub Copilot Chat truncates large MCP outputs (the big one).** Copilot caches MCP tool results above ~7 KB to a `content.json` file, where the entire markdown collapses to ONE physical JSON line (newlines escaped) — and its Read tool truncates any line at ~2,000 characters. So an output over ~7 KB is effectively chopped to ~2,000 chars *regardless of total size*; the model can't see most of the code and falls back to reading source files directly. This is Copilot's delivery layer, not fittok — **every MCP server hits this wall.** By default (0.10.0+) fittok returns *all* relevant code uncapped, which is correct for clients that deliver inline but will be truncated by Copilot. Two workarounds:\n  - **Cap the output for Copilot** so it's delivered inline (under the ~7 KB threshold). Set `FITTOK_MAX_BUDGET=1200` in your MCP server's env:\n    ```json\n    { \"servers\": { \"fittok\": { \"command\": \"uvx\", \"args\": [\"fittok\"], \"env\": { \"FITTOK_MAX_BUDGET\": \"1200\" } } } }\n    ```\n  - **Use Claude Code or the CLI** for multi-file questions. They deliver MCP output inline with no truncation — which is where fittok's complete (uncapped) results and anti-re-read design actually pay off.\n- **Vocabulary gap on abstract queries.** When the query uses words that don't appear in the code (e.g. \"isolation\" → `REVOKE`/`DENY`), neither semantic nor BM25 can bridge it. The codebase map (file names + docstrings) and round-robin diversity help; naming the function/file routes the model to it.\n- **Incremental edge-loss:** editing a file can drop call/import edges *into* it from unchanged files until a full re-parse. fittok auto-recovers on restart or `reset_graph`.\n- **Token counts are approximate:** counts use `cl100k_base`, so real usage drifts ~10–20% vs Claude's tokenizer (only matters when you opt into a `FITTOK_MAX_BUDGET` cap).\n- **Very large repos:** PageRank is not yet vectorized — fine through low-thousands of nodes, slower beyond that.\n\n---\n\n## Installation\n\nfittok ships as an **MCP server**, a **CLI**, and a **Python library**. It uses\n`torch` for embeddings, so a Python runtime must be present. **Pick one runtime**\nbelow, then follow the section for your client.\n\n> Every config below launches fittok as `uvx fittok`. If you chose Python or\n> `pipx`, swap that for `python -m fittok` or `pipx run fittok` respectively.\n\n### Prerequisites — choose a runtime (one of)\n\n**A. `uv` — recommended (no Python needed on the machine)**\n\n```bash\ncurl -LsSf https://astral.sh/uv/install.sh | sh   # Linux / macOS\nwinget install astral-sh.uv                        # Windows\nbrew install uv                                    # macOS (Homebrew)\n```\nLaunch command: `uvx fittok` — `uv` provisions its own Python + all deps in\nisolation. One static binary, so it's deployable org-wide via MDM/Intune/winget.\n\n**B. Python 3.10+ (already on the machine)**\n\n```bash\npython -m pip install fittok      # Linux / macOS\npy    -m pip install fittok       # Windows\n```\nLaunch command: `python -m fittok` (Windows: `py -m fittok`).\n> Managed Linux may reject `pip install` with PEP 668\n> (\"externally-managed-environment\") — use option A to avoid it.\n\n**C. `pipx` — isolated, no global install**\n\n```bash\nbrew install pipx                               # macOS\npip install --user pipx && pipx ensurepath      # Linux / Windows\n```\nLaunch command: `pipx run fittok`.\n\n### MCP server — Claude Code\n\n```bash\nclaude mcp add fittok -s user -- uvx fittok\n```\nRestart Claude Code → `/mcp` → confirm `fittok` is **connected**, then ask\ncodebase questions normally.\n\n### MCP server — VS Code / GitHub Copilot Chat\n\n```bash\ncode --add-mcp '{\"name\":\"fittok\",\"command\":\"uvx\",\"args\":[\"fittok\"]}'\n```\nOr paste into `.vscode/mcp.json` (workspace) or your user `mcp.json`:\n```json\n{ \"servers\": { \"fittok\": { \"type\": \"stdio\", \"command\": \"uvx\", \"args\": [\"fittok\"] } } }\n```\nThen in Copilot Chat: **Agent** mode → enable fittok's tools (*Configure Tools*).\n\n### MCP server — GitHub Copilot CLI\n\n```bash\ncopilot mcp add fittok -- uvx fittok\ncopilot mcp get fittok          # verify status + tools\n```\n\n### MCP server — Cursor / Windsurf / any MCP client\n\n```json\n{ \"mcpServers\": { \"fittok\": { \"command\": \"uvx\", \"args\": [\"fittok\"] } } }\n```\n\n### Auto-trigger (optional, every MCP client)\n\nTo make fittok fire on **every** codebase question — without naming it — **and**\nstop your client from re-reading files fittok already returned (which would\ndiscard the savings), add this one line to your client's instructions file:\n\n> *\"For any codebase question, call fittok first and answer from its output —\n> don't re-read files it already returned code from.\"*\n\nThe first half triggers fittok; the second keeps the client from opening the\nsame files afterward. They reinforce each other — one shapes *strategy* (use\nfittok), the other stops the *double-read*. For a stronger, more explicit block:\n\n> For any codebase question (\"how does X work\", \"where is Y\"):\n> 1. Call the fittok MCP tool first, once.\n> 2. Answer directly from its `optimized_context` — it is the real, authoritative\n>    source for that question.\n> 3. Do NOT read or grep the files fittok already returned code from. That\n>    discards the token savings fittok exists to provide.\n\nFor the strongest effect, put it in your **user-global** instructions so it\napplies to every repo, not just one:\n\n| Client | Instructions file |\n|---|---|\n| Claude Code | `CLAUDE.md` (repo) or `~/.claude/CLAUDE.md` (user-global) |\n| GitHub Copilot | `.github/copilot-instructions.md` or Copilot user instructions |\n| Cursor | `.cursor/rules/*.mdc` (or `.cursorrules`) |\n| Windsurf | `.windsurfrules` |\n\n> fittok also bakes this rule into every response (an \"answer from this, don't\n> re-read\" line above the code), so it works even without the snippet above —\n> the snippet just makes it the client's default across all questions.\n\n### CLI\n\n```bash\ncd /path/to/your/repo\n\nuvx fittok index                                      # optional pre-warm (~15s, cached)\nuvx fittok query \"how does auth work\"                 # LLM answers from relevant code\nuvx fittok query \"how does auth work\" --budget 1500   # cap the slice at 1500 tokens\nuvx fittok query \"how does auth work\" --code          # raw relevant code, no LLM\nuvx fittok graph                                      # interactive browser graph\nuvx fittok graph --query \"auth\"                       # graph with relevant nodes highlighted\n```\n\n`query` sends the relevant code slice to an LLM and streams the answer.\nSet one key in your shell and it just works:\n\n```bash\nexport ANTHROPIC_API_KEY=\"sk-ant-...\"   # → claude-haiku-4-5  (recommended)\nexport OPENAI_API_KEY=\"sk-...\"          # → gpt-4o-mini  (fallback)\n```\n\nUsers of Claude Code already have `ANTHROPIC_API_KEY` set — no extra step needed.\nIf neither key is set, fittok falls back to `--code` and prints a setup hint.\n\n`graph` requires `pyvis`: `uv pip install \"fittok[ui]\"`.\n\n### Python library\n\n```bash\nuv add fittok            # in a uv project   (or:  uv pip install fittok  in a venv)\n```\n```python\nfrom fittok import optimize\n\nresult = optimize(\"/path/to/repo\", \"how does authentication work\", token_budget=1500)\nprint(result[\"optimized_context\"])   # the relevant code slice\nprint(result[\"savings\"])             # token reduction stats\n```\n\n### Upgrading\n\n`uvx` caches the environment, so a new `fittok` release isn't picked up\nautomatically — server restarts reuse the cached version. Upgrade with one\ncommand (no need to re-register the MCP server):\n\n```bash\nuvx --refresh fittok        # re-resolve from PyPI → latest version\n```\n\nThen restart the MCP server (reload the window in VS Code, or restart the\nCopilot CLI) so it boots the new version. For other runtimes:\n\n- **pip:** `python -m pip install --upgrade fittok`\n- **pipx:** `pipx upgrade fittok`\n\n---\n\n## Why tree-sitter, not LSP?\n\nfittok uses **tree-sitter** (fast, syntactic AST parsing) instead of **LSP** (Language Server Protocol — semantic analysis with types, cross-file references, go-to-definition). This is a deliberate tradeoff:\n\n| | tree-sitter (fittok) | LSP (e.g. Serena MCP) |\n|---|---|---|\n| **What it returns** | Actual source code — the model answers directly | Symbol metadata (names, refs, types) — the model must still Read files |\n| **Setup** | Zero config, works on any directory | Needs language servers installed + project config (tsconfig, pyproject, etc.) |\n| **Languages** | 8 out of the box (Python, JS/TS/TSX, Java, Go, Rust) | Only as many as LSP servers you've installed |\n| **Startup** | ~15s (parse + embed) | Minutes (full project indexing per language) |\n| **Memory** | ~100 MB (graph + embeddings) | 500 MB+ per language server |\n| **Model calls per question** | 1–5 (one-shot retrieval) | 5–20+ (iterative symbol navigation) |\n| **Token cost** | ~2,500 tokens (code delivered directly) | ~15,000+ tokens (metadata + file reads) |\n\n**fittok's USP is token savings.** It returns the actual code in one call so the model doesn't need to read files. LSP-based tools return metadata (symbol names, reference lists) — precise, but the model still has to open the files to see the implementation. More round-trips, more tokens.\n\n**The tradeoff:** tree-sitter can't resolve cross-file references as accurately as LSP (a `fetch(\"/api/run\")` call in a .tsx file won't perfectly link to the route handler). fittok compensates with 4-signal retrieval (semantic + content-BM25 + structural summary-BM25 + PageRank, fused via RRF) and round-robin directory diversity — which cover the gap in practice.\n\n**Complementary, not competitive:** LSP-based tools like [Serena](https://github.com/oraios/serena) excel at symbol-level navigation (\"find all callers of `runSandboxQuery`\"). fittok excels at semantic retrieval (\"how does SQL execution work?\"). Install both — the model picks the right tool per task.\n\n---\n\n## Token savings — honest numbers\n\nOn a real Next.js/TS repo (~5k functions), fittok returns a **~1.5–3.5k-token\nslice** instead of the model reading **15–20k+ tokens** of files — an **~80–90%\nreduction on input**, deterministic and reported in the `savings` footer.\n\nOn Opus 4.8, a broad question cost **~84k total tokens without fittok vs ~27k\nwith it** — because fittok replaced a 58k-token Explore subagent with one tool call.\n\n**How to measure it honestly:**\n- Use the **`🪙 saved X%` footer** or your **API bill** (total tokens).\n- Do *not* judge by Claude Code's `/context` Messages number — it excludes\n  subagent tokens and is dominated by model reasoning, which fittok doesn't touch.\n\n---\n\n## Configuration\n\n| Variable | Default | Description |\n|---|---|---|\n| `ANTHROPIC_API_KEY` | — | Enables LLM answers via `claude-haiku-4-5` |\n| `OPENAI_API_KEY` | — | Fallback LLM via `gpt-4o-mini` |\n| `FITTOK_SHOW_SAVINGS` | `true` | `🪙 saved X%` footer on MCP answers; set `false` to disable |\n| `FITTOK_MAX_BUDGET` | `0` (unlimited) | Code-token cap. `0` = return ALL relevant code in full (default — complete results in Claude Code / CLI). Set `1200` for **GitHub Copilot**, which truncates MCP outputs >~7 KB (see Known limitations). |\n| `FITTOK_AUTOWATCH` | `true` | Auto-start the file watcher so graph updates are incremental (only changed files re-parse); set `false` to fall back to full re-parse on edits |\n| `FITTOK_EMBED_MODEL` | `all-MiniLM-L6-v2` | Embedding model |\n| `FITTOK_DEVICE` | `auto` | `auto` / `cuda` / `mps` / `cpu` |\n| `FITTOK_CACHE_DIR` | `~/.cache/fittok` | Cache location |\n\nFull reference: **[docs/HANDBOOK.md](https://github.com/likhithreddy/fittok/blob/main/docs/HANDBOOK.md)**\n\n---\n\n## Requirements\n\nPython ≥ 3.10. First run downloads a ~90 MB embedding model. Graph visualization\n(`fittok graph`) is included by default. Optional extras:\n- `uv pip install \"fittok[ui]\"` — Gradio web dashboard (`launch_ui` tool)\n- `uv pip install \"fittok[gpu]\"` — torch/CUDA for GPU-accelerated embeddings\n\n## License\n\nMIT\n",
  "bytes": 15723,
  "sha": "9bdeb01241b8c2b7a7837d250d8a9e49f431feeabe602812b708bf1a72f0994c",
  "repo_slug": "likhithreddy/fittok",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_likhithreddy_fittok_48583c1b/readme"
}