{
"markdown": "# github-twin\n\n[](https://pypi.org/project/github-twin/)\n[](https://pypi.org/project/github-twin/)\n[](https://github.com/ChristopherDavenport/github-twin/actions/workflows/ci.yml)\n[](LICENSE)\n\n> You reviewed a permission check six months ago. Claude doesn't remember\n> it. **github-twin does** — it indexes your commits and review comments,\n> and surfaces them as retrieval hits whenever an agent writes or reviews\n> new code in your style.\n\n\n
\n \n
\n
\n \n
\n\nTry it now from Claude Code — drop this into `~/.claude.json` and reload:\n\n```json\n{\n \"mcpServers\": {\n \"github-twin\": { \"command\": \"uvx\", \"args\": [\"github-twin\", \"serve\"] }\n }\n}\n```\n\n> **Your code stays on your box.** Embeddings are computed locally\n> (Ollama or sentence-transformers); only the LLM seam (`gt summarize`,\n> `gt distill`, `gt eval`) optionally calls a hosted provider, and even\n> that's swappable to local Ollama. The `gemini` embedder is the one\n> exception — opt-in only.\n\n---\n\nA personal RAG over your GitHub history, served to Claude Code (or any MCP\nclient) as a stdio server. Two scopes, one codebase:\n\n- **User mode** — index your own commits + review comments. Surfaces your\n past code as style examples and your past comments as review hints when\n an agent is writing or reviewing new code.\n- **Org mode** — index a whole GitHub org's files-at-HEAD, commits, and PR\n reviews across every member. Queries scope by repo, language, or\n reviewer login.\n\nRetrieval is hybrid (BM25 + vector via RRF), AST-aware via tree-sitter for\npython/scala/javascript/typescript/go/rust, and contextually enriched at\nembed time with per-chunk headers + optional LLM-generated summaries.\n\n## Install\n\nThe fastest path is [uvx](https://docs.astral.sh/uv/) — no virtualenv to\nmanage, isolated per-tool:\n\n```sh\n# One-shot\nuvx github-twin --help\n\n# Pinned version\nuvx github-twin@0.1.0 --help\n\n# With sentence-transformers for the alt embedder\nuvx --with 'github-twin[st]' github-twin --help\n```\n\nIf you prefer a project-local install:\n\n```sh\nuv add github-twin # or: pip install github-twin\ngt --help\n```\n\n`gt` and `github-twin` are the same Typer app — use whichever fits your\nmuscle memory.\n\n## Authenticate\n\nPick whichever is least friction — github-twin tries them in this order:\n\n1. **OAuth device flow (no `gh` install needed):**\n ```sh\n uvx github-twin auth login # opens browser, persists token\n uvx github-twin auth status # show which source is active\n ```\n Token persists in the OS keyring (macOS Keychain / Linux Secret\n Service / Windows Credential Manager) or, when unavailable, a 0600\n file under your data dir.\n2. **Existing `gh` CLI**: if you've already run `gh auth login`,\n `gt` picks up the token via `gh auth token` — nothing to do.\n3. **`GITHUB_TOKEN` env var**: a classic PAT works too; useful for CI /\n headless / docker. Required scopes: `repo`, `read:org`, `user:email`.\n\n## Wire into Claude Code\n\nThe MCP server runs over stdio via `github-twin serve` (or `gt serve`).\nRun `uvx github-twin auth login` once on the box that will host the\nserver.\n\n**Option A — via the Claude Code plugin marketplace** (lowest-friction):\n\n```\n/plugin marketplace add ChristopherDavenport/christopherdavenport-marketplace\n/plugin install github-twin@christopherdavenport\n```\n\nThis registers the MCP server entry automatically; set\n`GT_PATHS__DATA_DIR` in your environment (or in `~/.claude.json`'s\n`env` block for this server) to point at the DB directory.\n\n**Option B — manual wiring**: add an entry to `~/.claude.json` (or\nyour `mcp_servers.json`):\n\n```json\n{\n \"mcpServers\": {\n \"github-twin\": {\n \"command\": \"uvx\",\n \"args\": [\"github-twin\", \"serve\"],\n \"env\": {\n \"GT_PATHS__DATA_DIR\": \"/path/to/your/github-twin-data\"\n }\n }\n }\n}\n```\n\nIf you'd rather not persist a token and instead supply it inline (CI,\nephemeral container), add `\"GITHUB_TOKEN\": \"ghp_...\"` to that `env`\nblock; it acts as the lowest-priority fallback.\n\nRestart Claude Code; the `find_*`, `predict_review_outcome`,\n`summarize_review_patterns`, and `sync` tools will be available.\n\n## Quickstart\n\nPick a directory to hold the SQLite DB, config, and ingested cache —\neverything per-data-dir lives under this one root:\n\n```sh\nexport GT_PATHS__DATA_DIR=~/github-twin-data\nuvx github-twin auth login # one-time OAuth (or set GITHUB_TOKEN)\n\n# user mode (your own GitHub history)\nuvx github-twin init # discover identity via /user\nuvx github-twin sync # ingest + summarize + embed\nuvx github-twin serve # MCP server over stdio\n\n# layer an org into the SAME DB\nuvx github-twin init --kind org --org http4s\nuvx github-twin sync\n\n# OR keep the org in its own DB by switching data dirs\nGT_PATHS__DATA_DIR=~/twin-http4s \\\n uvx github-twin init --kind org --org http4s\nGT_PATHS__DATA_DIR=~/twin-http4s uvx github-twin sync\n```\n\n`gt sync` is incremental on subsequent runs.\n\n`config.toml` lives next to the DB at `/config.toml` and is\ncreated on the first `gt init --embed-backend ...` call. Default\n`` is `$XDG_DATA_HOME/github-twin` (or `~/.local/share/github-twin`)\nwhen `GT_PATHS__DATA_DIR` is unset.\n\n## LLM provider matrix\n\nThe retrieval surface (find_*, predict_review_outcome) always runs locally\non the SQLite index — no API call. **LLM calls only happen** in three\nplaces:\n\n- `gt distill` — clusters review comments / commits into rules.\n- `gt summarize` — generates per-chunk NL summaries used by the embed-time\n prefix.\n- `gt eval reviews` / `eval predictions` — held-out RAG-vs-baseline scoring.\n\nEach picks a backend by precedence **Claude → Gemini → Ollama** (whichever\nAPI key is set), or you can force one explicitly.\n\n| Provider | Env var | What it covers |\n|---|---|---|\n| Anthropic (Claude) | `ANTHROPIC_API_KEY` | Distill / summarize / eval LLM. Best quality. |\n| Google (Gemini, API key) | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Distill / summarize / eval LLM. Free tier is generous. |\n| Google (Gemini, Vertex / ADC) | `GT_GEMINI_PROJECT` (+ optional `GT_GEMINI_LOCATION`, default `us-central1`) | Same backends, but auth via `gcloud auth application-default login` — no key in your shell. API key wins if both are set. |\n| Ollama (local) | `OLLAMA_HOST` (default `http://127.0.0.1:11434`) | Distill / summarize / eval LLM. Fully offline. |\n\nThe Vertex / ADC path needs the `aiplatform.googleapis.com` API enabled\non your project, and **billing applies even for \"free\" Gemini models** —\nthe AI Studio free tier does not extend to Vertex. Project IDs are not\nsecrets; the credential itself lives at\n`~/.config/gcloud/application_default_credentials.json` and is refreshed\nby gcloud.\n\n### Embedder backends\n\nWe keep the embedder backend separate from the LLM backend. Choose one:\n\n- **Default — Ollama** (`nomic-embed-text`, 768-dim, ~50ms/chunk).\n Requires a running Ollama daemon. Zero cost, fully local.\n- **Alternative — sentence-transformers** (`uv add 'github-twin[st]'`,\n pulls `torch`). Useful when an Ollama daemon isn't available or you\n want a specific HuggingFace model. Local.\n- **Alternative — Gemini** (`gemini-embedding-001` at 3072-dim by\n default). Uses the `google-genai` dep that's already installed; auth\n via `GEMINI_API_KEY` / `GOOGLE_API_KEY`, or via `GT_GEMINI_PROJECT`\n + ADC (`gcloud auth application-default login`) to route through\n Vertex AI without managing a key. **Remote** — this is the only\n embedder that sends chunk text off-box. Pick it when you have Gemini\n auth but no Ollama / `[st]` install, and your corpus is okay to\n share with Google.\n\nThe embedder is a per-DB commitment — `sqlite-vec` bakes the vector\ndimension into the table at first creation. Stamp the choice into\n`/config.toml` at init time so every subsequent command\npicks it up:\n\n```sh\ngt init --embed-backend gemini # gemini-embedding-001, 3072\ngt init --embed-backend gemini --embed-dim 1536 # request shorter output\ngt init --embed-backend sentence_transformers \\\n --embed-model BAAI/bge-small-en-v1.5 --embed-dim 384\n```\n\nRe-running with the same values is a no-op; running with different\nvalues against an existing `config.toml` fails loud rather than\nsilently changing the corpus. `GT_EMBED__*` env vars still work for\none-off overrides and CI.\n\nA \"cloud-LLM only\" setup either needs an embedder process (Ollama /\n`[st]`) or has to opt into the remote Gemini embedder.\n\n## Required GitHub token scopes\n\nWhen you `gt init`, the GH client needs:\n\n- `repo` — private repos and PR comments on them\n- `user:email` — verified email addresses for the user-mode identity sweep\n- `read:org` — org member listing and private org repo discovery\n\nA fine-grained PAT works; classic tokens too.\n\n## Retrieval\n\nHybrid search by default: BM25 (SQLite FTS5) and vector similarity run in\nparallel, then fuse via Reciprocal Rank Fusion (k=60). The vector leg\nmatches semantic intent; the BM25 leg catches exact identifiers\n(`getUserById`, `SQLITE_OPEN_READWRITE`) that vector search routinely\nmisses. Design reference: [Anthropic — Contextual\nRetrieval](https://www.anthropic.com/engineering/contextual-retrieval).\n\nAt embed time, each chunk gets a deterministic header prepended:\n`# path :: symbol_name (node_kind)`, plus the function's leading\ndocstring/comment when present, plus an optional LLM-generated summary\n(see `gt summarize`). The header lets vector queries land on chunks\nwhose bodies only contain identifiers (e.g. natural-language queries\nagainst a `VaultSecretEq` function).\n\nBM25 query expansion is on by default (`cfg.retrieval.query_expansion =\n\"rule\"`), with rule-based code-shaped synonyms applied **only** to the\nBM25 leg — embeddings already capture synonymy, so expansion never\ntouches the vector query. Switch to `\"ollama\"` to add LLM-generated\nalternates on top, cached on disk per-token.\n\n`predict_review_outcome` stays on pure vector retrieval because its\ninverse-distance vote weighting depends on calibrated L2 distance.\n\n## MCP tools\n\nAll retrieval tools accept optional `repo=` and `author_login=` filters.\n\n| Tool | Returns |\n|---|---|\n| `find_review_comments(diff_hunk, language?, repo?, author_login?, k=5)` | Past review comments on diffs similar to the input. |\n| `find_style_examples(query, language?, repo?, author_login?, k=5)` | Past code chunks matching a description. |\n| `find_code(query, language?, repo?, path_glob?, node_kind?, k=5)` | Source snippets from files at HEAD (org mode). |\n| `find_applicable_rules(query, language?, repo?, author_login?, k=5)` | Distilled code-pattern rules relevant to a coding task. |\n| `predict_review_outcome(diff_or_summary, language?, repo?, author_login?, k=20)` | Weighted prediction over nearest past PRs: `{approved, changes_requested, commented}`. |\n| `summarize_review_patterns(language?, limit=20)` | Distilled rules from clustered review comments (run `gt distill` first). |\n| `sync(since?)` | Incremental ingest + summarize + embed. |\n\n## CLI\n\n```\ngt init [--kind user|org|repo] [--org N] [--repo owner/name]\ngt repos # list discovered org repos\ngt ingest # backfill\ngt summarize [--limit N] [--backend ...] # LLM NL summaries per chunk\ngt embed # embed pending chunks\ngt sync [--skip-summarize] # incremental: ingest → summarize → embed\ngt stats # corpus counts\ngt distill [--backend ...] [--author ...] # rule extraction\ngt clones prune [--older-than-days N] # GC the persistent clone cache\ngt eval reviews --since DATE [...] # held-out RAG-vs-baseline eval\ngt eval predictions --since DATE [...]\ngt eval search evals/queries/default.yaml # retrieval-quality dogfood\ngt serve # MCP stdio server\n```\n\nUse `github-twin ` interchangeably with `gt `.\n\n## Pluggable backends\n\n| Surface | Env / config key | Default | Alt |\n|---|---|---|---|\n| LLM (`cfg.distill.backend`, `cfg.summarize.backend`) | `ANTHROPIC_API_KEY` / `GEMINI_API_KEY` (or `GT_GEMINI_PROJECT` + ADC) / Ollama | `auto` (cloud > local) | force `claude` / `gemini` / `ollama` |\n| Embedder (`cfg.embed.backend`) | — / `GEMINI_API_KEY` (or `GT_GEMINI_PROJECT` + ADC) | `ollama` (`nomic-embed-text`) | `sentence_transformers` via `[st]` extra, or `gemini` (`gemini-embedding-001`, remote) |\n| Vector store (`cfg.vector_store.backend`) | — | `sqlite-vec` (brute-force KNN) | `faiss` via `[faiss]` extra |\n| BM25 query expansion (`cfg.retrieval.query_expansion`) | — | `rule` (deterministic) | `ollama` (LLM, cached) or `off` |\n\nAll settings are layered: defaults → `/config.toml` (or the\nexplicit `--config PATH`) → env vars prefixed `GT_` (nested via `__`,\ne.g. `GT_EMBED__BACKEND=sentence_transformers`).\n\n## Held-out evaluation\n\n`gt eval` runs the same prompt with and without retrieval and measures\nRAG's accuracy lift on real held-out data:\n\n```sh\n# Review-comment voice match (cosine distance to ground truth)\nuvx github-twin eval reviews --since 2025-01-01 --limit 100\n\n# Org-mode: scope to one reviewer (and optionally one repo)\nuvx github-twin eval reviews --since 2025-01-01 --author alice --repo http4s/http4s\nuvx github-twin eval predictions --since 2025-01-01 --author alice\n\n# Retrieval-quality dogfood (per-tier, per-backend pass rates)\nuvx github-twin eval search evals/queries/default.yaml --mode all\n```\n\nThe harness pre-flights eligibility counts so typo'd `--author` or\n`--repo` fail fast without burning LLM calls. The judge embedder\ndefaults to a different model than the retriever\n(sentence-transformers BGE-small with the `[st]` extra installed) to\navoid measuring how well retrieval clusters its own outputs.\n\n## Observability (OpenTelemetry)\n\nSpans for every MCP tool call, every embedder call, and every retrieval\nleg, exported via OTLP. **Auto-detected** — nothing fires unless the\nenvironment is configured. Specifically:\n\n1. Install the `[otel]` extra (carries the SDK + HTTP OTLP exporter):\n\n ```sh\n uvx --with 'github-twin[otel]' github-twin serve\n ```\n\n2. Point at an OTLP HTTP collector via env vars:\n\n ```sh\n export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318\n export OTEL_SERVICE_NAME=github-twin # optional\n ```\n\n`OTEL_SDK_DISABLED=true` forces it off even when an endpoint is set.\n\nWithout `[otel]` *or* without an endpoint env var, the code paths\nstill run but every span is a free no-op from `opentelemetry-api`'s\nbuilt-in tracer. **stdout is never used** — even with telemetry on —\nbecause MCP speaks JSON over stdin/stdout and a stray console exporter\nwould corrupt the channel. The OTLP HTTP exporter posts to your\ncollector; SDK warnings route through Python `logging` (stderr).\n\nWired into Claude Code:\n\n```json\n{\n \"mcpServers\": {\n \"github-twin\": {\n \"command\": \"uvx\",\n \"args\": [\"--with\", \"github-twin[otel]\", \"github-twin\", \"serve\"],\n \"env\": {\n \"GITHUB_TOKEN\": \"ghp_...\",\n \"GT_PATHS__DATA_DIR\": \"/path/to/twin-data\",\n \"OTEL_EXPORTER_OTLP_ENDPOINT\": \"http://localhost:4318\",\n \"OTEL_SERVICE_NAME\": \"github-twin\"\n }\n }\n }\n}\n```\n\nSpan names + key attributes you can pivot on:\n\n| Span | Useful attributes |\n|---|---|\n| `mcp.tool.{find_review_comments,find_style_examples,find_code,find_applicable_rules,predict_review_outcome,summarize_review_patterns,sync}` | `gh_twin.tool.k`, `gh_twin.filter.*`, `gh_twin.result.count` (or `.prediction`/`.confidence` for predict) |\n| `embedder.embed` | `gh_twin.embed.input_chars`, `gh_twin.embed.model` |\n| `retrieval.hybrid_search` | `gh_twin.retrieval.{chunk_kind,k,expander,hits,top_distance}` |\n| `retrieval.vector_search` (predict_review_outcome) | same shape, sans `expander` |\n\nA broken or unreachable collector emits a single `Failed to export\nspan batch` log line per flush attempt and never propagates into the\ntool handler — pinned by `tests/test_observability.py`.\n\ngRPC users: install `opentelemetry-exporter-otlp-proto-grpc` alongside\nthe `[otel]` extra and the SDK picks it up automatically based on\n`OTEL_EXPORTER_OTLP_PROTOCOL=grpc`.\n\n## Storage\n\nOne DB can hold many targets (user + N orgs + N repos); use a separate\n`GT_PATHS__DATA_DIR` per DB when you want them isolated. The resolved\ndata dir is pure with respect to env:\n\n- `GT_PATHS__DATA_DIR` when set\n- else `$XDG_DATA_HOME/github-twin/` when `XDG_DATA_HOME` is set\n- else `~/.local/share/github-twin/`\n\nThe current working directory is never consulted.\n\nLayout — everything per-DB lives under one root:\n\n```\n/\n db.sqlite # artifacts + chunks + vectors + FTS5 index\n config.toml # written by `gt init --embed-backend ...`\n raw/ # on-disk cache of raw GitHub responses\n clones/ # persistent shallow clones (if cache_clones=true)\n wiki/ # `gt wiki export` default output\n auth/token.json # OAuth file fallback\n query_expansion_cache.sqlite # only when retrieval.query_expansion=ollama\n```\n\nIf you ran an older release that wrote `./config.toml` or `./data/` into\nthe current working directory, every CLI invocation logs a one-time WARN\nwith the exact `mv` command to migrate.\n\n## Releasing\n\nVersions come from git tags via [hatch-vcs](https://github.com/ofek/hatch-vcs).\nCutting a release is one command:\n\n```sh\ngit tag v0.2.0 # PEP 440 forms: v0.2.0, v0.2.0a1, v0.2.0rc1, v0.2.0.post1\ngit push --tags\n```\n\nThe push to a `v*` tag triggers `.github/workflows/release.yml`, which:\n\n1. Runs pytest + ruff + `uv build` across Python 3.12 / 3.13.\n2. Publishes the wheel + sdist to PyPI via [Trusted Publishing\n (OIDC)](https://docs.pypi.org/trusted-publishers/). **No PyPI token\n is stored in repo secrets** — PyPI verifies the GitHub-signed OIDC\n token against the Trusted Publisher you register on the project\n page (workflow filename `release.yml`, environment `pypi`).\n3. Creates a GitHub Release with auto-generated notes (PRs since the\n previous tag) and attaches the wheel + sdist. Pre-release tags\n (`a/b/rc`) are flagged so they don't replace \"Latest\".\n4. Bumps `.claude-plugin/plugin.json` on `main` to match the tag —\n sets `version` and pins the MCP server invocation to\n `uvx github-twin@X.Y.Z serve`. The marketplace fetches the manifest\n from HEAD, so this is what users get when their marketplace cache\n refreshes.\n\nFirst-time setup, once per repo:\n\n1. Push the project to GitHub (any account / org).\n2. Register the Trusted Publisher on PyPI:\n - https://pypi.org/manage/account/publishing/\n - Owner: your GitHub user/org, Repository: `github-twin`,\n Workflow: `release.yml`, Environment: `pypi`.\n - (Or use the \"Pending Publisher\" flow if the project doesn't exist\n on PyPI yet.)\n3. On GitHub: Settings → Environments → New environment `pypi`. Add\n yourself as a Required Reviewer for an extra approval step before\n each publish (optional but recommended).\n\n`.github/workflows/ci.yml` runs on every PR and push to `main` — the\nrelease workflow re-runs the same checks before publishing, so a\nbroken main never produces a release.\n\n## Design notes\n\n- **Embed-time prefix** (`embed.prefix.prefix_chunk`): per-kind header\n spliced before each chunk's text at embed time, never written back to\n `chunk.text`. Bumps `EMBED_TEXT_VERSION` whenever the shape changes\n so the next `gt embed` re-derives vectors.\n- **AST chunking** (`process.chunkers`): tree-sitter walks emit chunks\n per declarable unit; falls back to line-window for unsupported\n languages or parse failures.\n- **Asymmetric query expansion**\n (`store.query_expansion`): BM25 leg only, vector leg always sees the\n raw embedding — pinned by `test_hybrid_search.py`.\n\nThe original design plan lives in\n[`getting_started.md`](./getting_started.md) along with the full\nwalkthrough.\n\n## License\n\nMIT — see [LICENSE](./LICENSE).\n\n\nmcp-name: io.github.ChristopherDavenport/github-twin\n",
"bytes": 20794,
"sha": "9b1f6339471e6a2253d7f755b08b40aa2c592d62b3005c6068b17cc2bb00aa31",
"repo_slug": "christopherdavenport/github-twin",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_christopherdavenport_github_tw_f49d63e5/readme"
}
![\"gt](\"docs/assets/demo.gif\")
\n![\"Claude](\"docs/assets/claude-code-screenshot.png\")
\n