{
  "markdown": "# DeepRepo — Local RAG Engine for Codebases\n\nA production-grade Python library for performing RAG (Retrieval Augmented Generation) on local codebases. **No heavy frameworks, no external vector DBs, no cloud required.**\n\n## What It Does\n\nDeepRepo ingests a codebase and builds three things simultaneously:\n\n| Layer | What it stores | Used for |\n|---|---|---|\n| **Code Knowledge Graph** | Classes, functions, imports, call edges (SQLite) | Symbol lookup, blast-radius analysis |\n| **Embeddings + FTS index** | Semantic vectors + full-text search | Relevant code retrieval |\n| **Hierarchical Wiki** | Plain-English `.md` files per module | AI explanations, chat context |\n\nA smart **query router** classifies every question and picks the cheapest context strategy, reducing LLM token usage by 5–50x compared to naive RAG.\n\n---\n\n## Features\n\n- **Zero dependencies on heavy frameworks** — pure Python, SQLite-backed\n- **Multiple AI providers** — Ollama (free/local), OpenAI, Anthropic, Gemini, HuggingFace\n- **CLI-first** — `deeprepo ingest .` / `deeprepo serve` / `deeprepo query \"…\"`\n- **Wiki viewer** — browsable, searchable HTML wiki with in-page chat (`deeprepo serve`)\n- **7 focused MCP tools** — drop DeepRepo into Cursor / Claude Desktop as an MCP server\n- **Branch isolation** — per-branch SQLite databases with copy-on-write from base branches\n- **3-tier retrieval** — Embeddings → FTS → Graph fallback for resilient search\n- **Incremental ingestion** — unchanged files are skipped; only deltas re-processed\n\n---\n\n## Quick Start\n\n### 1. Install\n\n```bash\ncd deeprepo_core\npip install -e .\n```\n\nFor MCP server support:\n```bash\npip install -e \".[mcp]\"\n```\n\n### 2. Install Ollama (free, local — recommended)\n\n```bash\n# macOS\nbrew install ollama\nollama serve                          # keep this running\n\nollama pull nomic-embed-text          # embedding model\nollama pull llama3.1:8b               # LLM\n```\n\n### 3. Ingest your codebase\n\n```bash\ncd /path/to/your/project\ndeeprepo ingest .\n```\n\n### 4. Browse the wiki\n\n```bash\ndeeprepo serve                        # opens http://localhost:8080\n```\n\n### 5. Ask questions\n\n```bash\ndeeprepo query \"how does authentication work?\"\ndeeprepo query \"what breaks if I change auth.py?\"\n```\n\n---\n\n## CLI Reference\n\n```\ndeeprepo <command> [options]\n```\n\n| Command | What it does |\n|---------|-------------|\n| `deeprepo init` | Detect provider setup, print the ingest command |\n| `deeprepo ingest [PATH]` | Scan repo → build graph + wiki + embeddings |\n| `deeprepo wiki [PATH]` | Regenerate wiki pages only (skip re-indexing) |\n| `deeprepo serve` | Launch wiki viewer + in-page chat at port 8080 |\n| `deeprepo query \"QUESTION\"` | Ask a question, get an AI answer |\n| `deeprepo status` | Show branch isolation & cache freshness |\n\n### Common flags (all commands)\n\n```bash\n--llm ollama|openai|anthropic|gemini|huggingface   # LLM provider\n--embed ollama|openai|huggingface                  # embedding provider (default: same as --llm)\n--branch-isolation                                 # enable per-branch databases\n--base-branch main                                 # seed feature-branch cache from main\n--wiki-dir .deeprepo/wiki                          # override wiki output directory\n```\n\n### ingest flags\n\n```bash\n--chunk-size N      # chars per text chunk (default: 1000)\n--overlap N         # overlap between chunks (default: 100)\n--workers N         # wiki parallel workers (default: 3)\n--no-wiki           # skip wiki generation\n```\n\n### serve flags\n\n```bash\n--port N            # HTTP port (default: 8080)\n```\n\n### Examples\n\n```bash\n# Ollama (free, fully local)\ndeeprepo ingest .\n\n# OpenAI embeddings + Anthropic LLM\ndeeprepo ingest . --embed openai --llm anthropic\n\n# Branch isolation for a feature branch\ndeeprepo ingest . --branch-isolation --base-branch main\n\n# Serve wiki with chat on a custom port\ndeeprepo serve --llm openai --port 9000\n\n# Query with specific top-k results\ndeeprepo query \"where is AuthService defined?\" --top-k 3\n```\n\n---\n\n## Python API\n\n```python\nfrom deeprepo import DeepRepoClient\n\n# Single provider (backward-compatible shorthand)\nclient = DeepRepoClient(provider_name=\"ollama\")\n\n# Split providers — Anthropic LLM + OpenAI embeddings\nclient = DeepRepoClient(\n    embedding_provider_name=\"openai\",\n    llm_provider_name=\"anthropic\",\n)\n\n# Branch isolation (team workflow)\nclient = DeepRepoClient(\n    provider_name=\"ollama\",\n    branch_isolation=True,\n    base_branches=[\"main\"],\n)\n\n# Ingest (incremental — unchanged files are skipped)\nresult = client.ingest(\"/path/to/your/code\")\nprint(f\"Files: {result['files_scanned']}, Wiki pages: {result['wiki_generated']}\")\n\n# Query — smart routing selects the cheapest context strategy\nresponse = client.query(\"How does authentication work?\")\nprint(response['answer'])\nprint(f\"Intent: {response['intent']}, Strategy: {response['strategy']}\")\nprint(f\"Sources: {response['sources']}\")        # list of file paths\n\n# Browse the generated wiki\nprint(f\"Wiki at: {client.get_wiki_dir()}\")\n```\n\n### `query()` return shape\n\n```python\n{\n    \"answer\":         str,           # LLM-generated answer\n    \"sources\":        list[str],     # file paths used as context\n    \"intent\":         str,           # navigate | impact | explain | debug | review | general\n    \"strategy\":       str,           # e.g. symbol_lookup, blast_radius, wiki_plus_skeleton, …\n    \"retrieval\":      str,           # embeddings | fts | graph\n    \"token_estimate\": int,           # estimated tokens consumed\n    \"history\":        list[dict],    # conversation history (last N exchanges)\n}\n```\n\n---\n\n## Supported AI Providers\n\n| Provider | Cost | Setup | Best For |\n|----------|------|-------|----------|\n| **Ollama** | FREE, unlimited | Install app + `ollama pull` | Local dev, privacy, offline |\n| **OpenAI** | Paid | `OPENAI_API_KEY` | Production, best quality |\n| **Anthropic** | Paid | `ANTHROPIC_API_KEY` | Production, excellent reasoning |\n| **Gemini** | Free tier | `GEMINI_API_KEY` | Experimentation |\n| **HuggingFace** | Free tier | `HUGGINGFACE_API_KEY` | Cloud embeddings, no GPU needed |\n\n> **Note:** Anthropic has no embeddings API. Pair it with another provider:\n> ```python\n> client = DeepRepoClient(embedding_provider_name=\"openai\", llm_provider_name=\"anthropic\")\n> ```\n\n---\n\n## Architecture\n\n```\ndeeprepo_core/src/deeprepo/\n├── client.py         # Main facade — branch isolation, freshness, provider wiring\n├── graph.py          # SQLite store: graph nodes/edges, embeddings, wiki index, state\n├── graph_builder.py  # Tree-sitter AST parser → code knowledge graph\n├── wiki.py           # Hierarchical wiki engine — bottom-up LLM synthesis\n├── router.py         # Intent classifier + 6 context strategy selectors\n├── ingestion.py      # File scanner, chunker, language detection\n├── interfaces.py     # Abstract base classes (EmbeddingProvider, LLMProvider)\n├── registry.py       # @register_embedding / @register_llm decorator system\n├── ui.py             # Wiki viewer (HTTP server + mermaid renderer + chat)\n├── mcp/\n│   └── server.py     # 7 MCP tools for AI assistants (Cursor, Claude Desktop)\n└── providers/\n    ├── ollama_v.py\n    ├── openai_v.py\n    ├── anthropic_v.py\n    ├── gemini_v.py\n    └── huggingface_v.py\n\n.deeprepo/            # Generated (gitignore this)\n├── default.db        # SQLite: graph + embeddings + wiki index + state\n├── <branch>.db       # Per-branch database when branch_isolation=True\n└── wiki/             # Browsable .md wiki files\n    ├── overview.md   # Whole-repo narrative overview\n    └── *.md          # One page per module\n```\n\n### Storage\n\nEverything lives in a **single SQLite file per branch** — no Redis, no Postgres, no Chroma.\n\n| Table | Contents |\n|-------|----------|\n| `nodes` | Files, classes, functions with metadata |\n| `edges` | Import / call relationships between nodes |\n| `embeddings` | Float vectors for semantic search |\n| `wiki_pages` | Generated wiki markdown (key → content) |\n| `wiki_fts` | Full-text search index over wiki |\n| `state` | Per-file SHA-256 hashes for incremental updates |\n\n### Design Patterns\n\n- **Facade** — `DeepRepoClient` is the single entry point; internals are hidden\n- **Strategy** — `LLMProvider` / `EmbeddingProvider` abstract interfaces; providers are swappable\n- **Registry** — `@register_llm(\"ollama\")` decorator auto-registers providers at import time\n- **Bottom-up synthesis** — wiki pages generated leaves-first; parent pages consume child summaries\n- **3-tier fallback** — Embeddings → FTS → Graph; queries work even when embeddings are cold\n- **Copy-on-write branching** — feature branches start from base-branch cache, then delta-update\n\n---\n\n## MCP Server (AI Assistant Integration)\n\nConnect DeepRepo as an MCP server so Cursor, Claude Desktop, or any MCP-compatible AI assistant can call it directly — without ever reading raw files.\n\n### Setup\n\n```bash\npip install deeprepo[mcp]\n```\n\n**Cursor** — create `~/.cursor/mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"deeprepo\": {\n      \"command\": \"python\",\n      \"args\": [\"-m\", \"deeprepo.mcp.server\"],\n      \"env\": {\n        \"LLM_PROVIDER\": \"ollama\"\n      }\n    }\n  }\n}\n```\n\n**Claude Desktop** — add to `~/Library/Application Support/Claude/claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"deeprepo\": {\n      \"command\": \"deeprepo-mcp\",\n      \"env\": {\n        \"EMBEDDING_PROVIDER\": \"openai\",\n        \"LLM_PROVIDER\": \"anthropic\",\n        \"OPENAI_API_KEY\": \"sk-...\",\n        \"ANTHROPIC_API_KEY\": \"sk-ant-...\"\n      }\n    }\n  }\n}\n```\n\n### Available MCP Tools (7 tools)\n\n| Tool | When to use | Token cost |\n|------|-------------|-----------|\n| `ingest_codebase` | One-time setup — index a repo directory | — |\n| `find_symbol` | \"Where is X defined / what line is X on\" | ~50 tokens |\n| `get_file_structure` | \"Show me the API / functions in X\" | ~150 tokens |\n| `explain_file` | \"How does X work / explain X / what does X do\" | ~300 tokens |\n| `find_change_impact` | \"What breaks if I change X\" | ~300 tokens |\n| `ask_codebase` | Any open-ended question about the code | ~600–2000 tokens |\n| `get_project_overview` | \"Give me an overview / what does this project do\" | ~600 tokens |\n\n### Token Reduction vs Naive RAG\n\n| Query type | Naive RAG | DeepRepo | Reduction |\n|---|---|---|---|\n| \"where is X defined\" | ~4 000 tokens | ~80 tokens | **50x** |\n| \"what breaks if I change X\" | ~4 000 tokens | ~300 tokens | **13x** |\n| \"how does X work\" | ~4 000 tokens | ~600 tokens | **7x** |\n| \"fix the bug in X\" | ~4 000 tokens | ~900 tokens | **4x** |\n\n### CLAUDE.md tip\n\nAdd this to your project's `CLAUDE.md` so Claude automatically uses DeepRepo:\n\n```markdown\n## Code navigation\nBefore reading any source file directly, use these MCP tools:\n- `find_symbol(name)` to locate a class or function\n- `get_file_structure(filepath)` to see a file's API without reading it\n- `explain_file(filepath)` to understand what a file does\n- `find_change_impact(filepath)` before editing any file\n- `ask_codebase(question)` for open-ended questions\n- `get_project_overview()` at the start of a new session\n\nOnly call Read/Grep on a file after the above tools have been tried.\n```\n\n---\n\n## Configuration\n\n### Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `LLM_PROVIDER` | `openai` | LLM provider name |\n| `EMBEDDING_PROVIDER` | same as `LLM_PROVIDER` | Embedding provider name |\n| `OPENAI_API_KEY` | — | Required for OpenAI |\n| `ANTHROPIC_API_KEY` | — | Required for Anthropic |\n| `GEMINI_API_KEY` | — | Required for Gemini |\n| `HUGGINGFACE_API_KEY` / `HF_TOKEN` | — | Required for HuggingFace |\n| `OLLAMA_MODEL` | `llama3.1:8b` | Ollama LLM model name |\n| `OLLAMA_EMBED_MODEL` | `nomic-embed-text` | Ollama embedding model |\n| `OLLAMA_BASE_URL` | `http://localhost:11434` | Ollama server URL |\n| `OLLAMA_TIMEOUT` | `300` | LLM response timeout (seconds) |\n\n---\n\n## Testing\n\n```bash\n# Full end-to-end test suite (runs ingest + all checks)\npython3 test_deeprepo_flow.py\n\n# Skip ingest, use cached index (faster iteration)\npython3 test_deeprepo_flow.py --skip-ingest\n\n# pytest unit tests\npytest tests/unit/ -v\n\n# pytest with coverage\npytest tests/unit/ --cov=deeprepo --cov-report=html\n```\n\nThe `test_deeprepo_flow.py` script tests all 7 sections end-to-end:\n1. Client initialisation & branch flags\n2. Ingest (graph + embeddings + wiki)\n3. WikiEngine — page generation, caching, repo overview\n4. Graph API — skeleton, blast-radius, symbol lookup\n5. RAG / Router — intent classification, query execution\n6. CLI commands — all subcommands + help\n7. Branch isolation flag combinations\n\n---\n\n## Adding a New Provider\n\n1. Create `src/deeprepo/providers/myprovider.py`\n2. Implement `EmbeddingProvider` and/or `LLMProvider` interfaces\n3. Decorate with `@register_embedding(\"myprovider\")` / `@register_llm(\"myprovider\")`\n4. Auto-discovered at import time — no other changes needed\n\n```python\nfrom deeprepo.interfaces import EmbeddingProvider, LLMProvider\nfrom deeprepo.registry import register_embedding, register_llm\n\n@register_embedding(\"myprovider\")\nclass MyEmbeddingProvider(EmbeddingProvider):\n    def embed(self, text: str) -> list[float]:\n        ...  # return a list of floats\n\n@register_llm(\"myprovider\")\nclass MyLLMProvider(LLMProvider):\n    def generate(self, prompt: str, context: str | None = None) -> str:\n        ...  # return generated text\n```\n\n---\n\n## Documentation\n\n- **[DEVELOPER_WORKFLOW_GUIDE.md](DEVELOPER_WORKFLOW_GUIDE.md)** — daily dev workflows and automation recipes\n- **[deeprepo_core/README.md](deeprepo_core/README.md)** — package README (PyPI)\n- **[docs/high-level-design.excalidraw](docs/high-level-design.excalidraw)** — process flow diagram\n- **[docs/class-interaction-design.excalidraw](docs/class-interaction-design.excalidraw)** — class diagram\n\n---\n\n## License\n\nMIT License — see LICENSE file for details.\n\n---\n\n*Built for developers who want full control over their RAG pipelines.*\n",
  "bytes": 13891,
  "sha": "a4d993aeda9f7e334b8a26a2105731d726a7283958ef538a8038cfccbbaea1fe",
  "repo_slug": "abhishek2432001/deeprepo",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_abhishek2432001_deeprepo_751a7cc8/readme"
}