{
  "markdown": "# ContextWeaver\n\n<p align=\"center\">\n  <strong>🧵 A codebase context engine woven for AI agents</strong>\n</p>\n\n<p align=\"center\">\n  <em>Semantic Code Retrieval for AI Agents — Hybrid Search • Graph Expansion • Token-Aware Packing</em>\n</p>\n\n<p align=\"center\">\n  <a href=\"https://www.npmjs.com/package/@chiway/contextweaver\">\n    <img src=\"https://img.shields.io/npm/v/@chiway/contextweaver?color=blue&label=npm\" alt=\"npm version\" />\n  </a>\n  <a href=\"https://www.npmjs.com/package/@chiway/contextweaver\">\n    <img src=\"https://img.shields.io/npm/dm/@chiway/contextweaver\" alt=\"npm downloads\" />\n  </a>\n  <a href=\"https://github.com/wchiway/contextweaver-mcp\">\n    <img src=\"https://img.shields.io/github/stars/wchiway/contextweaver-mcp?style=social\" alt=\"GitHub stars\" />\n  </a>\n  <a href=\"https://lobehub.com/mcp/wchiway-contextweaver-mcp\">\n    <img src=\"https://lobehub.com/badge/mcp/wchiway-contextweaver-mcp\" alt=\"MCP Bridge\" />\n  </a>\n  <a href=\"https://vscode.dev/redirect/mcp/install?name=contextweaver&config=%7B%22command%22%3A%22contextweaver%22%2C%22args%22%3A%5B%22mcp%22%5D%7D\">\n    <img src=\"https://img.shields.io/badge/Install%20in-VS%20Code-007ACC?logo=visualstudiocode\" alt=\"Install in VS Code\" />\n  </a>\n  <a href=\"https://insiders.vscode.dev/redirect/mcp/install?name=contextweaver&config=%7B%22command%22%3A%22contextweaver%22%2C%22args%22%3A%5B%22mcp%22%5D%7D\">\n    <img src=\"https://img.shields.io/badge/Install%20in-VS%20Code%20Insiders-1DB954?logo=visualstudiocode\" alt=\"Install in VS Code Insiders\" />\n  </a>\n</p>\n\n<p align=\"center\">\n  <strong>English</strong> ·\n  <a href=\"README.zh-CN.md\">简体中文</a>\n</p>\n\n---\n\n**ContextWeaver** is a semantic retrieval engine purpose-built for AI coding assistants. It combines hybrid search (vector + lexical), intelligent context expansion, and token-aware packing to deliver precise, relevant, and context-complete code snippets to LLMs.\n\n<p align=\"center\">\n  <img src=\"docs/architecture_news.png\" alt=\"ContextWeaver architecture overview\" width=\"800\" />\n</p>\n\n## ✨ Core Features\n\n### 🔍 Hybrid Retrieval Engine\n- **Vector Retrieval**: deep semantic understanding via similarity\n- **Lexical Retrieval (FTS)**: exact matching for function names, class names, and other technical terms\n- **RRF Fusion (Reciprocal Rank Fusion)**: intelligently merges multiple recall channels\n\n### 🧠 AST Semantic Chunking\n- **Tree-sitter parsing**: supports TypeScript, JavaScript, Python, Go, Java, Rust, C, C++, C#, and more\n- **Dual-Text strategy**: `displayCode` for presentation, `vectorText` for embedding\n- **Gap-Aware merging**: handles code gaps intelligently while preserving semantic integrity\n- **Breadcrumb injection**: vector text carries hierarchical paths to boost recall\n- **UTF-16 character-domain normalization**: offsets are unified via `SourceAdapter.toCharOffset` before writing metadata, preventing multi-byte character slicing errors (v1.4.0+)\n\n### 📊 Three-Stage Context Expansion\n- **E1 Neighbor expansion**: adjacent chunks within the same file, preserving block completeness\n- **E2 Breadcrumb completion**: sibling methods under the same class/function for structural understanding\n- **E3 Import resolution**: cross-file dependency tracking (configurable toggle)\n\n### 🎯 Smart TopK Cutoff\n- **Anchor & Floor**: dynamic threshold plus an absolute floor as dual safeguards\n- **Delta Guard**: prevents misjudgment in Top1-outlier scenarios\n- **Safe Harbor**: the first N results only check the floor, guaranteeing baseline recall\n\n### 🔌 Native MCP Support\n- **MCP Server mode**: launch a Model Context Protocol server with one command\n- **Multi-tool granularity** (v1.5.0+): beyond core semantic retrieval, adds dedicated tools for structure browsing, symbol references, symbol definitions, and statistics\n- **Intent/term separation**: an LLM-friendly API design\n- **Auto-indexing**: the first query triggers indexing automatically; incremental updates are transparent\n\n### ⚡ Query Cache & File Watching (v1.5.0+)\n- **Query cache (QueryCache)**: in-process per-project LRU cache (50 entries by default); a hit skips the entire vector recall / rerank / expansion pipeline\n- **Automatic cache invalidation**: the cache key is composed of `normalized query + projectId + index version + search-config fingerprint`, so it invalidates automatically after an index update or config change — stale results are never returned\n- **Watch mode**: `contextweaver watch` watches the filesystem and triggers incremental indexing automatically, with debouncing (500ms by default) and scan de-duplication (no concurrent scans)\n\n### 📈 Statistics & Observability (v1.5.0+)\n- **Three metric groups**: indexing process, search quality/behavior, health/consistency\n- **Dual exits**: `contextweaver stats` CLI (with `--json`) plus the MCP `stats` tool\n- **Consistency diagnostics**: automatically detects abnormal migration state, `pending_marks` backlog, missing vector rows, and more — with suggested fixes\n\n### 🛡️ Crash-Safe Data Architecture (v1.4.0+)\n- **Single source of truth for content**: LanceDB stores only vectors and locating metadata; content is read back from `files.content`, reducing index size by 30–50%\n- **Cross-store transactional compensation**: three-stage write LanceDB → FTS+outbox → SQLite mark, with automatic rollback or replay on any failure\n- **Migration state machine**: `pending/done/aborted` persisted, auto-rebuilt on crash recovery\n- **Cross-process mutual exclusion**: an advisory lock prevents the MCP server and CLI from triggering LanceDB migration concurrently\n- **chunk_id de-duplication**: pre-delete before write to avoid duplicate rows on retry\n\n## 📦 Quick Start\n\n### Requirements\n\n- Node.js >= 20\n- pnpm (recommended) or npm\n\n### Installation\n\n```bash\n# Global install\nnpm install -g @chiway/contextweaver\n\n# Or with pnpm\npnpm add -g @chiway/contextweaver\n```\n\n### Initialize Configuration\n\n```bash\n# Create the config file (~/.contextweaver/.env)\ncontextweaver init\n# Or the short alias\ncw init\n```\n\nEdit `~/.contextweaver/.env` and fill in your API keys:\n\n```bash\n# Embedding API config (required)\nEMBEDDINGS_API_KEY=your-api-key-here\nEMBEDDINGS_BASE_URL=https://api.siliconflow.cn/v1/embeddings\nEMBEDDINGS_MODEL=BAAI/bge-m3\nEMBEDDINGS_MAX_CONCURRENCY=10\nEMBEDDINGS_DIMENSIONS=1024\n\n# Reranker config (required)\nRERANK_API_KEY=your-api-key-here\nRERANK_BASE_URL=https://api.siliconflow.cn/v1/rerank\nRERANK_MODEL=BAAI/bge-reranker-v2-m3\nRERANK_TOP_N=20\n\n# Search parameters (optional, override built-in defaults)\nCW_SEARCH_WVEC=0.6\nCW_SEARCH_WLEX=0.4\nCW_SEARCH_RERANK_TOP_N=10\nCW_SEARCH_MAX_TOTAL_CHARS=48000\nCW_SEARCH_VECTOR_TOP_K=80\nCW_SEARCH_SMART_MAX_K=8\nCW_SEARCH_IMPORT_FILES_PER_SEED=3\n\n# Ignore patterns (optional, comma-separated)\n# IGNORE_PATTERNS=.venv,node_modules\n```\n\n### Index a Codebase\n\n```bash\n# Run from the codebase root\ncontextweaver index\n\n# Specify a path\ncontextweaver index /path/to/your/project\n\n# Force a full re-index\ncontextweaver index --force\n```\n\n### Watch Mode (v1.5.0+)\n\n```bash\n# Watch for file changes and auto-index incrementally (Ctrl+C to stop)\ncontextweaver watch\n\n# Specify a path and debounce window (ms)\ncontextweaver watch /path/to/project --debounce 800\n```\n\n`watch` runs one full incremental scan on startup, then listens to filesystem events; changes trigger a de-duplicated scan within the debounce window, and paths excluded by ignore rules never trigger a scan.\n\n### Local Search\n\n```bash\n# Semantic search\ncw search --information-request \"How is the user authentication flow implemented?\"\n\n# With exact terms\ncw search --information-request \"Database connection logic\" --technical-terms \"DatabasePool,Connection\"\n```\n\n### Structure Browsing & Symbol Lookup (v1.5.0+)\n\nThe following commands are CLI mirrors of MCP tools, with zero Embedding API cost:\n\n```bash\n# List indexed files (supports glob / language / count filters)\ncontextweaver list-files --glob \"src/**/*.ts\" --language typescript --max-results 100\n\n# Look up a symbol definition\ncontextweaver definition SearchService --hint-path src/search\n\n# Look up symbol references\ncontextweaver references handleStats --exclude-definition\n```\n\n### Statistics (v1.5.0+)\n\n```bash\n# Human-readable stats report\ncontextweaver stats\n\n# JSON output (for scripting)\ncontextweaver stats --json\n\n# Specify a project path\ncontextweaver stats --path /path/to/project\n```\n\n### Start the MCP Server\n\n```bash\n# Launch the MCP server (for use by Claude and other AI assistants)\ncontextweaver mcp\n```\n\n### Index Management (v1.4.0+)\n\n```bash\n# Show LanceDB migration state\ncontextweaver migrate\n\n# Clear the aborted state: wipe LanceDB and trigger a full rebuild\n# Triggered when: the Indexer refuses to write after sampling validation fails;\n# run this, then index again.\ncontextweaver migrate --reset\n\n# Specify a project path\ncontextweaver migrate --path /path/to/project\n```\n\n## 🔧 MCP Integration\n\n### Claude Desktop Configuration\n\nAdd the following to your Claude Desktop config file:\n\n```json\n{\n  \"mcpServers\": {\n    \"contextweaver\": {\n      \"command\": \"contextweaver\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\n### MCP Tools Overview (v1.5.0+)\n\nContextWeaver exposes 5 MCP tools, following a layered design of \"semantic retrieval first, structure browsing second\":\n\n| Tool | Purpose | Embedding cost |\n|------|---------|----------------|\n| `codebase-retrieval` | **Primary tool**: hybrid semantic + exact-match retrieval | Yes |\n| `list-files` | List indexed file structure (path/language/size) | No |\n| `find-references` | Find heuristic text references to a symbol | No |\n| `get-symbol-definition` | Find likely definition blocks for a symbol | No |\n| `stats` | Index/search/health statistics | No |\n\n#### `codebase-retrieval` Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `repo_path` | string | ✅ | Absolute path to the repository root |\n| `information_request` | string | ✅ | The semantic intent in natural language |\n| `technical_terms` | string[] | ❌ | Exact technical terms (class/function names, etc.) |\n| `mode` | string | ❌ | Retrieval profile: `quick`, `balanced`, or `deep` |\n| `include_globs` | string[] | ❌ | File glob allowlist applied after retrieval |\n| `exclude_globs` | string[] | ❌ | File glob denylist applied after retrieval |\n| `language` | string[] | ❌ | Language allowlist applied after retrieval |\n| `max_total_chars` | number | ❌ | Per-call output budget in characters |\n| `max_files` | number | ❌ | Maximum number of files returned after packing |\n| `max_segments_per_file` | number | ❌ | Maximum non-contiguous segments per file |\n| `return_debug` | boolean | ❌ | Include debug metadata in structured output |\n| `low_confidence_behavior` | string | ❌ | Low-confidence handling: `return_top1`, `return_empty`, or `return_with_warning` |\n| `output_format` | string | ❌ | Response format: `markdown`, `json`, or `both` |\n\n#### `list-files` Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `repo_path` | string | ✅ | Absolute path to the repository root |\n| `glob` | string | ❌ | Glob pattern to filter paths |\n| `language` | string | ❌ | Language filter (matched against `files.language`) |\n| `max_results` | number | ❌ | Max files to return (default 200) |\n\n#### `find-references` Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `repo_path` | string | ✅ | Absolute path to the repository root |\n| `symbol` | string | ✅ | Exact symbol name |\n| `exclude_definition` | boolean | ❌ | Exclude chunks whose breadcrumb tail matches the symbol name |\n| `max_results` | number | ❌ | Max references to return (default 50) |\n\n#### `get-symbol-definition` Parameters\n\n| Parameter | Type | Required | Description |\n|-----------|------|----------|-------------|\n| `repo_path` | string | ✅ | Absolute path to the repository root |\n| `symbol` | string | ✅ | Exact symbol name to resolve |\n| `hint_path` | string | ❌ | Preferred path to disambiguate same-name definitions |\n| `max_results` | number | ❌ | Max definitions to return (default 3) |\n\n> **Note**: `find-references` and `get-symbol-definition` are heuristic text lookups over indexed chunks, not compiler-accurate navigation. For exhaustive raw text matching, use `grep` outside MCP.\n\n#### Design Philosophy\n\n- **Intent/term separation**: `information_request` describes \"what to do\", `technical_terms` filters \"what it's called\"\n- **Same-file context first**: same-file context is provided by default; cross-file exploration is initiated by the agent\n- **Return to agent instincts**: the tool only locates; cross-file exploration is triggered by the agent on demand\n\n## 🏗️ Architecture\n\n```mermaid\nflowchart TB\n    subgraph Interface[\"CLI / MCP Interface\"]\n        CLI[contextweaver CLI]\n        MCP[MCP Server]\n    end\n\n    subgraph Search[\"SearchService\"]\n        QC[QueryCache<br/>LRU]\n        VR[Vector Retrieval]\n        LR[Lexical Retrieval]\n        RRF[RRF Fusion + Rerank]\n        QC -.cache hit.-> CP\n        VR --> RRF\n        LR --> RRF\n    end\n\n    subgraph Expand[\"Context Expansion\"]\n        GE[GraphExpander]\n        CP[ContextPacker]\n        GE --> CP\n    end\n\n    subgraph Storage[\"Storage Layer\"]\n        VS[(VectorStore<br/>LanceDB)]\n        DB[(SQLite<br/>FTS5)]\n    end\n\n    subgraph Index[\"Indexing Pipeline\"]\n        CR[Crawler<br/>fdir] --> SS[SemanticSplitter<br/>Tree-sitter] --> IX[Indexer<br/>Batch Embedding]\n    end\n\n    Interface --> Search\n    RRF --> GE\n    Search <--> Storage\n    Expand <--> Storage\n    Index --> Storage\n```\n\n### Core Modules\n\n| Module | Responsibility |\n|--------|----------------|\n| **SearchService** | Hybrid search core: coordinates vector/lexical recall, RRF fusion, rerank; integrates QueryCache |\n| **QueryCache** | Per-project in-process LRU cache (v1.5.0+); a hit skips the entire retrieval pipeline |\n| **GraphExpander** | Context expander: runs the E1/E2/E3 three-stage expansion strategy |\n| **ContextPacker** | Context packer: segment merging and token budget control |\n| **ChunkContentLoader** | Slices `files.content` by `(path, start_index, end_index)` (v1.4.0+) |\n| **VectorStore** | LanceDB adapter; exposes pure vector operations only |\n| **Database (SQLite)** | Metadata storage + FTS5 full-text index + statistics counters, schema_version=3 |\n| **Bootstrap** | Cross-store init coordinator: pending_marks replay + LanceDB schema migration (v1.4.0+) |\n| **SemanticSplitter** | AST semantic chunker (Tree-sitter); normalizes offsets to the UTF-16 character domain on write |\n| **Watcher** | File-watch coordinator (v1.5.0+): debounce + scan de-duplication + ignore filtering |\n| **Stats** | Statistics aggregation layer (v1.5.0+): combines index/search/health metrics |\n\n### Data Architecture (v1.4.0+)\n\n```\n~/.contextweaver/<projectId>/\n├── index.db                 # SQLite\n│   ├── files                # File metadata + full content (content column, the only source for text slicing)\n│   ├── files_fts            # External-content table, inverted index pointing to files\n│   ├── chunks_fts           # Chunk-level inverted index, per-file wholesale replacement\n│   ├── metadata             # schema_version / lancedb_migration_state / lock\n│   ├── stats                # Cumulative index/search counters (v1.5.0+)\n│   └── pending_marks        # Outbox: replayed when a vector_index_hash mark failed\n└── vectors.lance/           # LanceDB chunks table (vectors + locating metadata only, no content)\n```\n\n**Key invariants**:\n- The single source of truth for content is `files.content`; `ChunkContentLoader` slices via `start_index/end_index` (same source as `displayCode`)\n- All LanceDB offset fields live in the UTF-16 character domain; multi-byte files are never sliced incorrectly\n- Cross-store write order: LanceDB → (FTS + outbox single transaction) → SQLite mark + clear outbox\n- LanceDB migration state `pending/done/aborted` is persisted, with cross-process mutual exclusion via an advisory lock\n- The query cache key is bound to the index version and search-config fingerprint; it invalidates on any index or config change\n\n## 📁 Project Structure\n\n```\ncontextweaver/\n├── src/\n│   ├── index.ts              # CLI entry (init / index / watch / search / mcp / migrate / stats)\n│   ├── config.ts             # Config management (environment variables)\n│   ├── defaultEnv.ts         # Default .env template\n│   ├── cli/\n│   │   └── mirrorCommands.ts # CLI mirrors of MCP tools (list-files / definition / references)\n│   ├── api/                  # External API wrappers\n│   │   ├── embedding.ts      # Embedding API\n│   │   └── reranker.ts       # Reranker API\n│   ├── chunking/             # Semantic chunking\n│   │   ├── SemanticSplitter.ts   # AST semantic chunker\n│   │   ├── SourceAdapter.ts      # Source adapter (UTF-16/UTF-8 domain normalization)\n│   │   ├── LanguageSpec.ts       # Language spec definitions\n│   │   ├── ParserPool.ts         # Tree-sitter parser pool\n│   │   └── types.ts              # Chunking type definitions\n│   ├── scanner/              # File scanning\n│   │   ├── index.ts          # Scan orchestration\n│   │   ├── crawler.ts        # Filesystem traversal\n│   │   ├── processor.ts      # File processing\n│   │   ├── watcher.ts        # File-watch coordinator (v1.5.0+)\n│   │   ├── filter.ts         # Filter rules\n│   │   ├── hash.ts           # File hash\n│   │   └── language.ts       # Language detection\n│   ├── indexer/              # Indexer\n│   │   └── index.ts          # Three-stage transaction (LanceDB → FTS+outbox → SQLite mark)\n│   ├── vectorStore/          # Vector storage\n│   │   └── index.ts          # LanceDB adapter (pure vector operations)\n│   ├── db/                   # Database\n│   │   ├── index.ts          # SQLite + FTS5 + pending_marks + migration state machine + stats counters\n│   │   └── bootstrap.ts      # Cross-store init coordinator (v1.4.0+)\n│   ├── search/               # Search service\n│   │   ├── SearchService.ts      # Core search service (cache-integrated)\n│   │   ├── QueryCache.ts         # Per-project LRU query cache (v1.5.0+)\n│   │   ├── GraphExpander.ts      # Context expander\n│   │   ├── ContextPacker.ts      # Context packer\n│   │   ├── ChunkContentLoader.ts # Slices by (path, start_index, end_index) (v1.4.0+)\n│   │   ├── fts.ts                # Full-text search (per-file wholesale replacement)\n│   │   ├── config.ts             # Search default config + value bounds\n│   │   ├── loadConfig.ts         # Env-var overrides + config fingerprint (v1.5.0+)\n│   │   ├── types.ts              # Type definitions\n│   │   ├── utils.ts              # Token-overlap scoring\n│   │   └── resolvers/            # Multi-language import resolvers\n│   │       ├── JsTsResolver.ts\n│   │       ├── PythonResolver.ts\n│   │       ├── GoResolver.ts\n│   │       ├── JavaResolver.ts\n│   │       ├── RustResolver.ts\n│   │       ├── CppResolver.ts\n│   │       └── CSharpResolver.ts\n│   ├── stats/                # Statistics aggregation layer (v1.5.0+)\n│   │   └── index.ts          # Aggregates and renders index/search/health metrics\n│   ├── mcp/                  # MCP server\n│   │   ├── server.ts         # MCP server implementation (registers 5 tools)\n│   │   ├── main.ts           # MCP entry\n│   │   └── tools/\n│   │       ├── index.ts                 # Tool registry\n│   │       ├── shared.ts                # Shared tool logic\n│   │       ├── codebaseRetrieval.ts     # Code retrieval tool\n│   │       ├── listFiles.ts             # File structure browsing (v1.5.0+)\n│   │       ├── findReferences.ts        # Symbol reference lookup (v1.5.0+)\n│   │       ├── getSymbolDefinition.ts   # Symbol definition lookup (v1.5.0+)\n│   │       └── stats.ts                 # Statistics tool (v1.5.0+)\n│   └── utils/                # Utilities\n│       ├── logger.ts         # Logging system\n│       ├── encoding.ts       # Encoding detection\n│       └── lock.ts           # File lock\n├── tests/                    # Unit + integration tests (28 test files, 156 test cases)\n│   ├── chunking/             # SourceAdapter / chunking\n│   ├── cli/                  # mirrorCommands\n│   ├── db/                   # migration, outbox, advisory lock, index-version\n│   ├── indexer/              # transaction compensation, GC, aborted guard\n│   ├── integration/          # real LanceDB end-to-end\n│   ├── mcp/                  # list-files / find-references / get-symbol-definition / shared / tool registry\n│   ├── scanner/              # watcher / index-version\n│   ├── search/               # FTS, ChunkContentLoader, Packer, cache, loadConfig\n│   ├── stats/                # statistics aggregation\n│   └── vectorStore/          # chunk_id de-duplication, sampling validation\n├── package.json\n└── tsconfig.json\n```\n\n## ⚙️ Configuration Reference\n\n### Environment Variables\n\n| Variable | Required | Default | Description |\n|----------|----------|---------|-------------|\n| `EMBEDDINGS_API_KEY` | ✅ | - | Embedding API key |\n| `EMBEDDINGS_BASE_URL` | ✅ | - | Embedding API URL |\n| `EMBEDDINGS_MODEL` | ✅ | - | Embedding model name |\n| `EMBEDDINGS_MAX_CONCURRENCY` | ❌ | 10 | Embedding concurrency |\n| `EMBEDDINGS_DIMENSIONS` | ❌ | 1024 | Vector dimensions |\n| `RERANK_API_KEY` | ✅ | - | Reranker API key |\n| `RERANK_BASE_URL` | ✅ | - | Reranker API URL |\n| `RERANK_MODEL` | ✅ | - | Reranker model name |\n| `RERANK_TOP_N` | ❌ | 20 | Rerank return count |\n| `IGNORE_PATTERNS` | ❌ | - | Extra ignore patterns |\n\n### Search Parameter Env Overrides (v1.5.0+)\n\nThe following environment variables override built-in defaults; out-of-range values are automatically clamped to the valid interval. When only one of `wVec`/`wLex` is set, the other is automatically set to `1 - x`.\n\n| Variable | Default | Bounds | Description |\n|----------|---------|--------|-------------|\n| `CW_SEARCH_WVEC` | 0.6 | 0–1 | Vector weight (fusion stage) |\n| `CW_SEARCH_WLEX` | 0.4 | 0–1 | Lexical weight (complements `wVec`) |\n| `CW_SEARCH_RERANK_TOP_N` | 10 | 5–20 | Results kept after rerank |\n| `CW_SEARCH_MAX_TOTAL_CHARS` | 48000 | 20000–80000 | Token budget (in chars, ~12k tokens) |\n| `CW_SEARCH_VECTOR_TOP_K` | 80 | 40–200 | Vector recall candidates |\n| `CW_SEARCH_SMART_MAX_K` | 8 | 5–15 | Smart TopK hard upper bound |\n| `CW_SEARCH_IMPORT_FILES_PER_SEED` | 3 | 0–5 | E3 import files resolved per seed (0 disables cross-file expansion) |\n\n### Search Config Parameters (built-in defaults)\n\n```typescript\ninterface SearchConfig {\n  // === Recall ===\n  vectorTopK: number;        // Vector recall candidates (default 80)\n  vectorTopM: number;        // Vectors kept after dedup (default 60)\n  ftsTopKFiles: number;      // FTS recall file count (default 20)\n  lexChunksPerFile: number;  // Lexical chunks per file (default 2)\n  lexTotalChunks: number;    // Total lexical chunks (default 40)\n\n  // === Fusion ===\n  rrfK0: number;             // RRF smoothing constant (default 20)\n  wVec: number;              // Vector weight (default 0.6)\n  wLex: number;              // Lexical weight (default 0.4)\n  fusedTopM: number;         // Candidates fed into rerank after fusion (default 60)\n\n  // === Rerank ===\n  rerankTopN: number;        // Results kept after rerank (default 10)\n  maxRerankChars: number;    // Max chars per chunk sent to reranker (default 1000)\n  maxBreadcrumbChars: number;// Max chars for breadcrumb context (default 250)\n  headRatio: number;         // Head/tail ratio when truncating (default 0.67)\n\n  // === Expansion ===\n  neighborHops: number;      // E1 neighbor hops (default 2)\n  breadcrumbExpandLimit: number;  // E2 breadcrumb completions (default 3)\n  importFilesPerSeed: number;     // E3 import files per seed (default 3)\n  chunksPerImportFile: number;    // E3 chunks per import file (default 3)\n\n  // === ContextPacker ===\n  maxSegmentsPerFile: number;     // Max non-contiguous segments per file (default 3)\n  maxTotalChars: number;          // Token budget (chars, default 48000)\n\n  // === Smart TopK ===\n  enableSmartTopK: boolean;       // Enable smart cutoff (default true)\n  smartTopScoreRatio: number;     // Dynamic threshold ratio (default 0.5)\n  smartTopScoreDeltaAbs: number;  // Max absolute drop from Top1 (default 0.25)\n  smartMinScore: number;          // Absolute floor (default 0.25)\n  smartMinK: number;              // Safe Harbor count (default 2)\n  smartMaxK: number;              // Hard upper bound (default 8)\n}\n```\n\n## 🌍 Multi-Language Support\n\nContextWeaver natively supports AST parsing for the following languages via Tree-sitter:\n\n| Language | AST Parsing | Import Resolution | Extensions |\n|----------|-------------|-------------------|------------|\n| TypeScript | ✅ | ✅ | `.ts`, `.tsx` |\n| JavaScript | ✅ | ✅ | `.js`, `.jsx`, `.mjs`, `.cjs` |\n| Python | ✅ | ✅ | `.py` |\n| Go | ✅ | ✅ | `.go` |\n| Java | ✅ | ✅ | `.java` |\n| Rust | ✅ | ✅ | `.rs` |\n| C | ✅ | ✅ | `.c`, `.h` |\n| C++ | ✅ | ✅ | `.cpp`, `.cc`, `.cxx`, `.hpp` |\n| C# | ✅ | ✅ | `.cs` |\n\nOther languages fall back to line-based chunking and can still be indexed and searched normally.\n\n## 🔄 Workflows\n\n### Indexing Flow\n\n```\n0. Bootstrap   → pending_marks replay + LanceDB schema migration (first launch)\n1. Crawler     → traverse the filesystem, filter ignored items\n2. Processor   → read file content, compute hash\n3. Splitter    → AST parse, semantic chunking (offsets normalized to UTF-16 char domain)\n4. Indexer     → batch embedding\n5. Stages 4-6 pseudo-transaction:\n   ├─ LanceDB write (pre-delete (path, hash) to avoid duplicates → add → clear old versions)\n   ├─ FTS + outbox single SQLite transaction (rolls back LanceDB on failure)\n   └─ SQLite mark + clear outbox single transaction (outbox kept on failure, replayed next launch)\n6. Trailing GC → clean up LanceDB orphan chunks (time budget 5s)\n```\n\n### Search Flow\n\n```\n1. Query Parse     → parse the query, separate semantics from terms\n2. Cache Lookup    → return immediately on hit (v1.5.0+, key includes index version + config fingerprint)\n3. Hybrid Recall   → dual-channel vector + lexical recall\n4. RRF Fusion      → Reciprocal Rank Fusion\n5. Rerank          → cross-encoder reranking\n6. Smart Cutoff    → intelligent score cutoff\n7. Graph Expand    → neighbor/breadcrumb/import expansion\n8. Context Pack    → segment merging, token budget\n9. Cache Store     → write to cache (v1.5.0+)\n10. Format Output  → format and return to the LLM\n```\n\n## 📊 Performance Characteristics\n\n- **Query cache**: repeated queries hit the LRU cache, skipping the entire recall/rerank/expansion pipeline (v1.5.0+)\n- **Incremental indexing**: only changed files are processed; re-indexing is 10x+ faster\n- **Batch embedding**: adaptive batch size with concurrency control\n- **Rate-limit recovery**: automatic backoff on 429 errors, gradual recovery\n- **Connection pool reuse**: pooled Tree-sitter parsers\n- **File index caching**: lazy-loaded file-path index in GraphExpander\n- **Zero-cost metadata tools**: `list-files`/`find-references`/`get-symbol-definition` do not call the Embedding API (v1.5.0+)\n\n## 📈 Statistics & Observability (v1.5.0+)\n\n`contextweaver stats` outputs three sections:\n\n- **Indexing process**: cumulative index run count, last index time, last-run snapshot (added/modified/deleted/unchanged/skipped/errors + vector index details)\n- **Search quality/behavior**: cumulative queries, cache hit rate, actual compute runs, plus average per-stage latency (retrieve / rerank / expand / pack) and average recalled seed count\n- **Health/consistency**: file count and total content size, LanceDB vector row count, embedding dimensions, index version, migration state, `pending_marks`, language breakdown\n\nWhen an abnormal migration state, `pending_marks` backlog, or missing vector rows are detected, the report appends **diagnostic warnings** with the corresponding fix commands. The `--json` output maps to `StatsReport` for scripts and monitoring systems.\n\n## 🐛 Logging & Debugging\n\nLog file location: `~/.contextweaver/logs/app.YYYY-MM-DD.log`\n\nSet the log level:\n\n```bash\n# Enable debug logging\nLOG_LEVEL=debug contextweaver search --information-request \"...\"\n```\n\n## 🚨 Troubleshooting (v1.4.0+)\n\n### LanceDB Migration Stuck (`aborted` state)\n\n**Symptom**: `contextweaver index` errors with \"LanceDB is in the aborted state, refusing to write to prevent schema pollution.\"\n\n**Cause**: during the v1.4.0 upgrade, the old LanceDB index's `display_code` differs from the current `files.content` by >1% on sampling (typically on legacy indexes whose chunk offsets used the UTF-8 byte domain).\n\n**Fix**:\n```bash\ncontextweaver migrate --reset   # Clear the LanceDB chunks table + reset state to done\ncontextweaver index             # Full rebuild (new schema)\n```\n\nYou can also run `contextweaver stats` first to view diagnostic warnings and confirm the current migration state and `pending_marks` backlog.\n\n### Cross-Process Migration Race\n\nIf the MCP server is long-running and another terminal runs `contextweaver index`, the two processes contend for migration. v1.4.0 introduces an advisory lock with a 10-minute zombie threshold, automatically letting one process skip migration while the other completes it.\n\nIf the lock gets stuck (after `kill -9`), clear it manually:\n```bash\nsqlite3 ~/.contextweaver/<projectId>/index.db \\\n  \"DELETE FROM metadata WHERE key = 'lancedb_migration_lock';\"\n```\n\n### Wasted Duplicate Embeddings\n\nv1.4.0 solves this via the `pending_marks` outbox: when an FTS write succeeds but the vector_index_hash mark fails, it is replayed automatically on the next launch, avoiding duplicate embeddings.\n\n### Search Results Don't Reflect Recent Changes\n\nConfirm incremental indexing has run (or enable `contextweaver watch` for automatic increments). The query cache key is bound to the index version, so old cache entries invalidate automatically after an index update — no manual clearing needed.\n\n## 📄 License\n\nThis project is licensed under the MIT License.\n\n## 🙏 Acknowledgements\n\n- [Tree-sitter](https://tree-sitter.github.io/tree-sitter/) - high-performance syntax parsing\n- [LanceDB](https://lancedb.com/) - embedded vector database\n- [MCP](https://modelcontextprotocol.io/) - Model Context Protocol\n- [Source](https://github.com/hsingjui/ContextWeaver) - Source repository\n---\n\n<p align=\"center\">\n  <sub>Made with ❤️ for AI-assisted coding</sub>\n</p>\n",
  "bytes": 30317,
  "sha": "751037750169397449d598266ebe55f73fe2d3b8eaf52c9c7f8d43d7821df201",
  "repo_slug": "wchiway/contextweaver-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_wchiway_contextweaver_8e02c63e/readme"
}