{
"markdown": "# docsonar\n\n\n\n[](https://pypi.org/project/docsonar/)\n[](https://github.com/ribhav-jain/docsonar/actions/workflows/ci.yml)\n[](https://pypi.org/project/docsonar/)\n[](LICENSE)\n\n**Chat with your folders.** A local document search MCP server — your files never leave your machine.\n\ndocsonar indexes folders of documents into a single SQLite database and gives any MCP client (Claude Desktop, Claude Code, and others) hybrid keyword + semantic search over them. Fully offline, zero configuration, read-only by design.\n\n## Quickstart\n\n### Claude Code\n\nAdd to your project's `.mcp.json` (or `~/.claude.json` for all projects):\n\n```json\n{\n \"mcpServers\": {\n \"docsonar\": {\n \"command\": \"uvx\",\n \"args\": [\"docsonar\"]\n }\n }\n}\n```\n\n### Claude Desktop\n\nAdd to `claude_desktop_config.json` (Settings → Developer → Edit Config):\n\n```json\n{\n \"mcpServers\": {\n \"docsonar\": {\n \"command\": \"uvx\",\n \"args\": [\"docsonar\"]\n }\n }\n}\n```\n\nRunning from a source checkout instead: `\"command\": \"uv\", \"args\": [\"run\", \"--directory\", \"/path/to/docsonar\", \"docsonar\"]`.\n\nThen just talk to it: _\"Add my `~/Documents/notes` folder and find everything about quarterly planning.\"_ The first index downloads the embedding model (~130 MB, one time); keyword search works immediately while that happens.\n\nFor HTTP instead of stdio: `uvx docsonar --transport http --port 8365`.\n\n## Tools\n\n| Tool | Purpose |\n| --------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |\n| `add_folder(path, include_globs?, exclude_globs?)` | Register a folder; indexing starts in the background |\n| `remove_folder(path)` | Unregister and purge its index data |\n| `list_folders()` | Registered folders with counts and last index time |\n| `search(query, top_k?, folder?, file_type?, mode?)` | Hybrid keyword+semantic search (RRF-fused); ranked passages with file path, heading/page location, and snippet |\n| `read_file(path, start_line?, end_line?)` | File text (extracted text for pdf/docx/html); refuses paths outside registered folders |\n| `find_similar(path, top_k?)` | Documents most similar in meaning to a given file |\n| `reindex(folder?, force?)` | Incremental refresh: new/changed files reindexed, deleted files purged |\n| `index_status()` | Index totals, embedding model state, background job progress, failed files |\n\nSupported formats: `txt`, `md`, `pdf` (page-aware, cites `p. 12`), `docx`, `html`.\n\n## Architecture\n\n```mermaid\nflowchart LR\n Client[\"MCP client
(Claude Desktop / Code)\"] <-->|stdio or HTTP| Server[\"FastMCP server
8 tools\"]\n Server --> Sec[\"path security
(registered folders only)\"]\n Server --> Search[\"hybrid search
BM25 + cosine KNN + RRF\"]\n Server --> Worker[\"background indexer
(worker thread + queue)\"]\n Worker --> Parsers[\"parsers
txt · md · pdf · docx · html\"]\n Parsers --> Chunker[\"heading/page-aware chunker
~400 tokens, 60 overlap\"]\n Chunker --> DB[(\"SQLite
FTS5 + sqlite-vec\")]\n Search --> DB\n Embed[\"sentence-transformers
bge-small-en-v1.5 (local)\"] --> Worker\n Embed --> Search\n```\n\nEvery chunk stores its location (heading path like `Setup > Windows`, or PDF page range), so search results can cite `report.pdf, p. 12`.\n\n## Design decisions\n\n- **Read-only by design.** There are no write, move, or delete tools, and there never will be. The server only reads files inside folders you explicitly register — with symlink-escape protection and strict path resolution — so the blast radius of a misbehaving client is zero.\n- **SQLite as the single store.** FTS5 gives production-grade BM25 keyword search in the standard library, and [sqlite-vec](https://github.com/asg017/sqlite-vec) puts vectors in the same file. One database file, no services to run, trivial to back up or delete.\n- **Hybrid search by default.** BM25 and cosine-KNN rankings are fused with reciprocal-rank fusion (k=60). Exact identifiers and rare terms win on the keyword side; paraphrased questions win on the semantic side; RRF needs no score calibration between them. `mode` lets the caller force either side.\n- **Local embeddings.** [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) (384-dim) — same size class as the classic all-MiniLM-L6-v2 but stronger on retrieval benchmarks. Downloads on first index, runs on CPU, lazy-loaded so server startup stays instant. If the model can't load, everything degrades gracefully to keyword search and tool responses say so.\n- **A tool surface built for an LLM caller.** `search` returns enough per hit (path, location, score, snippet) to decide what to read next without another round trip; results carry stable `chunk_id`s; every degradation is reported in-band via `note`/`embedding_note` fields instead of failing. There's no \"answer the question\" tool on purpose — the calling model does the reasoning; docsonar does retrieval.\n- **Incremental by content, not just mtime.** Reindexing checks mtime+size first, then falls back to a SHA-256 content hash — touched-but-identical files are skipped, and deleted files are purged from both the FTS and vector indexes.\n- **Scanned PDFs fail loudly.** A PDF with no extractable text is reported as failed with a clear reason rather than silently indexed as empty. OCR is out of scope.\n\n## Benchmarks\n\nSynthetic corpus of 200 markdown files (600 chunks); Intel Core Ultra 5 125U (laptop, CPU-only), Windows 11, Python 3.12. Reproduce with `uv run python scripts/benchmark.py`.\n\n| Operation | Result |\n| --------------------------------------- | ------------------------------------- |\n| Index, keyword-only | 0.7 s (≈280 files/s) |\n| Index, with embeddings | 59 s (≈3.4 files/s — embedding-bound) |\n| Incremental reindex, nothing changed | 0.03 s |\n| Search, keyword | 8.9 ms median |\n| Search, semantic | 35 ms median |\n| Search, hybrid | 54 ms median |\n| Embedding model load (once per process) | ~21 s |\n| Database size | 2.6 MB (1.1 MB keyword-only) |\n\n## Configuration\n\nZero config needed. To customize, create `config.toml` in the platform config directory (Windows: `%LOCALAPPDATA%\\docsonar\\`, macOS: `~/Library/Application Support/docsonar/`, Linux: `~/.config/docsonar/`):\n\n```toml\nembedding_model = \"BAAI/bge-small-en-v1.5\" # any sentence-transformers model\nchunk_target_tokens = 400\nchunk_max_tokens = 512\nchunk_min_tokens = 100\nchunk_overlap_tokens = 60\nmax_file_size_mb = 50\nextra_ignore_dirs = [\"Archive\"]\n```\n\nCLI flags: `--transport stdio|http`, `--host`, `--port`, `--db-path`, `--config`.\n\nThe index database lives in the platform data directory (Windows: `%LOCALAPPDATA%\\docsonar\\`, macOS: `~/Library/Application Support/docsonar/`, Linux: `~/.local/share/docsonar/`).\n\n## Running from source\n\nRequires [uv](https://docs.astral.sh/uv/) (it installs the right Python automatically):\n\n```sh\n# Windows\npowershell -ExecutionPolicy ByPass -c \"irm https://astral.sh/uv/install.ps1 | iex\"\n# macOS / Linux\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\ngit clone https://github.com/ribhav-jain/docsonar.git\ncd docsonar\nuv sync --dev # creates .venv and installs everything, incl. dev tools\n```\n\n### Run the tests\n\n```sh\nuv run pytest # full suite (~2 s, no model download needed)\nuv run pytest -v # verbose, one line per test\nuv run pytest tests/test_search.py # one file\nuv run pytest -k incremental # tests matching a keyword\nuv run ruff check . # lint\nuv run mypy src tests # strict typecheck\nuv run python scripts/benchmark.py # perf numbers (downloads the real model)\n```\n\n### Run the server\n\n```sh\nuv run docsonar # stdio — for MCP clients (Claude Desktop/Code)\nuv run docsonar --transport http --port 8365 # HTTP — for manual testing (Postman, curl)\n```\n\nVS Code: press F5 — launch configs for the HTTP server and the test suite are in `.vscode/launch.json`.\n\n### Try the tools without an MCP client\n\nWith the HTTP server running, the endpoint is `http://127.0.0.1:8365/mcp`. Recent Postman versions can connect directly (**New → MCP Request**, transport HTTP) and show all 8 tools as forms. For raw HTTP, MCP is JSON-RPC over POST with two setup calls, then tool calls. Send every request with headers `Content-Type: application/json` and `Accept: application/json, text/event-stream`:\n\n```jsonc\n// 1. initialize — copy the `mcp-session-id` RESPONSE header and send it\n// back as a request header on every call below\n{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"2025-03-26\",\"capabilities\":{},\"clientInfo\":{\"name\":\"postman\",\"version\":\"1.0\"}}}\n\n// 2. initialized notification (expect 202, empty body)\n{\"jsonrpc\":\"2.0\",\"method\":\"notifications/initialized\"}\n\n// 3. list the tools\n{\"jsonrpc\":\"2.0\",\"id\":2,\"method\":\"tools/list\"}\n```\n\nThen call tools — the repo ships a sample corpus in `examples/sample-docs` to play with (use forward slashes in JSON to avoid escaping; they work on Windows too):\n\n```jsonc\n// Register the sample folder (first ever call downloads the embedding model, ~30 s)\n{\"jsonrpc\":\"2.0\",\"id\":3,\"method\":\"tools/call\",\"params\":{\"name\":\"add_folder\",\n \"arguments\":{\"path\":\"C:/path/to/docsonar/examples/sample-docs\"}}}\n\n// Watch indexing progress until indexing.active is false\n{\"jsonrpc\":\"2.0\",\"id\":4,\"method\":\"tools/call\",\"params\":{\"name\":\"index_status\",\"arguments\":{}}}\n\n// Hybrid search — finds the expense-policy PDF, cites its page\n{\"jsonrpc\":\"2.0\",\"id\":5,\"method\":\"tools/call\",\"params\":{\"name\":\"search\",\n \"arguments\":{\"query\":\"meal reimbursement limit\",\"top_k\":3,\"mode\":\"hybrid\"}}}\n\n// Semantic search — no keyword overlap needed\n{\"jsonrpc\":\"2.0\",\"id\":6,\"method\":\"tools/call\",\"params\":{\"name\":\"search\",\n \"arguments\":{\"query\":\"how hot should water be for green tea\",\"mode\":\"semantic\"}}}\n\n// Read a hit (works on PDFs too — returns extracted text)\n{\"jsonrpc\":\"2.0\",\"id\":7,\"method\":\"tools/call\",\"params\":{\"name\":\"read_file\",\n \"arguments\":{\"path\":\"C:/path/to/docsonar/examples/sample-docs/expense-policy.pdf\"}}}\n\n// \"More like this\"\n{\"jsonrpc\":\"2.0\",\"id\":8,\"method\":\"tools/call\",\"params\":{\"name\":\"find_similar\",\n \"arguments\":{\"path\":\"C:/path/to/docsonar/examples/sample-docs/espresso-guide.md\",\"top_k\":3}}}\n\n// Refresh after files change on disk (force:true rebuilds everything)\n{\"jsonrpc\":\"2.0\",\"id\":9,\"method\":\"tools/call\",\"params\":{\"name\":\"reindex\",\"arguments\":{}}}\n\n// Clean up — unregisters the folder and purges its index data\n{\"jsonrpc\":\"2.0\",\"id\":10,\"method\":\"tools/call\",\"params\":{\"name\":\"remove_folder\",\n \"arguments\":{\"path\":\"C:/path/to/docsonar/examples/sample-docs\"}}}\n```\n\nSecurity check worth trying: `read_file` with any path outside a registered folder (e.g. `C:/Windows/System32/drivers/etc/hosts`) returns a refusal, not file content.\n\nCI runs lint, typecheck, and the test suite on Linux, macOS, and Windows.\n\n## License\n\nMIT\n",
"bytes": 12263,
"sha": "0e33beb56d99906e28cb61cfe17bc999e74ca57072b2a50f3de40ea9f9176720",
"repo_slug": "ribhav-jain/docsonar",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_ribhav_jain_docsonar_85509fe1/readme"
}