{
  "markdown": "# DocSift\n\n> Convert documents once. Give agents only what they need.\n\n<!-- Ownership marker for the official MCP Registry; it verifies this line\n     against the README published on PyPI. Keep it. -->\n`mcp-name: io.github.anishmoncivarghese/docsift`\n\nA 300-page PDF does not fit in a language model's context window, and pasting it\nin would be expensive if it did. DocSift converts documents into clean Markdown\nonce, indexes them, and then hands back only the passages that answer a\nquestion — with page numbers and section headings, so the answer can be cited.\n\nIt runs on your machine. PDFs go through [Docling](https://github.com/docling-project/docling),\neverything else through [MarkItDown](https://github.com/microsoft/markitdown),\nboth behind one interface. No cloud APIs, no accounts, no telemetry.\n\n**Where the saving comes from:** retrieval, not conversion. Cleaning barely\nreduces tokens on PDFs, because Docling already strips headers and footers with\nits layout model. What changes the bill is asking a question and getting three\nrelevant chunks back instead of a whole document.\n\n## Quickstart\n\nPick the engine you need — `markitdown` for Word, Excel, PowerPoint, HTML, CSV\nand EPUB; `docling` for PDFs (a large download: ML layout models); `all` for both\nengines plus the HTTP API and MCP server.\n\n    pip install \"docsift[markitdown]\"\n    pip install \"docsift[docling]\"\n    pip install \"docsift[all]\"\n\n    docsift convert report.pdf\n\nThat writes cleaned Markdown, token-budgeted chunks and a JSON summary to\n`./output/`. `pip install docsift` on its own installs the CLI but no engine, and\nconversion will tell you so rather than failing obscurely.\n\n**Office means the modern formats:** `.docx`, `.xlsx`, `.pptx`. The pre-2007\nbinaries — `.doc`, `.xls`, `.ppt` — are not supported, and DocSift refuses them\nby name rather than failing halfway through. Open one in PowerPoint, Keynote,\nLibreOffice or Google Slides and save it as `.pptx` first.\n\n## Use it from Claude, Codex, or another MCP client\n\nThe shortest path to the point of this tool: let an assistant search your own\ndocuments, without pasting them anywhere.\n\n### Which clients this works with\n\nDocSift speaks MCP over **stdio** — the client starts it as a program on your\nmachine. Anything that can do that is supported:\n\n| Client | Supported | How |\n|---|---|---|\n| Claude Code | yes | `claude mcp add` — [step 2](#2-register-it-with-your-client) |\n| Claude Desktop | yes | `claude_desktop_config.json` |\n| VS Code (Copilot agent mode) | yes | `.vscode/mcp.json` |\n| Codex CLI | yes | `~/.codex/config.toml` |\n| Cursor | yes | JSON config, same shape as VS Code |\n| **claude.ai in the browser** | **no** | needs a remote server |\n| **ChatGPT (web or desktop)** | **no** | needs a remote server |\n\nThe last two are worth being clear about before you install anything. Their\nconnector features only accept a **remote** MCP server at a public HTTPS URL,\nand DocSift has no remote transport — there is no configuration that makes a\nlocal one appear in those interfaces.\n\nThat is a deliberate position rather than an oversight. Reaching them means\nrunning DocSift on a server and uploading your documents to it, which is the\nopposite of the thing this tool is for. A self-hosted remote transport is a\nreasonable future addition; sending your files to someone else's machine is not.\n\n### 1. Install it as a command, not into a project\n\nAn MCP client starts DocSift as a program, so it has to exist outside any\nvirtualenv. Install it as a standalone tool:\n\n    uv tool install --python 3.12 \"docsift[mcp,docling,markitdown]\"\n\nor with pipx:\n\n    pipx install --python python3.12 \"docsift[mcp,docling,markitdown]\"\n\nDocSift needs **Python 3.11 or newer**. If your default is older, the version\nflag above is what avoids an unsatisfiable-requirements error. Expect a large\ndownload: `docling` brings PyTorch and layout models.\n\n**On Linux, add `--torch-backend auto`.** The default resolves to the CUDA build\nof PyTorch — 5.3 GB installed, roughly 2 GB of it `nvidia-*` wheels that a\nmachine without an NVIDIA GPU never loads. `auto` detects your driver and picks\nthe right build, which is 1.6 GB on a machine without a GPU and leaves CUDA in\nplace on one with a GPU:\n\n    uv tool install --python 3.12 --torch-backend auto \"docsift[mcp,docling,markitdown]\"\n\nmacOS wheels are CPU-only already, so the flag changes nothing there. It is a uv\nfeature: with `pipx` or `pip` there is no equivalent, because the CPU builds live\non a separate index and no published package can redirect an installer to it. If\nyou install that way on a CPU-only Linux box, DocSift says so after your first\nconversion rather than letting several unused gigabytes pass unmentioned.\n\nCheck it landed. Run these one at a time; the second prints the path to the\nexecutable, which the next step needs.\n\n    docsift --version\n\n    which docsift\n\n### 2. Register it with your client\n\n**Pick the one you use. You only need one of these.**\n\n#### Claude Code\n\n`$(which docsift)` fills in the path for you, so this works exactly as written:\n\n    claude mcp add --scope user docsift -- \"$(which docsift)\" mcp\n\nThen confirm it started — look for `docsift ... ✔ Connected`:\n\n    claude mcp list\n\n`--scope user` makes it available in every project; without it, the server is\nregistered only for the directory you were in.\n\n**That is the whole setup for Claude Code.** Skip the other clients below and go\nto step 3.\n\n#### VS Code\n\nCopilot agent mode reads `.vscode/mcp.json` for one project, or the file behind\nthe **MCP: Open User Configuration** command for all of them:\n\n```json\n{\n  \"servers\": {\n    \"docsift\": {\n      \"type\": \"stdio\",\n      \"command\": \"/replace/with/the/path/from/which/docsift\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\n#### Codex CLI\n\n`~/.codex/config.toml`:\n\n```toml\n[mcp_servers.docsift]\ncommand = \"/replace/with/the/path/from/which/docsift\"\nargs = [\"mcp\"]\nstartup_timeout_sec = 60\n```\n\nRaise the timeout as shown. Codex allows ten seconds by default and DocSift\nloads PyTorch on the way up, so the default reports a server that failed to\nstart when it was only still starting.\n\n#### Claude Desktop, Cursor\n\nThese read a JSON config file instead. For Claude Desktop on macOS that is\n`~/Library/Application Support/Claude/claude_desktop_config.json`; create it if\nit does not exist.\n\nIf the file is empty or new, this is the whole contents — replacing the command\nwith the path `which docsift` printed in step 1:\n\n```json\n{\n  \"mcpServers\": {\n    \"docsift\": {\n      \"command\": \"/replace/with/the/path/from/which/docsift\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\nIf it already has other servers, add `docsift` beside them rather than replacing\nthe file — note the comma after the previous entry:\n\n```json\n{\n  \"mcpServers\": {\n    \"something-you-already-had\": {\n      \"command\": \"...\"\n    },\n    \"docsift\": {\n      \"command\": \"/replace/with/the/path/from/which/docsift\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\nUse the absolute path, not a bare `docsift`. These clients do not reliably\ninherit your shell's `PATH`, and a wrong or bare path fails with `ENOENT: no\nsuch file or directory`.\n\nThen **restart the app** — the config is read at startup.\n\n**Use the absolute path from `which docsift`, not a bare `docsift`.** MCP\nclients do not reliably inherit your shell's `PATH`, and this is the most common\nreason a local server silently fails to start.\n\n### 3. Ask\n\nNo commands to learn — describe what you want:\n\n> search ~/Documents/contract.pdf for the termination clause\n\n> what does report.pdf say about Q3 revenue?\n\nThe first question about a new file converts it, and on a PDF that is slow —\nabout three minutes. That is startup cost, not page count: Docling downloads its\nlayout and table models from HuggingFace on the very first conversion, then\nloads PyTorch. A three-page test file takes about as long as a thirty-page\nreport, so picking something small to \"try it quickly\" does not help.\n\nIt happens once. Afterwards the file is recognised by its content and answers\ncome back immediately, even if you move or rename it.\n\n**For a long PDF, convert it first and ask afterwards:**\n\n    docsift convert big-report.pdf\n\nThat fills the same cache the MCP server reads, so the first question is as fast\nas the rest. It also sidesteps a real limit — MCP clients apply their own\ntimeouts to a tool call, and a long enough conversion can exceed one and surface\nas an error even though it would have finished.\n\n### 4. Check it works\n\nBefore pointing it at a real PDF, prove the wiring with a file that converts\ninstantly. Make one:\n\n    printf '# Test\\n\\nDocSift returns only the passages that match a question.\\n' > /tmp/docsift-check.md\n\nThen ask your assistant, naming the tool:\n\n> use the docsift search_document tool on /tmp/docsift-check.md to search for\n> \"passages\"\n\n**Name it explicitly, and watch which tool actually runs.** Asked casually, an\nassistant will often just open a small file with its own file-reading tool and\nanswer from that — you get the right answer having never touched DocSift, which\nmakes a broken setup look like a working one. If your assistant reports reading\na file rather than calling `docsift`, the check has told you nothing.\n\nThe first call will ask your permission to run the tool — approve it, and choose\nthe \"don't ask again\" option if your client offers one, so later questions are\nnot interrupted mid-thought.\n\nSuccess looks like a `docsift` tool call in the transcript, returning that one\nsentence. This needs no PDF and no model weights, so a failure here is a setup\nproblem — the command not on `PATH`, or the server not registered — and not a\nconversion one.\n\nThen try a real PDF of your own, and expect the first question to take a minute.\nOn a document of that size the choice takes care of itself: reading it whole is\nexpensive, which is when searching it becomes the obvious move.\n\nOne test worth running deliberately: **ask about something the document does not\nmention.** You should get nothing back rather than a plausible-sounding answer.\nSearch is lexical, so a word that is not in the text matches nothing — and\nseeing that once tells you more about what you can trust than a dozen successful\nqueries.\n\n### What you get\n\nTwo tools:\n\n- **`search_document`** — the one that matters. Give it a file path and a\n  question; it converts the file the first time it sees it, then returns only\n  the passages that match, with page numbers and section headings.\n- **`convert_document`** — converts and indexes a file, returning a summary\n  (page count, token estimate, chunk count) rather than the text.\n\n### What to expect\n\nMeasured on an Apple-silicon MacBook Air with a 34-page, 1.7 MB PDF — a\nBusinessEurope economic outlook, chosen because it is the awkward kind of\ndocument this tool exists for: dense tables, footnotes, multi-column stretches.\n\n| | |\n|---|---|\n| Whole document | ~21,000 tokens, 42 chunks |\n| One question answered | ~4,000 tokens, 5 passages, with pages to cite |\n| First conversion, ever | ~3 minutes — mostly a one-time model download |\n| First conversion of any later file | seconds to a minute, by size |\n| Every question after | immediate |\n\nThe token gap is the point, and it widens with document length.\n\nBe clear about where the three minutes goes, because it is easy to misread as\n\"big documents are slow\". Most of it is Docling fetching its layout and table\nmodels the first time it ever runs, plus loading PyTorch. On a clean Linux\nmachine a **three-page, 1.8 KB** PDF took 186 seconds — essentially the same as\nthe 34-page report. After that first run the models are on disk and conversion\nscales with the document. `docsift convert` shows a live spinner throughout, so\nyou can see it working rather than guessing.\n\n`search_document` takes `limit` and `max_tokens` as well, and a model will set\nthem when you ask for more or less. The defaults return five passages within a\n5,000-token budget. Dense documents produce large chunks — around 1,000 tokens\neach in the report above — so if answers feel truncated, a larger `max_tokens`\nis the dial, and it is cheaper than the model asking the same question several\ntimes over.\n\nOne caveat the numbers do not show: **search is lexical, not semantic.** DocSift\nindexes chunks in SQLite FTS5 and ranks them with BM25. It matches the words you\ntype — not synonyms, not paraphrases, not meaning. Ask for \"termination clause\"\nand a section that only ever says \"ending the agreement\" will not come back.\nAsking that report about \"energy prices\" surfaced a section on shipping and\nfertiliser costs because those passages happen to contain the phrase; a question\nworded \"why did transport costs rise?\" might have missed it entirely.\n\nThat is a deliberate trade, not an oversight: no embedding model to download, no\nindex to rebuild, no GPU, nothing leaving your machine, and a first run measured\nin minutes rather than tens of minutes. If you need semantic retrieval, the\nMarkdown and chunk JSON DocSift writes are clean input for a vector store — a\nreasonable thing to want, and not what this tool does today. See\n[Known limitations](https://github.com/anishmoncivarghese/docsift/blob/main/docs/LIMITATIONS.md).\n\n**PowerPoint answers cite slide numbers.** A deck is chunked one slide at a\ntime, so a passage comes back saying which slide it is from and what that slide\nwas titled — the same way a PDF passage carries its page. The field is called\n`pages` for both; for a presentation it means slides. Speaker notes are\nextracted too, and are searchable.\n\n**What a deck will not give you: diagrams.** A slide with three boxes and two\narrows extracts as three labels. The words survive, the relationships do not, so\nDocSift cannot tell you what an arrow between two boxes meant. Images inside\nslides are not read at all — text in a screenshot is invisible to search.\n\nEverything happens in that process, on your machine. Nothing listens on a port\nand no document content crosses the network. The server has exactly the\nfilesystem access of the user who started it.\n\n> **Not in the claude.ai connector directory, and it cannot be.** claude.ai runs\n> in the cloud and cannot start a program on your computer. A local MCP server\n> works in apps running on your own machine — Claude Code, Claude Desktop, Codex,\n> Cursor — and searching the web app's connector list for DocSift will never find\n> it.\n\n## Command line\n\n    docsift convert report.pdf --max-tokens 800 --overlap 100\n    docsift convert report.pdf --engine markitdown\n    docsift inspect report.pdf\n    docsift compare report.pdf\n    docsift search doc_xxxxxxxxxxxx \"operational risk\"\n    docsift search doc_xxxxxxxxxxxx '\"operational risk\"' --limit 5 --context 1\n    docsift cache info\n    docsift cache clear\n\n`inspect` reports what DocSift would do with a file — engine, identity, cache\nstatus — without converting it. `compare` runs both engines on the same document\nand writes a diff of the results.\n\nConversion cleans repeated headers and footers, page numbers and image\nreferences, then splits the text into token-budgeted chunks that carry their\nheading context. `--keep-furniture` and `--keep-image-refs` turn the cleaning\nstages off.\n\nResults are cached, so an unchanged file with unchanged settings returns\ninstantly. `--no-cache` forces a re-run.\n\nSearch is local SQLite FTS5 keyword ranking with quoted-phrase support. It reads\nthe store the HTTP service and the MCP server write to — `docsift convert`\nwrites standalone files to an output directory and does not add them to it.\n`--context` pulls in adjacent chunks, `--max-tokens` caps the whole response, and\nonly selected chunks are ever printed. Scores order results within one response\nand are not comparable across requests.\n\n## HTTP API\n\n    pip install \"docsift[all]\"\n    docsift serve\n\nConversion always runs in the background: a long PDF can take minutes, and a\nclient expecting a synchronous response will time out.\n\nUpload, and get back `202` with a job id and a document id:\n\n    curl -sS -F file=@report.pdf http://127.0.0.1:8000/v1/documents\n\nPoll until the status is `succeeded` or `failed`:\n\n    curl -sS http://127.0.0.1:8000/v1/jobs/job_xxxxxxxxxxxxxxxx\n\nThen retrieve the whole document:\n\n    curl -sS http://127.0.0.1:8000/v1/documents/doc_xxxxxxxxxxxx/markdown\n    curl -sS http://127.0.0.1:8000/v1/documents/doc_xxxxxxxxxxxx/chunks\n\nOr ask a question and get only the relevant chunks:\n\n    curl -sS --get --data-urlencode 'q=operational risk' \\\n      --data 'limit=5' --data 'max_tokens=5000' \\\n      http://127.0.0.1:8000/v1/documents/doc_xxxxxxxxxxxx/search\n\n`DELETE /v1/documents/{id}` removes the document, its search index and its cached\nconversions — including cancelling a conversion still in flight, so a delete\ncannot be undone by a worker finishing afterwards. The OpenAPI document is at\n`/openapi.json`.\n\n**Set `DOCSIFT_API_KEY` before anything else can reach it.** Every `/v1/*` route\nthen requires an `X-API-Key` header, while `/health`, `/version` and the API docs\nstay open for health checks and connector imports. It is one shared secret for\nthe whole service — not per-user identity, and no substitute for network\ncontrols. The service converts whatever it is given, so run it on infrastructure\nyou control.\n\n## Power Platform, Copilot Studio and n8n\n\n    DOCSIFT_PUBLIC_URL=https://docsift.example docsift openapi --format swagger2 -o connector.json\n\nImport that as a Power Platform custom connector. The service's own\n`/openapi.json` is OpenAPI 3.1, which custom connectors reject; this command\nemits the Swagger 2.0 they accept. Verified against a real tenant: the file\nimports, authenticates and returns schema-valid responses.\n\nWorked examples live in\n[`examples/`](https://github.com/anishmoncivarghese/docsift/blob/main/examples/)\n— an importable n8n workflow, a Copilot Studio connector walkthrough, and the\nPower Automate *Do until* flow that waits for a conversion. A longer guide is in\n[docs/USING_DOCSIFT.md](https://github.com/anishmoncivarghese/docsift/blob/main/docs/USING_DOCSIFT.md).\n\n## More\n\n| | |\n|---|---|\n| [Known limitations](https://github.com/anishmoncivarghese/docsift/blob/main/docs/LIMITATIONS.md) | What DocSift does not do, stated plainly. Worth reading before you rely on it. |\n| [Configuration](https://github.com/anishmoncivarghese/docsift/blob/main/docs/CONFIGURATION.md) | Every environment variable. |\n| [Docker](https://github.com/anishmoncivarghese/docsift/blob/main/docs/DOCKER.md) | Running the service in a container. |\n| [Deploying to Azure](https://github.com/anishmoncivarghese/docsift/blob/main/docs/DEPLOY_AZURE.md) | A hosted deployment, end to end. |\n| [Privacy](https://github.com/anishmoncivarghese/docsift/blob/main/PRIVACY.md) | What lands on disk, and the three times the network is touched. |\n| [Security](https://github.com/anishmoncivarghese/docsift/blob/main/SECURITY.md) | The threat model, and how to report a vulnerability. |\n| [Contributing](https://github.com/anishmoncivarghese/docsift/blob/main/CONTRIBUTING.md) | Setup, the gates, and what review looks for. |\n| [Changelog](https://github.com/anishmoncivarghese/docsift/blob/main/CHANGELOG.md) | What shipped, and when. |\n\n## License\n\nMIT\n",
  "bytes": 19179,
  "sha": "0cfb7a84a98f868d59efffbd5f66791100bff3ac5c4c783a6c4dc594bb6b9e9e",
  "repo_slug": "anishmoncivarghese/docsift",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_anishmoncivarghese_docsift_3bfbe1c5/readme"
}