{
  "markdown": "# shelfmark\n\n[![PyPI](https://img.shields.io/pypi/v/shelfmark)](https://pypi.org/project/shelfmark/)\n[![CI](https://github.com/Dankaro-projects/shelfmark/actions/workflows/ci.yml/badge.svg)](https://github.com/Dankaro-projects/shelfmark/actions/workflows/ci.yml)\n[![Python](https://img.shields.io/pypi/pyversions/shelfmark)](https://pypi.org/project/shelfmark/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n\n**Give AI agents the right context, not your entire filesystem.**\n\nShelfmark turns years of scattered documents into a governed context map for\nAI agents. An agent can discover what exists, understand what kind of\nmaterial it is, and select the documents relevant to a task — before\nspending context opening files.\n\nNo document migration. No duplicated content store. No need for a perfect\nfolder structure.\n\n```sh\nuv tool install shelfmark          # or: pipx install shelfmark\n```\n\n*Local by design · metadata only · governed discovery · built for MCP*\n\n---\n\n## Your best knowledge is probably sitting in your folders\n\nReports, presentations, models, research, proposals and working documents\naccumulate over years. Some are carefully organised. Others sit inside\ncrowded project folders, old archives, download directories, or collections\nthat made sense only at the time.\n\nThe value is still there. The problem is that agents cannot use what they\ncannot discover — and giving an agent unrestricted filesystem access does\nnot solve that. It transfers the work of finding, interpreting and filtering\nthousands of files into the context window.\n\nShelfmark gives the agent a map first.\n\n## Context is the scarce resource\n\nAn agent does not need every document. It needs to know what exists, what is\nlikely to matter, where it came from, and whether it should be used at all.\n\nShelfmark is a discovery layer between the agent and your files. The agent\nsearches the catalogue, narrows the field, and requests only the material the\ntask actually needs — so an existing document estate becomes working context\nwithout turning the filesystem into one enormous prompt.\n\n## A catalogue of pointers, not another document store\n\nShelfmark builds a local SQLite catalogue of **references and derived\nmetadata**: paths, filenames, formats, sizes, document types, authors,\ndates, classifications, selected Office properties, presentation titles, and\noptional content hashes.\n\nIt does not copy your documents into the catalogue. It does not index\ndocument body text. It does not create a second repository to govern,\nsynchronise and maintain. Your files stay where they are; the catalogue\npoints at them and describes what can be established from their metadata.\n\n**Finding a document does not expose its contents.**\n\n## Turn messy folders into usable agent context\n\nShelfmark does not require a designed information architecture. Point it at\naccumulated project files, forgotten archives, or folders where documents\nhave simply been dropped over the years.\n\nIt builds an inventory from signals that already exist — filenames,\nextensions, paths, authorship, dates, Office properties, and your own\nclassification rules — creating a virtual structure across the material.\nAgents can then search by document type, client, project, purpose, author,\nyear or location without anything being moved or renamed.\n\nA messy folder can become navigable even when it never becomes tidy.\n\nShelfmark also stays honest about the limits of metadata. A file called\n`final7.pdf` with no useful properties cannot reveal its meaning without\nsomeone reading it. Shelfmark reports what is known rather than inventing\ncertainty — the same reason it drops placeholder slide titles instead of\nlisting twelve headings a deck does not have.\n\n## Built for selective context\n\n- **Discover before opening.** Search thousands of references before\n  deciding which few files deserve attention.\n- **Preserve the context window.** Concise catalogue results instead of\n  whole documents that may not be relevant.\n- **Build on previous work.** Reports, models, proposals, research and\n  methods stay discoverable across new tasks and future engagements.\n- **Keep provenance visible.** Path, date, author, document type and\n  surrounding context help an agent judge relevance.\n- **Reduce blind exploration.** Structured search instead of repeatedly\n  walking directories and inspecting files one at a time.\n- **Separate discovery from access.** Shelfmark helps identify material;\n  opening the original remains a separate, controllable decision.\n\n## Your roots are the trust boundary\n\nShelfmark walks only the roots you configure.\n\n**Symlinks are not followed.** A link inside a root reads as an ordinary\nfile and would otherwise walk straight out of the tree you configured — and\n`hash` opens files. Skipped links are reported, never silently dropped. To\nindex another tree, add it as an extra root: the boundary widens by saying\nso in config, not by planting a link.\n\nThe catalogue is refused inside *any* root, because a database that indexes\nitself grows on every refresh. Both checks compare resolved paths, so `..`\nand a symlinked root cannot slip past them.\n\n## Governance belongs in the retrieval layer\n\nNot every useful document should be treated the same way. Shelfmark\nseparates two questions that usually get confused:\n\n- **Who owns or may reuse this?** → `rights`: `OWN` / `REFERENCE` /\n  `RESTRICTED`\n- **May this document leave its current context?** → `confidential`: `0` / `1`\n\nA method may belong to you while the client deliverable containing it stays\nconfidential. Modelling the two separately lets agents discover reusable\nknowledge without treating everything discoverable as freely shareable.\n\nFiles matching your private/secret patterns become RESTRICTED: no tool\nreturns their path, name, metadata or content, no argument overrides it,\nthey are never opened for hashing, and the database is opened read-only.\n`corpus_stats()` reports a single corpus-wide count of sealed files and\nnothing else about them — not which root, not which folder. That count is\nthe one thing disclosed, deliberately: silence about it would misrepresent\nthe size of the corpus.\n\nGovernance is applied by the catalogue, not left to the wording of a prompt.\n\n## Know whether the map can be trusted\n\nA search result is only useful if the agent knows the catalogue is current.\n\nThe MCP server keeps its own index current while it runs, so nothing has to\nbe scheduled and no agent has to remember. When it cannot — never built,\nstale, a failed refresh, a clock it cannot reason from, or an index that no\nlonger agrees with the filesystem — **every tool says so above its answer**,\nand `corpus_stats()` compares index against disk in full.\n\nAn old snapshot is never presented as complete knowledge.\n\n## Designed for knowledge-intensive work\n\n| | |\n|---|---|\n| **Consultants and advisors** | Find previous analyses, proposals, frameworks and deliverables without exposing unrelated client material. |\n| **Researchers and analysts** | Navigate large collections of reports, datasets and source material through consistent metadata. |\n| **Product and strategy teams** | Reconnect decisions, research, roadmaps and previous thinking across projects and time. |\n| **Studios and independents** | Turn years of accumulated work into reusable context while keeping control over client files and IP. |\n| **Agent builders** | Give local agents a governed discovery layer over MCP. |\n\n## How it works\n\n1. **Point Shelfmark at your existing folders.** One or more roots. Files\n   stay where they are.\n2. **Build the local catalogue.** It walks the permitted roots, extracts\n   available metadata, applies classification rules, and writes references\n   into SQLite.\n3. **Review ownership and confidentiality.** `shelfmark review` asks a few\n   questions about your own folders and writes the answers to config.\n4. **Connect an MCP-compatible agent.** It searches, browses and inspects\n   catalogue records through structured tools.\n5. **Retrieve only what matters.** The agent identifies the relevant\n   artefacts before any separate content access takes place.\n\n## Install\n\n```sh\nuv tool install shelfmark          # or: pipx install shelfmark\n# from a checkout:\nuv tool install /path/to/shelfmark\n```\n\nPython ≥ 3.11. macOS, Linux and Windows — the full suite runs on all three\nin CI, including the Windows-specific behaviours (OneDrive placeholder\ndetection, junction refusal at the root boundary).\n\n### Codex plugin\n\nInstall the MCP server and its catalogue-management, context-finding, and\narchive-research skills as one versioned plugin:\n\n```sh\ncodex plugin marketplace add Dankaro-projects/shelfmark\ncodex plugin add shelfmark@shelfmark\n```\n\nThen create the local catalogue once with `uvx shelfmark init` followed by\n`uvx shelfmark refresh`. The plugin starts the pinned Shelfmark MCP server\nover stdio; document paths and metadata stay on the local machine.\n\nTo install the three companion skills without the MCP configuration:\n\n```sh\npnpm dlx skills add Dankaro-projects/shelfmark --full-depth\n```\n\nEmail ingestion is optional, and the extra you want depends on the format\nyou have. `.msg` resolves to wheels everywhere; `.pst` needs\n`libpff-python`, which publishes no wheels and compiles from C source, so\nit requires a build toolchain (Visual C++ Build Tools, `build-essential`,\nor the Xcode command line tools):\n\n```sh\nuv tool install \"shelfmark[msg]\"      # .msg — no compiler needed\nuv tool install \"shelfmark[pst]\"      # .pst — compiles from C source\nuv tool install \"shelfmark[email]\"    # both\n```\n\n## Quickstart\n\nThree commands, and `init` finds your documents for you — when the default\nroot misses, it sweeps for the folders that do hold documents and one\nkeypress fixes the config:\n\n```text\n$ shelfmark init\nWrote ~/.config/shelfmark/config.toml\n\nWARNING: ~/Documents does not exist on this machine.\n\nThese folders do hold documents:\n  1. ~/Paperwork  (~6 document files)\n  2. ~/Downloads  (~1 document file)\nIndex which folder? [1] — a number, a path, or 'k' to keep ~/Documents:\n  Root set to ~/Paperwork. Edit ~/.config/shelfmark/config.toml any time.\n\n$ shelfmark refresh\ncataloguing ~/Paperwork -> ~/.local/share/shelfmark/catalog.db\nseen 6  new 6  updated 0  unchanged 0  rematerialised 0\nevicted 0  corrupt 0  restricted 0\n\n$ claude mcp add shelfmark -s user -- shelfmark-mcp\n```\n\nThat's install to connected. The agent's first call then looks like this\n(a real `corpus_stats()` answer over the corpus above):\n\n```text\n# shelfmark corpus\n6 files · 0.0 GB\n✓ index fresh — matches disk, last refresh 0 min ago\n\n## Roots\nroot                     files  own+shareable\nClients                      4              0\nDecks                        2              0\n\n## Rights × confidential\n  REFERENCE    may leave                5\n  UNKNOWN      unreviewed → held        1\n\n## doc_type (what files ARE)\n  report 4, deck 2\n```\n\nOptional but worth the five minutes: `shelfmark review` asks a few\nquestions about your biggest unclassified subtrees and writes the answers\nto config, so rights stop being UNKNOWN; `shelfmark stats` prints the\ncensus any time; `shelfmark config` shows every rights rule in the order\nit is checked, with the number of files each one currently claims.\n\nThen in a session: `corpus_stats()` to orient, `browse_folder()` to\nnavigate, `search_docs()` / `get_file()` to find and inspect.\n\n## MCP tools\n\n| Tool | What it answers |\n|---|---|\n| `corpus_stats()` | What is here overall + an honest freshness line. Call first. |\n| `browse_folder(prefix)` | What is inside a folder: counts, sizes, facet mix. |\n| `search_docs(query, …)` | Metadata full-text search with facet filters. |\n| `get_file(path)` | Full record for one file: rights, authorship, slide titles, identical copies, on-disk status. |\n| `search_emails(query, …)` | Full-text over an ingested .pst/.msg email archive (optional). |\n\nResult lists always say when they are cut (`showing 100 of 195 …`), unknown\nfilter values are reported as bad filters with suggestions (never as an\nempty corpus), and excerpts mark their truncation point.\n\n## Configuration\n\nEverything corpus-specific lives in `config.toml` — the code ships with\nneutral defaults only. Resolution order: `--config` flag →\n`$SHELFMARK_CONFIG` → `~/.config/shelfmark/config.toml`. See\n`config.example.toml` for the full annotated reference. Highlights:\n\n| Section | What it controls |\n|---|---|\n| `[[roots]]` | The trees to index. One unlabelled primary root; extra roots get a label prefix. |\n| `[index]` | Where the SQLite catalogue lives. **Must be outside every indexed root and outside cloud-synced folders** — it is a mutating binary DB, and this is enforced. |\n| `[privacy]` | Regexes for secrets and private subtrees → RESTRICTED. Built-ins already cover `.env`, key/cert files, `id_rsa`, backup codes, identity documents. |\n| `[authors]` | Regexes for your own name/company, for client authors, and for generator tools — drives OWN/REFERENCE classification from OOXML authorship. |\n| `[rights]` | Path-prefix rules for the two-axis model: `rights` (may I reuse it) × `confidential` (may it leave). |\n| `[facets]` | Which top-level folders count as work/personal; where client/project names sit in the path. |\n| `[doc_types]` / `[context_types]` | Extra filename/folder rules, checked before the built-in bilingual (EN/ES) defaults; built-ins can be disabled by name. |\n\n### What `shareable_only` means\n\n`shareable_only=True` means *positively* classified: `confidential=0 AND\nrights IN (OWN, REFERENCE)`. Never-reviewed files are held back — unreviewed\nis not the same as cleared.\n\n### Getting rights set: `shelfmark review`\n\nMost files carry no OOXML author — a corpus is mostly PDFs, markdown and\nimages — so authorship classifies almost nothing and **path rules are the\nonly mechanism that can**. They ship empty, which is why a fresh catalogue\nis mostly `UNKNOWN` and `shareable_only` comes back nearly empty.\n\nThat knowledge is yours, not the corpus's, so `review` asks for it — biggest\nwin first, with what the catalogue knows on screen:\n\n```\n[1/8] Projects\n      493 unclassified of 662 files · folder reads as: pitch\n      is: note 202, document 89, code 88, pdf 51\n      authored by: R. Okonjo (31), A. Lindqvist (12), openpyxl (9)\n      e.g. programme-overview.pptx · phase-two-proposal.pptx\n      answer [skip]:\n```\n\nAnswer `own`, `own-private`, `reference`, `client`, `personal`, `scratch`\nor `skip`. On a ~1,900-file corpus, five answers settled 80% of the\nunclassified files and eight settled 90%.\n\n```sh\nshelfmark review                 # dry run — shows what it would write\nshelfmark review --apply         # writes the prefixes, re-derives rights\n```\n\nIt stops whenever you do, only ever asks about subtrees that are still\nunclassified (so re-running continues where you left off), and writes\n**config, never rights values in the data** — so any answer is undone by\nediting the config and re-running `shelfmark rights`. Where authorship is\nlopsided enough to be evidence it offers a default; where it is thin it\nstays quiet rather than guessing.\n\n### Editing classification rules\n\nTwo things people trip over, learned the hard way:\n\n1. **Anchor short regex alternatives on both sides.** An unanchored `rfi`\n   matches inside \"Docke**rfi**le\" and the Spanish word \"pe**rfi**l\". Before\n   adding or deleting an alternative, list the filenames it actually\n   matches.\n2. **A rule edit does not relabel existing files.** The builder is\n   incremental, so after any rule change run:\n\n   ```sh\n   shelfmark reclassify all          # dry run — shows what would change\n   shelfmark reclassify all --apply\n   ```\n\n   Do **not** use `build --rebuild` for this: it re-walks everything (and\n   on cloud-synced trees drags every evicted file back down) when the\n   filenames are already in the DB.\n\n## Keeping it current\n\n```sh\nshelfmark refresh                # build + rights + prune + assertions\nshelfmark refresh --if-needed    # only if a write landed or the index is old\n```\n\n`refresh` is incremental by (size, mtime, cloud-residency) — a couple of\nseconds over a ~30k-file tree — and it asserts its own correctness on every\nrun: walk coverage (an OS-denied walk must not read as success), guarded\npruning of deleted files (refused when it looks like a mass deletion or an\nunmounted root), and governance invariants (no RESTRICTED row in the search\nindex, no unsealed secret, no unsearchable non-RESTRICTED row). Results\nland in `REFRESH_STATUS.json` next to the DB, and the MCP server reports\nthem on every `corpus_stats()` call.\n\nAny guard that declines to touch the index says so on stderr and exits\nnon-zero — you never get a clean-looking summary hiding a refusal.\n\n### When something is wrong and nothing says so\n\n```sh\nshelfmark doctor\n```\n\nChecks the setup for the failures that stay quiet: a catalogue inside a\ncloud-synced folder (a sync client uploads it on every write, and one that\ncopies the `.db`, `-wal` and `-shm` at different moments restores a torn\ndatabase that opens and answers *wrongly*), a root the process cannot read\n— on macOS usually Full Disk Access, which is what an empty walk is hiding\n— a database folder that is not writable, a corpus that is mostly\ncloud-evicted and what that costs you, and any email format sitting in the\ncatalogue with no reader installed.\n\nIt names the fix rather than the fault, prints the evidence behind each\nguess so you can overrule it, and exits non-zero on anything fatal so it\ncan gate a setup script.\n\nNo flag changes what is checked. The one flag there is changes only who\nthe output is for:\n\n```sh\nshelfmark doctor --report\n```\n\nThe same verdicts as JSON, with the corpus taken out — counts, states and\nverdict codes, never a path, a filename, a root label or a config string.\nBecause nothing here phones home, a broken catalogue on your machine is\ninvisible to anyone who could help; this is the one channel a local-only\ntool gets, and at roughly 1.5 KB it fits in an issue. It carries the\nfailure streak, which is the difference between \"it is broken\" and \"it has\nbeen broken since Tuesday, 113 runs\".\n\nThe whole shape, nothing elided:\n\n```json\n{\n  \"shelfmark\": \"0.4.7\", \"python\": \"3.13.13\",\n  \"platform\": \"darwin\", \"machine\": \"arm64\", \"release\": \"25.5.0\",\n  \"roots\": {\"configured\": 3, \"readable\": 3},\n  \"catalogue\": {\"files\": 32942, \"evicted\": 0,\n                \"rights\": {\"OWN\": 3933, \"REFERENCE\": 27818,\n                           \"RESTRICTED\": 955, \"UNKNOWN\": 236}},\n  \"status\": {\"state\": \"ok\", \"detail_kind\": \"clean\",\n             \"failing_since\": null, \"consecutive_failures\": 0},\n  \"findings\": [{\"code\": \"reader_missing_msg\", \"status\": \"warn\",\n                \"count\": 2318}],\n  \"worst\": \"warn\"\n}\n```\n\nThe redaction is not a convention anyone has to remember: the report\ncopies no free text, so a check added later cannot leak through it, and a\ntest plants a canary string in the root name, the filename and the config\nand fails if it comes out the other end.\n\n### When a guard stops you\n\nThe two size guards cannot tell \"the root was unreadable\" from \"those files\nreally were deleted\" — both look like a short walk. So they refuse, name\nboth possibilities, and leave the index untouched:\n\n```\nFAILED — walk saw 63/123 catalogued files, below the 80% floor — either the\nroot was unreadable to this process or that many files really went away.\nIndex NOT updated; re-run with --force if the deletion was real.\n```\n\nCheck which it was. If the files are genuinely gone:\n\n```sh\nshelfmark refresh --force        # accept the short walk, prune past the ceiling\n```\n\n`--force` backs the catalogue up to `catalog.db.bak-preprune` before\ndeleting anything. If instead a root was merely unmounted or unreadable,\nfix that and refresh normally — the rows are still there.\n\n### A worked example: somebody renames a folder\n\nThis is the failure the guards exist for, start to finish. A 200-file\ncorpus: 60 documents in `Clients/Acme`, 140 in `Admin`. Every line below is\nreal output; only the paths are shortened.\n\n**1 — a clean build.**\n\n```text\n$ shelfmark refresh\ncataloguing ~/Paperwork -> ~/.local/share/shelfmark/catalog.db\nseen 200  new 200  updated 0  unchanged 0  rematerialised 0\nevicted 0  corrupt 0  restricted 0\n```\n\n**2 — `Clients/Acme` is renamed to `Clients/ACME Corp`.** Nothing was\ndeleted, but every catalogued path under the old name now points at\nnothing, and the walk finds 60 files it has never seen:\n\n```text\n$ shelfmark refresh\nseen 200  new 60  updated 0  unchanged 140  rematerialised 0\n\nprune REFUSED — 60 of 200 rows are no longer on disk, over the 2% ceiling.\n  Nothing was deleted; the index still lists them. A real deletion, an\n  unreadable subtree, or an upgrade widening the default skip list all look\n  like this. Check what went missing, then re-run with --force to accept it.\n```\n\nThe refusal is the right call: from here a rename and a mass deletion are\nthe same event. The status is now `degraded`, not `ok` — the run finished,\nbut the catalogue knowingly lists 60 rows whose files are gone.\n\n**3 — run it again** and the catalogue is carrying both copies, so the\ncoverage floor takes over from the prune ceiling:\n\n```text\n$ shelfmark refresh\nFAILED — walk saw 200/260 catalogued files, below the 80% floor — the root\nwas unreadable to this process, that many files really went away, or an\nupgrade widened the default skip list. Index NOT updated; re-run with\n--force if the deletion or the new skip list is right.\n```\n\nIt stays failed on every subsequent run. It does not quieten down.\n\n**4 — meanwhile, what an agent gets.** No tool answers as if nothing\nhappened; the warning leads, and the streak is in it:\n\n```text\n⚠ the last refresh FAILED (walk saw 200/260 catalogued files, below the 80%\nfloor … — 2 consecutive runs since 2026-08-11T08:46:01Z) — answers below\ncome from the previous snapshot.\n\nshowing 3 of 120 match(es) for '\"acme\"':\n\n- Clients/Acme/acme_note_055.md\n    note  [UNKNOWN]\n```\n\nNote the paths: it is still answering, and answering from the old names.\nThat is why the banner is not optional.\n\n**5 — ask what is wrong.**\n\n```text\n$ shelfmark doctor\n  [ok  ]  root (primary) is readable\n  [ok  ]  the catalogue is outside any sync folder\n  [ok  ]  the corpus is materialised on disk\n  [FAIL]  the last refresh FAILED — 2 consecutive runs since 2026-08-11T08:46:01Z\n     walk saw 200/260 catalogued files, below the 80% floor — …\n     -> The refresh already said this on stderr; if you are only seeing it now,\n       whatever runs it is discarding its output.\n\n2 of 5 checks need attention.\n$ echo $?\n1\n```\n\n**6 — the files really did move, so accept it.**\n\n```text\n$ shelfmark refresh --force\nseen 200  new 0  updated 0  unchanged 200  rematerialised 0\n--force: accepting coverage 200/260 (below the 80% floor)\n--force: pruned 60 rows, over the 2% ceiling (backup: catalog.db.bak-preprune)\n```\n\n**7 — confirm, and note that `stats` ends with a real walk, not a clock:**\n\n```text\n$ shelfmark doctor\n  [ok  ]  the last refresh completed cleanly\n1 of 5 checks need attention.\n\n$ shelfmark stats\n✓ index fresh — matches disk, last refresh 0 min ago\n```\n\nThe one warning left is `100% of the catalogue has UNKNOWN rights`, which\nis true of any corpus before `shelfmark review` has run — see\n[Getting rights set](#getting-rights-set-shelfmark-review).\n\n### You do not schedule this\n\nThe MCP server keeps its own index current. It is spawned by your client,\nlives for the whole session, and refreshes once on startup and then\nwhenever `refresh.max_age_seconds` has passed. Nothing to install, no\ntimer to configure, no command to remember.\n\nThat works because a resident process the client already starts is a\nbetter trigger than a scheduler: it inherits the same file access the\nclient has, and it is running exactly when you are asking questions. If\nyou maintain the catalogue some other way, `shelfmark-mcp\n--no-auto-refresh` leaves it alone.\n\nEditor hooks are now an optimisation, not the mechanism — they cut the\ndelay between an agent writing a file and that file being searchable, from\none refresh interval to the end of the turn. With Claude Code, in\n`~/.claude/settings.json`:\n\n```json\n{\n  \"hooks\": {\n    \"SessionStart\": [{\"hooks\": [{\"type\": \"command\",\n      \"command\": \"shelfmark hook session-start\", \"timeout\": 60}]}],\n    \"PostToolUse\": [{\"matcher\": \"Write|Edit|MultiEdit|NotebookEdit\",\n      \"hooks\": [{\"type\": \"command\", \"command\": \"shelfmark mark-dirty\"}]}],\n    \"Stop\": [{\"hooks\": [{\"type\": \"command\",\n      \"command\": \"shelfmark hook stop\"}]}]\n  }\n}\n```\n\n`mark-dirty` drops a marker only when a write landed under an indexed root\n(near-free on every other write); `hook stop` picks it up at the end of the\nturn. `hook session-start` refreshes and then checks the index against the\ndisk — the same check the MCP server runs — telling you and the agent only\nwhen something is wrong.\n\nDo not wrap these in `>/dev/null 2>&1`. An earlier revision of this page\ntaught exactly that, and it is how a catalogue stayed silently wrong for\nfour days: every refusal and failure the refresh reports goes to stderr,\nso the redirect discarded the only delivery of the news. `shelfmark hook`\nis silent when the catalogue is healthy and always exits 0 — there is\nnothing left to suppress.\n\n### If you also want it current with no client running\n\nOnly worth it for non-MCP use, or an instant first query on a very large\ntree. On Linux, a `systemd --user` timer running `shelfmark refresh\n--if-needed` is clean.\n\nOn macOS, **the obstacle is TCC, not launchd**. A LaunchAgent is denied\n`~/Documents`, `~/Desktop` and `~/Downloads` by default; `os.walk` swallows\nthe error, so the build walks a handful of files and **exits 0** — every\nlayer reports success while the index never updates. Grant Full Disk Access\nto the interpreter that runs the job and a LaunchAgent is fine. The\ncoverage assertion catches the ungranted case either way: the refresh\nfails loudly rather than quietly indexing nothing.\n\nOn Windows, Task Scheduler running the same command is the equivalent; no\nscheduled-run recipe has been field-tested there yet, so if you set one up,\nthe coverage assertion is again what stands between a broken schedule and a\nsilently stale index.\n\n## Content hashing and duplicates\n\nThe refresh never reads file contents (it must stay fast). To populate\nhashes for duplicate detection:\n\n```sh\nshelfmark hash                   # reads every unhashed, non-sensitive file\nshelfmark hash --limit 2000      # chip away at it\n```\n\nSensitive rows are never opened, and neither are symlinks. On cloud-synced\ntrees, dataless placeholder files are skipped — reading one silently yields\nthe hash of the empty string, which would make distinct files look\nidentical. Re-run after large materialisations.\n\n## What makes Shelfmark different\n\n- **Desktop search** helps a *person* find text inside files.\n- **Document management systems** require documents to be imported and\n  managed inside a new environment.\n- **Retrieval systems** parse, chunk and copy document content into search\n  indexes or vector stores.\n- **Filesystem tools** give an agent direct access and leave every discovery\n  decision to the agent.\n\nShelfmark sits at a different layer: a local, governed, metadata-based map\nof the documents you already have, built specifically for selective agent\ncontext.\n\n## What Shelfmark does not try to be\n\nIt is not a document management system. It does not replace your\nfilesystem. It does not require embeddings. It does not reorganise your\nfolders. It does not claim to understand content it has never read. And it\ndoes not assume every discoverable document is safe to share.\n\nIt gives agents a better starting point.\n\nThese are decisions, not a backlog:\n\n- **No embeddings.** Metadata FTS + facets first.\n- **No content extraction.** Body text stays out of the index by design.\n\n### Letting real misses decide\n\nThose two refusals should be revisited on evidence, not on a competitor's\nfeature list — so shelfmark records the searches that found nothing, locally,\nand tells you what they mean:\n\n```sh\nshelfmark misses            # the evidence\nshelfmark misses --clear    # forget it\n```\n\nThe report answers one question: **could metadata search ever have found\nit?** A term appearing in no filename, path, author, title or slide title\nwas unreachable however it was phrased — that is the shape content\nextraction fixes. A term that *is* in your corpus but still missed was a\nphrasing or filter problem, which is a different repair. Without that split,\na miss log only proves that people search.\n\n```\n73 searches returned nothing   (2026-06-02 → 2026-08-07)\n\nMost-missed terms:\n    9  abatement\n    7  timeline      (nowhere in your metadata)\n    6  commitments   (nowhere in your metadata)\n\n41 of 62 distinct terms (66%) appear nowhere in your filenames,\npaths, authors, titles or slide titles.\n\nThese are mostly things metadata search could NEVER have found, however\nphrased. That is the pattern the README says should reopen content\nextraction — not embeddings.\n```\n\nBad filters, impossible year ranges and searches against a stale index are\n**not** recorded as misses: each is already explained to the caller, and\nlogging them would bury the real signal. It stays local, is capped, and\nnever leaves your machine. Turn it off with `[misses] enabled = false` —\nrecording your own search terms is a privacy-affecting choice, and whether\nthat is acceptable genuinely varies by corpus.\n\n## FAQ\n\n**Does Shelfmark upload my documents?**\nNo. It runs locally and builds a local catalogue. Document contents are not\ncopied into it.\n\n**What does the catalogue contain?**\nFile references and derived metadata: paths, filenames, types, sizes, dates,\nauthors, classifications, selected Office properties, presentation titles,\nand optional content hashes.\n\n**Does it read document contents?**\nThe file catalogue does not index body text. Two deliberate exceptions:\n`shelfmark hash` opens files to compute content hashes for duplicate\ndetection, and optional **email ingestion** can index message bodies so\n`search_emails` works. Both are opt-in commands, both skip symlinks, and\nboth honour your privacy rules — a file or archive matching your\nsecret/private patterns is never opened by either.\n\n**Do I need to reorganise my folders first?**\nNo. Better filenames and metadata improve classification, but no formal\nstructure is required.\n\n**Can it understand every badly named file?**\nNo. A file with a meaningless name and no embedded metadata stays hard to\nidentify without reading it. Shelfmark says what it knows rather than\nguessing.\n\n**Why not just give an agent filesystem access?**\nFilesystem access lets an agent *open* files. Shelfmark helps it decide\nwhich files are worth opening — and which should stay out of its results\nentirely.\n\n**Why not a vector database?**\nA vector database is for semantic retrieval *from contents*. Shelfmark\nsolves the earlier problem: discovering and governing what exists, before\ndeciding what content should be processed at all. They are compatible; this\none comes first.\n\n**Which agents can use it?**\nAny client that supports local MCP servers. Register with\n`claude mcp add shelfmark -s user -- shelfmark-mcp`, or point your client at\nthe `shelfmark-mcp` command.\n\n## Development\n\n```sh\nuv run --group dev pytest        # the suite ships in the sdist, so this\n                                 # runs from the release artifact too\n```\n\nThe test corpus — including its OOXML files — is synthesised on every run;\nno fixture binaries are committed and no real document is ever read.\n\n## License\n\nMIT — see [LICENSE](LICENSE). Free to use, modify and redistribute.\n\n<!-- MCP registry ownership marker — the registry validates the PyPI\n     package by finding this name in the published README. -->\nmcp-name: io.github.Dankaro-projects/shelfmark\n",
  "bytes": 31842,
  "sha": "626149ad5aa10f36101221ed5bf80c7da81eadaaf227bbc9fe28a34a31eb04d6",
  "repo_slug": "dankaro-projects/shelfmark",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_dankaro_projects_shelfmark_928c9121/readme"
}