{
  "markdown": "# llm-wiki-base\n\n[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue)](https://www.python.org/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n[![Platform](https://img.shields.io/badge/platform-macOS%20%7C%20Linux%20%7C%20Windows-lightgrey)](#install)\n\nAn LLM-maintained wiki — compile knowledge **once**, maintain it **forever**.\nUnion retrieval (BM25 page + BM25 chunk + vector, RRF fusion) with a measurable\neval harness, an MCP bridge for AI tools, and init for both personal wikis and\ncodebase wikis.\n\nLocal-first. Python + SQLite (FTS5 BM25) + on-device vectors (fastembed).\nNo external services, no data lock-in.\n\n> **How it runs:** [`docs/wiki-flow.html`](docs/wiki-flow.html) — animated canvas\n> diagram, switchable personal ↔ project, showing exactly what is deterministic,\n> what needs an LLM, and where the human decides.\n\n## Contents\n\n- [Install](#install)\n- [Quickstart](#quickstart)\n- [How it works](#how-it-works)\n- [Wiki flavors](#wiki-flavors)\n- [CLI](#cli)\n- [Skills](#skills)\n- [Configuration](#configuration)\n- [Retrieval & eval](#retrieval--eval)\n- [Human authority](#human-authority)\n- [MCP bridge](#mcp-bridge)\n- [Upgrading](#upgrading)\n- [Safety & limitations](#safety--limitations)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Install\n\nRequirements: **git** (`uv` brings its own Python, so any Python 3.10+ — or none\nat all — works). One line per OS: installs the `llm-wiki-base` CLI globally\n(isolated, no sudo, no venv to activate) plus the machine runtime\n(`~/.llm-wiki-base/{tools/,rag/,scripts/,.venv/,registry.toml}`):\n\n```bash\n# macOS / Linux\ncurl -LsSf astral.sh/uv/install.sh | sh && export PATH=\"$HOME/.local/bin:$PATH\" && uv tool install \"git+https://github.com/nihmtaho/llm-wiki-base.git\" && llm-wiki-base setup tools\n```\n\n```powershell\n# Windows (PowerShell)\nirm https://astral.sh/uv/install.ps1 | iex; $env:Path = \"$env:USERPROFILE\\.local\\bin;$env:Path\"; uv tool install \"git+https://github.com/nihmtaho/llm-wiki-base.git\"; llm-wiki-base setup tools\n```\n\nVerify, then create a wiki:\n\n```bash\nllm-wiki-base setup doctor\nmkdir my-wiki && cd my-wiki && llm-wiki-base setup   # interactive wizard\n```\n\n### Install from a local clone (developers)\n\n```bash\ngit clone https://github.com/nihmtaho/llm-wiki-base.git && cd llm-wiki-base\npip install -e .[dev]\nllm-wiki-base setup tools\n```\n\n### Making a local install available globally (manual fallback)\n\nOnly needed for the local-clone install above: `pip install -e .` usually runs\ninside the repo venv, so the script lives at `<repo>/.venv/bin/llm-wiki-base`\n(macOS/Linux) or `<repo>/.venv/Scripts/llm-wiki-base.exe` (Windows) — **not on\nPATH**. Options:\n\n```bash\n# macOS / Linux — symlink (recommended)\nsudo ln -s \"$(pwd)/.venv/bin/llm-wiki-base\" /usr/local/bin/llm-wiki-base\n\n# macOS / Linux — PATH, no sudo\necho 'export PATH=\"'\"$(pwd)\"'/.venv/bin:$PATH\"' >> ~/.zshrc && source ~/.zshrc\n```\n\n```powershell\n# Windows — symlink (Admin CMD/PowerShell)\nmklink C:\\Windows\\llm-wiki-base.exe \"%CD%\\.venv\\Scripts\\llm-wiki-base.exe\"\n\n# Windows — PATH, no Admin\n[Environment]::SetEnvironmentVariable(\"Path\", \"$env:Path;$PWD\\.venv\\Scripts\", \"User\")\n```\n\nFallback on any OS — call through the venv: `./.venv/bin/llm-wiki-base setup tools`\n(Windows: `.venv\\Scripts\\llm-wiki-base.exe`).\n\n## Quickstart\n\n```bash\nmkdir demo && cd demo\nllm-wiki-base setup personal -c commandcode --lang vi   # or -c claude / opencode / zed\necho \"# Notes\\n\\nExpo Router SplitView uses \\`unstable_splitView\\`.\" > raw/inbox/note.md\n\n# One rule — ingest INDEXES, it never writes pages:\nllm-wiki-base wiki ingest raw/inbox/note.md   # (A) deterministic: file INTO search DB\n# (B) writing wiki pages is the SKILL's job: open an AI tool, run skill `llm-wiki-base-ingest`\n\nllm-wiki-base wiki reindex && llm-wiki-base check lint && llm-wiki-base setup doctor\n```\n\n`llm-wiki-base setup doctor` tells you which commands the CLI handles alone and which\nrequire an AI tool. Full command reference: [`docs/cli.md`](docs/cli.md).\n\n## How it works\n\nA document's journey — 9 steps, 3 handoffs of \"who is working\":\n\n1. **Sources land in staging.** You drop files, `scripts/extract_{url,pdf,youtube}.py`\n   generate them, or another AI calls MCP `wiki_submit` — always into `raw/inbox/`.\n   No shortcut writes straight into `wiki/`.\n2. **Ingest = skill + LLM.** Reads the whole source, 3–5 takeaways, discusses with\n   you, picks a domain, then writes `wiki/<domain>/{source,entity,concept}/*.md`\n    with provenance. CLI `llm-wiki-base wiki ingest` does **not** do this — it only indexes.\n3. **Catalog.** `index.md` + `log.md` are re-derivable so the agent writes them;\n   every other assertion stays out. `verified` left blank.\n4. **Derived index.** `llm-wiki-base wiki reindex` by content-hash → `pages_fts`,\n   `chunks_fts`, and vector chunks (when `vector = true`).\n5. **Question → candidate pool.** Skill calls MCP `wiki_search` with\n   `top_k = 2 × top_n_final`.\n6. **RRF.** Three independent ranking channels → fused by rank, never raw scores.\n7. **LLM rerank** (skill, not an in-code model) → cut to `top_n_final`.\n8. **Cited answer** + unverified/stale flags. Good synthesis gets filed back as a\n   new page so the wiki compounds.\n9. **Human gate.** To change/claim: `wiki_propose_edit` → `.proposals/` →\n    `llm-wiki-base review apply --by <you>` → `verify`.\n\nTwo reverse flows keep the wiki from rotting: `llm-wiki-base watch` loops 1→4 on new\nfiles; `llm-wiki-base check lint` (deterministic) then skill `llm-wiki-base-review` (semantic)\npush gaps into `wiki/alerts/`.\n\n**Two-layer architecture:**\n\n- **Global runtime** `~/.llm-wiki-base/` — tools, rag, scripts, one venv. Shared by all wikis.\n- **Per-wiki data** — each wiki is a folder of data only (`raw/`, `wiki/`,\n  `rag/.rag_index/`, `.env`, `.llm-wiki-base.toml`). No code, no venv.\n\n**Three authority tiers** (explains the rest of the design):\n\n| tier | what | written by | deletable |\n|---|---|---|---|\n| `wiki/*.md` | source of truth, with provenance | skill + LLM, human reviews | no |\n| `raw/` | cache of origins (URLs in `sources:` are the real provenance) | you drop / `wiki_submit` | **yes** |\n| `.wiki.db`, `rag/.rag_index/` | derived indexes | `llm-wiki-base wiki reindex` | **yes**, rebuildable |\n\n## Wiki flavors\n\n| | Personal | Project (codebase) |\n|---|---|---|\n| **For** | personal knowledge wiki | wiki of a code repo |\n| **Location** | in-place (cwd) | `<root>/<wiki-dir>/` subfolder |\n| **`[wiki].profile`** | `personal` | `codebase` |\n| **Skills** | `llm-wiki-base-{ingest,query,lint,reindex,review,consolidate,translate}` | same set **+ `llm-wiki-base-research` at repo root** |\n\nBoth flavors **share skill names** — they differ by `[wiki].profile`, not by skill\nset. MCP is centralized (`llm-wiki-base-mcp`): one machine-wide server entry\nreads `registry.toml` to find wikis. Setup guide: [`docs/init.md`](docs/init.md).\n\n## CLI\n\n```bash\nllm-wiki-base setup personal --name \"My Knowledge\" --lang vi\nllm-wiki-base setup project -c claude -c opencode\nllm-wiki-base wiki list                                   # wikis in registry.toml\nllm-wiki-base wiki ingest raw/inbox/foo.md                # index one source (writes no pages)\nllm-wiki-base wiki reindex && llm-wiki-base check lint && llm-wiki-base setup doctor\nllm-wiki-base check eval --compare                        # retrieval A/B with verdict\nllm-wiki-base review apply <name> --by you\nllm-wiki-base upgrade --dry-run                           # skills+configs → newest GitHub tag\nllm-wiki-base translate enable --lang vi --lang ja\n```\n\nFull reference (init flags, per-wiki commands, proposals, translation, doctor):\n[`docs/cli.md`](docs/cli.md).\n\n## Skills\n\n8 skills, **no personal/project name split** — mode comes from `[wiki].profile`.\nInstalled by `llm-wiki-base setup` in two scopes:\n\n| Skill | Scope | Role |\n|---|---|---|\n| `llm-wiki-base-ingest` | wiki | raw → source/entity/concept pages + cross-links + index/log + reindex |\n| `llm-wiki-base-query` | wiki | answer from **the wiki you're in**: retrieval → rerank → cite → file synthesis |\n| `llm-wiki-base-lint` | wiki | **deterministic** health-check: orphans, broken links, frontmatter, index sync |\n| `llm-wiki-base-reindex` | wiki | build/diagnose derived indexes: `--check`, `--full`, dead channels |\n| `llm-wiki-base-review` | wiki | **semantic** health-check: contradictions, staleness, trust gaps → `wiki/alerts/` |\n| `llm-wiki-base-consolidate` | wiki | merge scattered logs/notes → canonical concepts (additive, distill-verify) |\n| `llm-wiki-base-translate` | wiki | translate pages into `[translate].langs` (AI tool's LLM) |\n| `llm-wiki-base-research` | **codebase root** | research **across wikis** via centralized MCP |\n\n`llm-wiki-base-research` lives at `<repo>/.agents/skills/` instead of inside a wiki:\nit must see every wiki, while wiki-scoped skills mind one wiki each. Boundary\nwith `query`: *query* = the wiki you're standing in (may file synthesis),\n*research* = many wikis (staging only).\n\nInstalled skills are **copies** — after upgrading the package, refresh each wiki\nwith `llm-wiki-base upgrade` (backup + overwrite, prunes shipped-away skills, keeps\nyours). See [`docs/upgrading.md`](docs/upgrading.md).\n\n## Configuration\n\nBehavior config per wiki (`.llm-wiki-base.toml`, **committed**). Precedence:\n**env > TOML > default.** Key groups: `[wiki]` (profile, lang), `[retrieval]`\n(fusion, channels, budgets), `[retrieval.weights]`, `[models]` (skill-layer LLM\ncontract — Python never calls an LLM), `[eval]`, `[review]`, `[lifecycle]`,\n`[lint]`, `[translate]`. Changing `embed_model` / `chunk_tokens` / `vector` /\n`fusion` requires `llm-wiki-base wiki reindex --full`. Full annotated example:\n[`docs/cli.md`](docs/cli.md) (config section).\n\n## Retrieval & eval\n\nUnion retrieval + RRF over three independent channels (`bm25_page`,\n`bm25_chunk`, `vector_chunk`), LLM rerank at the skill layer, silent-channel\nalarms instead of silent degradation, and a golden-query eval harness\n(`P@k / R@k / MRR`, `eval --compare` with a vector verdict). Measured findings\n(RRF trade-offs, when vector earns its keep): [`docs/retrieval-eval.md`](docs/retrieval-eval.md).\n\n## Human authority\n\n- **Trust tiers.** Every page has `generated: {by, at}`; optional\n  `verified: {by, at}`. AI never sets `verified` —\n  `llm-wiki-base check verify <page> --by <human-id>`. Retrieval still serves unverified\n  pages, but skills must flag them.\n- **Proposals = the write gate.** AI reads freely; asserting facts requires a\n  human signature: `proposals new/list/show/apply/discard`. MCP\n  `wiki_propose_edit` writes the same format.\n- **Pins.** Important hand edits go in `wiki/pins.yml` and survive regeneration;\n  contradicted pins route to `wiki/alerts/`, never silent reverts.\n\n## MCP bridge\n\nMCP is a **bridge** for AI tools — not a channel for writing to the wiki\ndirectly. Tools: `wiki_search`, `semantic_search`, `wiki_read`, `wiki_list`,\n`list_raw_source`, `read_raw_source`, `wiki_submit` (into `raw/inbox/` only),\n`wiki_propose_edit`, `wiki_lint`. Resources: `registry://wikis`,\n`wiki://<name>/index`, `wiki://<name>/log`. Per-client setup table and entry\nformat: [`docs/mcp.md`](docs/mcp.md).\n\n## Upgrading\n\n```bash\nllm-wiki-base status                 # core local/latest tags + VERSION per wiki\nllm-wiki-base upgrade --dry-run      # preview: which files change, per wiki\nllm-wiki-base upgrade --to latest    # backup → sync skills + agent configs → stamp VERSION\nllm-wiki-base upgrade --to v0.2.0 --wiki my-wiki\n```\n\nUpgrades every wiki in `registry.toml` to a GitHub tag (`vX.Y.Z`); project wikis\nalso re-sync the `codebase` skill at the project root. Migration notes for old\nlayouts and flag details: [`docs/upgrading.md`](docs/upgrading.md).\n\n## Safety & limitations\n\n- **Raw is cache** (gitignored) — delete freely. URLs in `sources:` are the real provenance.\n- **AI proposes, human decides.** Re-derivable writes (index, log) are automatic; factual assertions wait in `.proposals/` for `apply`.\n- **Contradiction = tell the human**, never materialize as an edge or pick a side silently.\n- **Provenance required.** Every claim has a `[[wiki page]]` in-body or a URL in `sources:`.\n- **No writes outside the wiki.** Proposed/proposal paths are normalized and confined to the target wiki's `wiki/`.\n- **Known limits:** eval covers one wiki (no cross-wiki metric yet — see\n  [`docs/tier3-roadmap.md`](docs/tier3-roadmap.md)); `[models]` is a paper\n  contract for the skill layer (no headless ingest yet); `pages.embedding` is\n  written but only read by `fusion=\"weighted\"` (RRF fuses ranks, not vectors).\n\n## Contributing\n\n```bash\npip install -e .[dev]              # CLI venv + pytest/ruff/mypy\nllm-wiki-base base install           # sync src/llm_wiki_base/base_tools → ~/.llm-wiki-base/tools/\nPYTHONPATH=src python -m pytest tests/ -q   # gate 1: tests\nruff check src tests                       # gate 2: lint\nmypy src/llm_wiki_base/registry.py src/llm_wiki_base/cli.py  # gate 3: types\n```\nAll three gates run in CI (`.github/workflows/ci.yml`) and must be green\nbefore any commit/PR.\n\n- Commits follow Conventional Commits (`type(scope): subject`, no AI trailers,\n  never on `main`) — enforced by a pre-commit guard; see skill `git-commit`.\n- Releases (stable + beta, SemVer tags, changelog, `gh release`) — see skill `llm-wiki-base-release`.\n- Source layout: `src/llm_wiki_base/` (CLI + installer + registry),\n  `src/llm_wiki_base/base_tools/` (runs in the base venv, **must not** import the\n  package), `src/llm_wiki_base/base_rag/`, `src/llm_wiki_base/skills/{wiki,codebase}/`\n  (init copy source), `src/llm_wiki_base/templates/`.\n- ⚠️ `src/llm_wiki_base/config_file.py` and `src/llm_wiki_base/base_tools/config_file.py`\n  must match key-for-key (the latter is the fallback when tools run in the base\n  venv without the package). `llm-wiki-base setup doctor` diffs `tools/` against the\n  package and reports drift.\n- Detailed docs: agent schema\n  ([`_schema.md`](src/llm_wiki_base/templates/agents/_schema.md)) · wiki runbook\n  ([`AGENTS.md`](src/llm_wiki_base/templates/agents/AGENTS.md),\n  [`CLAUDE.md`](src/llm_wiki_base/templates/agents/CLAUDE.md)) ·\n  init guide ([`docs/init.md`](docs/init.md)) · roadmap\n  ([`docs/tier3-roadmap.md`](docs/tier3-roadmap.md)) · session history\n  ([`docs/session/`](docs/session/)) · credits ([`docs/credits.md`](docs/credits.md)).\n\n## License\n\nMIT — see [LICENSE](LICENSE) (or `pyproject.toml`).\n",
  "bytes": 14452,
  "sha": "a3e34e28464f5aa354849192e3d79f8e02e6b2e19851adc6900ab0c7e351cd6f",
  "repo_slug": "nihmtaho/llm-wiki-base",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_nihmtaho_llm_wiki_base_docs_human_ideas__5840fbb7/readme"
}