{
  "markdown": "# llm-wiki\n\n[![PyPI](https://img.shields.io/pypi/v/llm-compounding-wiki.svg)](https://pypi.org/project/llm-compounding-wiki/)\n[![Python](https://img.shields.io/pypi/pyversions/llm-compounding-wiki.svg)](https://pypi.org/project/llm-compounding-wiki/)\n[![Docs](https://img.shields.io/badge/docs-github%20pages-blue.svg)](https://krishddd.github.io/llm-wiki/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](./LICENSE)\n\n> A self-healing local knowledge base where a local LLM compounds your\n> documents across four memory tiers — working, episodic, semantic, and\n> procedural — with bi-temporal facts, automatic contradiction resolution,\n> and scheduled memory maintenance.\n\n`llm-wiki` is a FastAPI service that turns a folder of raw documents into a\n**continuously self-organising Markdown wiki**. Drop PDFs, DOCX, PPTX, XLSX,\nHTML, or Markdown into the ingest endpoint and the system extracts entities,\nclaims, and relations; writes confidence-scored pages; and keeps them honest\nover time through bi-temporal fact tracking, Ebbinghaus decay, and weekly\nself-lint runs.\n\nThe wiki conforms to **[Google's Open Knowledge Format (OKF) v0.1](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)**\n— every page carries typed YAML frontmatter and bundle-relative markdown links,\nso the whole knowledge base can be exported as a portable OKF bundle (and\nexternal OKF bundles import directly as curated, high-trust pages).\n\nThe schema is fully described in [`CLAUDE.md`](./CLAUDE.md) and the agent\ntool catalogue in [`AGENTS.md`](./AGENTS.md).\n\n---\n\n## Architecture at a glance\n\n```mermaid\nflowchart LR\n    subgraph IN[\"Input\"]\n        DOCS[\"Raw documents<br/>PDF · DOCX · PPTX · XLSX · HTML · MD\"]\n        OKFIN[\"External OKF bundles<br/>(curated, no LLM pass)\"]\n    end\n\n    subgraph ING[\"Ingest pipeline\"]\n        direction TB\n        REDACT[\"Privacy redaction\"] --> PLAN[\"Agentic chunk plan\"]\n        PLAN --> SUMM[\"Summarise + extract entities/claims\"]\n        SUMM --> CONF[\"Merge + confidence gate\"]\n        CONF --> D2Q[\"Doc2Query questions\"]\n        D2Q --> AUTO[\"Review Autopilot<br/>staged pages verified vs source\"]\n    end\n\n    subgraph STORE[\"Knowledge store\"]\n        direction TB\n        WIKI[\"Markdown wiki = OKF bundle<br/>sources · entities · procedures · episodic\"]\n        KG[\"Knowledge graph<br/>bi-temporal facts, SQLite\"]\n        IDX[\"Indexes<br/>BM25 + dense sub-chunks + hq units\"]\n    end\n\n    subgraph QRY[\"Query pipeline\"]\n        direction TB\n        CACHE{\"Semantic answer cache<br/>cosine ≥ 0.95? (opt-in)\"}\n        CACHE -- \"miss\" --> ORCH[\"Agentic orchestrator\"]\n        ORCH --> RET[\"Hybrid retrieval<br/>RRF · rerank · small-to-big · MMR\"]\n        RET --> SYNTH[\"Synthesis + citations<br/>+ NLI claim verification\"]\n    end\n\n    SCHED[\"Scheduler<br/>decay · promote · review autopilot<br/>lint · procedures · topics\"]\n\n    DOCS --> ING\n    OKFIN --> WIKI\n    ING --> STORE\n    STORE --> QRY\n    CACHE -- \"hit\" --> ANS[\"Cited answer\"]\n    SYNTH --> ANS\n    ANS -- \"save-back if conf ≥ 0.80<br/>(after NLI verification)\" --> WIKI\n    ANS -. \"cache confident answers\" .-> CACHE\n    SCHED --> STORE\n    WIKI -- \"export\" --> OKFOUT[\"Shareable OKF bundle\"]\n```\n\nEvery LLM call is role-based and provider-agnostic — local Ollama by default,\nor any hosted model with one env var (see\n[Bring your own model](#bring-your-own-model--the-provider-fleet)):\n\n```mermaid\nflowchart LR\n    ROLE[\"LLM role<br/>summary / reason / fast / solver / embed / vision\"]\n    ROLE --> Q{\"PROVIDER_ROLE set<br/>+ key + model present?\"}\n    Q -- \"no (default)\" --> OLL[\"Ollama - local models\"]\n    Q -- \"yes\" --> P[\"OpenAI-compatible provider<br/>Groq / GitHub / Gemini / OpenAI /<br/>Claude / Grok / OpenRouter / custom\"]\n    P -- \"HTTP error\" --> OLL\n```\n\n---\n\n## Why this exists\n\nMost \"chat-with-your-docs\" stacks throw documents into a vector store and\nwalk away. After three months they are full of stale claims, duplicate\nentities, and dangling references. `llm-wiki` treats the knowledge base as a\n**living artefact that has to be maintained** — it promotes recurring ideas,\ndecays unreinforced ones, supersedes facts when newer sources contradict\nolder ones, and crystallises repeated query patterns into reusable\nprocedures.\n\n---\n\n## The four memory tiers\n\n| Tier           | Where                              | Lifetime          | Contents                                              |\n|----------------|------------------------------------|-------------------|-------------------------------------------------------|\n| **Working**    | in-process state                   | one request       | retrieved candidates, draft answer                    |\n| **Episodic**   | `wiki/episodic/<date>.md`          | 14 days (config)  | every ingest / query / lint event with correlation IDs|\n| **Semantic**   | `wiki/sources/`, `wiki/entities/`  | indefinite, decays| consolidated pages, auto-generated entity pages       |\n| **Procedural** | `wiki/procedures/` + `procedures.db`| indefinite        | recurring query patterns crystallised into procedures |\n\n**Promotion rules:**\n- A query becomes part of episodic on every successful answer.\n- A topic that recurs ≥ 3 times across ≥ 14 days of episodic auto-promotes\n  to semantic via the daily `promote_episodic_to_semantic` job (04:00 UTC).\n- A query pattern recurring ≥ 5 times with a similar retrieval set becomes a\n  procedure via the weekly `detect_procedures` job (Sun 06:00 UTC).\n- A high-confidence answer (≥ 0.80, ≥ 2 citations) is saved back as\n  `wiki/sources/synthesis-<slug>.md` immediately at query time.\n\n---\n\n## The ingest pipeline\n\n```\nfile (PDF/DOCX/PPTX/XLSX/HTML/MD/TXT)\n   │\n   ▼\nload_elements                     ← multi-format loaders, structure-aware\n   │\n   ▼\nprivacy redaction                 ← strip API keys / JWTs / private keys / passwords\n   │\n   ▼\nagentic ingest plan               ← adaptive chunk size/overlap by content density\n   │\n   ▼\nlayout_aware_chunks               ← atomic tables / images, semantic blocks\n   │\n   ▼\nsummarise per chunk (summary)     ← summary-role model extracts entities + relations\n   │\n   ▼\nmerge (reason role, 3-tier)       ← reasoning-role model consolidates + scores conf\n   │\n   ▼\nextraction-signal floor           ← rich text → confidence bump\n   │\n   ▼\ncontextual preamble (Anthropic)   ← short doc context attached to chunks\n   │\n   ▼\nDoc2Query                         ← the questions this doc answers, indexed as <pid>#hq\n   │\n   ▼\nconfidence gate\n   │\n   ├─ ≥ 0.60 → wiki/sources/<slug>.md\n   └─ <  0.60 → wiki/review/<slug>.md   (awaits human accept)\n   │\n   ▼\nupsert entities + relations into graph.db\n   │\n   ▼\nextract S-P-O claims (reason role)  →  add_fact(valid_from=today)\n   │\n   ▼\ncontradiction detection vs related pages\n   │\n   ├─ concrete contradiction → supersede_fact() on older page\n   └─ composite-score auto-resolver (margin ≥ 0.2) → keep winner, mark loser\n   │\n   ▼\nreconciler — edits pre-existing pages that overlap on ≥ 2 entities\n   │  refines, contradicts, or adds context; staged in wiki/review/edits/\n   │\n   ▼\nmedia nodes (multimodal graph)    ← tables/images/code/formulas → graph + own embeddings\n   │\n   ▼\nrebuild_index + rebuild_entity_pages\n   │\n   ▼\nepisodic_log_entry (correlation_id)\n```\n\nEvery step is logged in JSON to `logs/app.log`; security-relevant events\n(writes, accepts/rejects, contradictions, supersessions) also go to\n`logs/audit.log`.\n\n### Review & the Autopilot\n\nPages scoring below the confidence gate (0.60) land in `wiki/review/` instead of\ngoing live — the ingest-time score is the model's *self-assessment* and errs\ncautious. The **Review Autopilot** then closes the loop automatically with a\nstrictly stronger verification:\n\n```mermaid\nflowchart TD\n    STAGED[\"Staged page<br/>(below 0.60 gate)\"] --> SRC{\"Original source<br/>re-readable?\"}\n    SRC -- no --> HUMAN[\"Left for human<br/>GET /review → accept / reject\"]\n    SRC -- yes --> JUDGE[\"Evidence-grounded judge<br/>page vs source → faithfulness + coverage\"]\n    JUDGE --> GROUND[\"+ deterministic entity-grounding<br/>composite = 0.7·judge + 0.3·grounding\"]\n    GROUND --> BORDER{\"near a<br/>threshold?\"}\n    BORDER -- yes --> SECOND[\"Second judge vote<br/>(reason role) → average\"]\n    BORDER -- no --> DECIDE{\"composite\"}\n    SECOND --> DECIDE\n    DECIDE -- \"≥ 0.70\" --> PROMOTE\n    DECIDE -- \"≤ 0.30\" --> ARCHIVE[\"Auto-archive<br/>→ wiki/archive/ (reversible)\"]\n    DECIDE -- \"0.30–0.70\" --> ANNOTATE[\"Stay in review<br/>annotated with scores + reasons\"]\n    HUMAN -- \"accept\" --> PROMOTE\n    HUMAN -- \"reject\" --> ARCHIVE\n\n    subgraph PROM[\"promote_review_page (shared)\"]\n        direction TB\n        PROMOTE[\"Move review/ → sources/\"] --> PURGE[\"Purge stale review-id units<br/>parent · #hq · #n · #media\"]\n        PURGE --> REIDX[\"Re-index under new id<br/>small-to-big + preamble + Doc2Query\"]\n        REIDX --> REASSIGN[\"reassign_page_id in graph<br/>facts · entities · relations · media\"]\n    end\n```\n\nBoth accept paths — the Autopilot and the human `POST /review/{id}/accept` — funnel\nthrough the shared **`promote_review_page`** helper, so a promoted page is indexed\nexactly like a freshly-ingested one (small-to-big sub-chunks, contextual preamble,\nDoc2Query `#hq`, dense metadata) and its knowledge-graph rows follow it to the new id\ninstead of dangling at the old `review/*` path. Rejection **archives** (reversible),\nnever hard-deletes.\n\n1. an LLM judge re-reads the staged page **against the original source document**\n   and scores faithfulness + coverage (evidence-grounded, not self-assessed);\n2. a deterministic cross-check measures how many of the page's extracted entities\n   literally appear in the source (composite = 0.7 × judge + 0.3 × grounding);\n3. borderline composites get a **second judge vote** from the reason role\n   (a different model when your roles are split) and the votes average;\n4. decision: **≥ 0.70 auto-accept** (moved to `sources/`, indexed, audit-logged),\n   **≤ 0.30 auto-archive** (moved to `wiki/archive/` — reversible, never deleted),\n   **in between → stays in review**, annotated with the judge's scores + reasons\n   (visible via `GET /review` and in the page frontmatter as `auto_review`).\n\nIt runs inline right after ingest for each staged page, daily at 04:30 UTC for\nthe backlog, and on demand via `POST /admin/run/review_autopilot`. Pages whose\nsource can't be re-read (deleted files, machine-generated pages) are always left\nfor a human. Knobs: `REVIEW_AUTOPILOT_ENABLED`, `REVIEW_ACCEPT_THRESHOLD`,\n`REVIEW_REJECT_THRESHOLD`, `REVIEW_SECOND_OPINION`.\n\nFor the (now rare) pages left in review: `GET /review` lists them with the\njudge's annotation, then `POST /review/{id}/accept` or `/reject` — or use the\ndashboard at `/dashboard`.\n\n---\n\n## The query pipeline\n\n```mermaid\nflowchart TD\n    Q[\"User question\"] --> CACHE{\"Semantic answer cache<br/>(opt-in) — cosine ≥ threshold?\"}\n    CACHE -- \"hit\" --> DONE[\"Return stored answer<br/>cached=true\"]\n    CACHE -- \"miss\" --> PROC{\"Crystallized<br/>procedure match?\"}\n    PROC -- \"yes\" --> ANCHOR[\"Recall anchor pages directly\"]\n    PROC -- \"no\" --> RET[\"Decompose · multi-query · HyDE<br/>→ hybrid retrieval → CRAG filter\"]\n    ANCHOR --> SYNTH\n    RET --> SYNTH[\"Adaptive routing → synthesis<br/>numbered citations + per-claim conf\"]\n    SYNTH --> VERIFY[\"Grounding check · CRAG ceiling<br/>NLI-lite claim verification (recalibrates conf)\"]\n    VERIFY --> SAVE{\"conf ≥ 0.80 ∧ ≥ 2 cits?<br/>(evaluated AFTER verification)\"}\n    SAVE -- \"yes\" --> BACK[\"Save-back → wiki/sources/\"]\n    SAVE -- \"no\" --> OUT\n    BACK --> OUT[\"Cited answer + episodic log<br/>+ populate answer cache\"]\n```\n\nThe linear detail:\n\n```\nuser question\n   │\n   ▼\nsemantic answer cache (opt-in)    ← embed Q; cosine ≥ threshold vs recent answers → return cached\n   │                                (complements the EXACT-hash procedural recall)\n   ▼\nintent classifier                 ← factual / multi_hop / synthesis / exhaustive\n   │\n   ▼\ndecompose (if compound)\n   │\n   ▼\nmulti-query paraphrase            ← RAG-Fusion: N rewrites\nHyDE seed for dense retrieval     ← LLM hallucinates a hypothetical doc\n   │\n   ▼\nhybrid retrieval\n   ├─ BM25 over wiki/sources/ (sub-chunks + #hq question units)\n   ├─ dense over Chroma (or numpy fallback)\n   ├─ RRF fuse\n   ├─ FlashRank cross-encoder rerank (graceful passthrough if not installed)\n   ├─ small-to-big: rerank the MATCHED sub-chunks ± neighbours, not page[:4000]\n   ├─ machine-page down-weight ×0.85 (anti-feedback-loop for save-backs)\n   ├─ graph 2-hop expansion via entity links\n   └─ MMR diversification\n   │\n   ▼\nmark_accessed() on every retrieved page → reinforces lifecycle counter\n   │\n   ▼\nCRAG relevance filter             ← drop off-topic candidates\n   │\n   ▼\nadaptive model routing            ← quantitative Q → solver reasons, reasoner formats\n   │\n   ▼\nmultimodal expansion              ← surface tables/figures linked to retrieved entities\n   │\n   ▼\nlost-in-the-middle reorder        ← ends-load context: best page first, runner-up last\n   │\n   ▼\nsynthesis\n   ├─ numbered citations\n   ├─ [Page]^conf markers per claim\n   └─ structured answer blocks\n   │\n   ▼\ngrounding check + CRAG ceiling    ← detect ungrounded statements\n   │\n   ▼\nNLI-lite claim verification       ← ONE batched judge call per answer; unsupported\n   │                                claims drag per-claim + overall confidence down\n   ▼\nreflection critique → optional refinement\n   │\n   ▼\nrecord_query_pattern() in procedural store\n   │\n   ▼\nsave-back if confidence ≥ 0.80 ∧ citations ≥ 2   ← evaluated AFTER NLI verification, so a\n   │                                               page is never persisted with an inflated score\n   ▼\npopulate semantic answer cache (grounded ∧ conf ≥ min)   [QUERY_ANSWER_CACHE, off by default]\n   │\n   ▼\nepisodic_log_entry\n```\n\n---\n\n## 2026 adaptive upgrades\n\nBeyond the base pipeline, the system adapts to *what kind* of content and question\nit is handling. Each upgrade is flag-gated and degrades gracefully when its model\nisn't installed.\n\n| Upgrade | What it does | Flag (default) |\n|---|---|---|\n| **Adaptive model routing** | Quantitative questions (maths, economics, science, engineering) are reasoned by [VibeThinker](https://github.com/WeiboAI/VibeThinker) — a maths/STEM specialist — then the reasoning-role model formats + cites the result. Plain-English questions skip it. | `ROUTE_SOLVER_ENABLED` (on; self-disables if `MODEL_SOLVER` not served) |\n| **Domain detection + tagging** | Every page and query is classified general / math / science / economics / engineering, driving routing and retrieval. | always on |\n| **Agentic retrieval** | The `/query` front door auto-routes simple questions to a fast single pass and complex ones to an iterative plan → retrieve → sufficiency-check → gap-rewrite loop. | `AGENTIC_ENABLED` (on) |\n| **Agentic ingestion** | Per-document adaptive chunk sizing — dense technical content gets smaller chunks, narrative prose larger. | `INGEST_AGENTIC_PLANNING` (on) |\n| **Privacy redaction** | Strips API keys, JWTs, private keys, and passwords from raw sources before ingest; audit-logged as `PRIVACY_REDACT`. | `INGEST_REDACT_SECRETS` (on) |\n| **STEM embeddings** | A stronger, notation-aware embedder (`bge-m3`) in a separate dense collection for quantitative content; routed by domain. | `EMBED_STEM_ENABLED` (off) |\n| **Multimodal graph** | Tables/images/code/formulas become first-class graph nodes linked to entities and embedded as their own units; retrieval surfaces media linked to the entities in play. | `GRAPH_MULTIMODAL_NODES` (off) |\n| **Semantic answer cache** | Embeds each question and short-circuits near-duplicates (cosine ≥ threshold) with a stored answer — a fuzzy layer above the procedural store's EXACT pattern hash. Only grounded, confident answers are cached; entries carry a TTL + count cap. A **staleness guard** re-answers instead of serving a hit whose cited pages have since been archived/rejected/superseded, so a loosely-tuned threshold can never surface an answer built on pages that are gone. | `QUERY_ANSWER_CACHE` (off) |\n\nSee [`CLAUDE.md`](./CLAUDE.md) for the schema details and\n[`docs/design/multimodal-graph.md`](./docs/design/multimodal-graph.md) for the\nmultimodal-graph rollout.\n\n---\n\n## Governance & learning\n\nTwo capabilities keep the corpus *correct* and let it *learn* — the wiki isn't just\nretrieved from, it's governed and improved over time.\n\n### Profile / schema contract (runtime-enforced)\n\nThe wiki's conventions — allowed `kind`s, required frontmatter, valid `domain`/type\nvalues, confidence bounds, the entity/relation vocabularies — are a **declarative\ncontract validated at the page write surface**, not just prose in `CLAUDE.md`. A\nmalformed page is caught deterministically instead of drifting in.\n\n- **Modes** (`PROFILE_ENFORCEMENT`): `off` · `warn` (default — logs + audits, still\n  writes) · `strict` (raises `ProfileViolation` so ingest routes the page to review).\n- The built-in `DEFAULT_PROFILE` matches the current schema exactly (nothing the\n  pipeline already produces is rejected); override any subset of keys with a\n  `PROFILE_PATH` JSON file.\n- `GET /profile` shows the active contract; `POST /admin/profile/validate` audits the\n  whole live corpus for drift without rewriting anything.\n\n### Feedback curator (corrections → memory)\n\nThe episodic tier logs what *happened*; the feedback curator captures what the user\n*corrected or preferred* and turns it into durable memory. Submit feedback on an\nanswer via `POST /feedback`; it is classified (a cheap heuristic drops generic acks\nlike \"thanks\" with no LLM call, else one reason-role call):\n\n| Kind | On promotion |\n|---|---|\n| **correction** | writes a curated high-confidence `sources/feedback-*.md` page, indexed like any source |\n| **preference** | becomes an **active preference** injected into every future synthesis prompt (\"honour these\") |\n| **approval** | reinforces the cited page's lifecycle access counter |\n| **rejection** | recorded and flagged for review |\n| **noise** | dropped |\n\nPromotion is explicit by default (`GET /feedback` → `POST /feedback/{id}/promote` or\n`/dismiss`); set `FEEDBACK_AUTO_PROMOTE=true` to auto-apply high-signal\ncorrections/preferences at capture time. Validated live against hosted models — all\nfive categories classified correctly, actionable content extracted cleanly.\n\n---\n\n## Best-of-best RAG package (v5)\n\nSix further techniques, each flag-gated and on by default:\n\n| Technique | What it does | Flag (default) |\n|---|---|---|\n| **Small-to-big retrieval** | Reranks/synthesises the 1500-char sub-chunks that actually matched (± neighbours), re-derived exactly as indexed — instead of the first 4000 chars of the page. Fixes relevant content beyond the prefix never reaching the LLM. | `QUERY_CHUNK_CONTEXT` (on) |\n| **Doc2Query** (Nogueira & Lin) | At ingest, generates the questions each document answers and indexes them as `<pid>#hq`, so question-phrased queries match declarative text. | `INGEST_DOC2QUERY` (on) |\n| **Lost-in-the-middle reorder** (Liu et al. 2023) | Ends-loads the synthesis context — best page first, runner-up last — to counter positional attention decay. | `QUERY_LITM_REORDER` (on) |\n| **NLI-lite claim verification** | One batched judge call checks every cited claim sentence against its cited snippet; unsupported claims get ×0.35 confidence. Catches \"right page, wrong claim\". | `QUERY_CLAIM_VERIFY` (on) |\n| **Machine-page down-weight** | Synthesis/promoted/crystallized pages score ×0.85 at rerank so save-backs never outrank the primary sources they came from. | `RETRIEVAL_SYNTH_DOWNWEIGHT` (0.85) |\n| **RAPTOR-lite topics** (Sarthi et al. 2024) | Weekly clustering of live pages into `topic-*.md` overview pages, so corpus-level questions (\"main themes across my documents?\") have a retrievable answer. | `JOB_BUILD_TOPICS_ENABLED` (on) |\n\nPages ingested before v5 need a one-off backfill for the `#hq` units and topics:\n\n```bash\npython scripts/backfill_v5.py                  # uses configured models\npython scripts/backfill_v5.py --summary-model llama3.2:latest --reason-model llama3.2:latest\n```\n\n---\n\n## Evaluation harness\n\nWith ~16 stacked techniques, measure what actually pays for its latency on\n**your** corpus:\n\n```bash\npython scripts/gen_golden.py --n 15            # LLM-generate golden Q/page pairs → eval/golden.jsonl\npython scripts/run_eval.py                     # retrieval baseline: recall@k, MRR, hit-rate, latency\npython scripts/run_eval.py --ablate            # + one-flag-off variants (chunk-context, MMR, down-weight, graph)\npython scripts/run_eval.py --answers           # + full answer eval: keyword coverage, groundedness, confidence\n```\n\nRetrieval eval needs only the embedding model (cheap; run per ablation).\nAnswer eval runs the full pipeline per question. Results land in\n`eval/results-<label>.json`. Hand-edit `eval/golden.jsonl` freely — an\nLLM-generated golden set inherits its generator's blind spots.\n\n---\n\n## Validation — tested end-to-end on hosted models\n\nThe pipeline was exercised end-to-end against **NVIDIA `build.nvidia.com`** hosted\nmodels (fully local-GPU-free), routed through the provider fleet — text via\n`meta/llama-3.1-8b-instruct`, embeddings via `nvidia/nv-embedqa-e5-v5` (1024-dim).\nEvery stage ran through a real model, not a mock.\n\n**What was tested and the numbers that came back:**\n\n| Stage tested | Result |\n|---|---|\n| Provider connectivity (chat + embed) | chat ≈ 0.8 s/call · embed ≈ 0.3 s/call · `/models` auth ✓ |\n| Ingest — 3 docs (RAG, vector DBs, Transformers) | all went **live** at **confidence 0.95**; entities + relations + Doc2Query extracted |\n| Query retrieval | correct source page retrieved for **every** question |\n| Synthesis factual accuracy | facts correct (RAG → Patrick Lewis / Facebook AI / 2020; Transformer → Vaswani / Google / 2017; FAISS/HNSW for vector DBs) |\n| Eval harness — recall@5 / MRR / hit-rate | **1.000 / 1.000 / 1.000** across baseline + 4 ablations (chunk-context, MMR, down-weight, graph); ≈ 3–5 s/variant |\n| Answer-cache precision @ threshold 0.80 | paraphrase → **cache hit**, unrelated question → **miss** (correct both ways) |\n| Unit + integration suite | **145 passed**, ruff-clean |\n\n**Answer-cache threshold, calibrated on real `nv-embedqa` cosines** (replacing the\nconservative 0.95 guess — cosine scale is embedder-specific):\n\n| Question pair | Cosine |\n|---|---|\n| \"What is RAG and who introduced it?\" ~ \"Who created RAG and what is it?\" | **0.928** |\n| \"What are the two components of RAG?\" ~ \"What are RAG's main parts?\" | **0.857** |\n| loose reformulation (hallucination wording) | 0.589 |\n| RAG question **vs** \"How do transformers use attention?\" | 0.201 |\n| RAG question **vs** \"capital of France?\" / \"quicksort complexity?\" | ≈ 0.19–0.20 |\n\nParaphrases cluster at **0.86–0.93**, unrelated questions at **≈ 0.20** — a wide safe\ngap, so **`ANSWER_CACHE_SIM_THRESHOLD ≈ 0.80`** is a high-precision cut for this\nembedder (validated live: it hit paraphrases and rejected unrelated questions).\n\n**Honest caveats (what these numbers do and don't prove):**\n- The eval corpus was **3 very distinct docs**, so retrieval is trivial and every\n  ablation scores a perfect 1.000 — this proves the harness is **push-button and\n  correct**, not that any single technique lifts recall. Differentiating the\n  techniques needs a larger, more *confusable* corpus.\n- `meta/llama-3.1-8b-instruct` is fast but doesn't reliably emit the `[Title]^0.NN`\n  citation markers, so `grounded` is penalized (the answer *content* is still\n  correct). For production-grade citation/grounding, use a larger reasoner\n  (`meta/llama-3.3-70b-instruct` or a `qwen` tier) at the cost of latency.\n- Live testing also surfaced a real robustness fix: hosted models emit JSON with\n  **literal newlines inside strings** (from `##` markdown), which strict `json.loads`\n  rejected — hardened to `strict=False` across all 20 LLM-output parse sites, which\n  lifted a query answer from an unparsed blob (confidence 0.2) to clean markdown\n  (confidence 0.75). This is exactly the class of bug that only appears under a real\n  model.\n\nReproduce with the NVIDIA block in `.env.example` (`PROVIDER_*=nvidia` + `NVIDIA_API_KEY`),\nthen the ingest / query / eval commands above.\n\n### Local single-LLM run — vLLM (Nemotron-4B) + hosted reason + OpenAI embeddings\n\nA second end-to-end run exercised a **mixed local/hosted fleet** on a fresh corpus of\n3 DOCX files (agent reliability, long-horizon planning, metamorphic testing):\n\n| Role | Provider · model |\n|---|---|\n| summary / fast / solver | **vLLM** `nvidia/Nemotron-Mini-4B-Instruct` (self-hosted, `custom` provider) |\n| reason (synthesis / lint / claim-verify) | **OpenAI** `gpt-4.1-mini` (fast, 128K context) |\n| embeddings | **OpenAI** `text-embedding-3-small` |\n\nThis split — a tiny self-hosted model for the high-volume per-chunk work, a fast\nlarge-context hosted model for the whole-corpus reasoning, hosted embeddings — is the\npractical shape for a single-box setup.\n\n**Results:**\n\n| Endpoint / stage | Result |\n|---|---|\n| `/ingest` — 3 DOCX | ✅ 3 pages live, **385 entities / 406 relations**, no crash |\n| `/query` | ✅ grounded, 3 sources cited **with their tables**, intent `factual`, ~62 s |\n| `/query/agentic` | ✅ **HTTP 200 in ~41 s** — 7 sub-queries, multi-hop synthesis, 3 pages cited, clean prose (after the bug fix below) |\n| `/lint` | ✅ **HTTP 200 in ~3 s** — surfaced 18 missing entity pages |\n| Read endpoints (`/entities`, `/facts/{name}`, `/entities/{id}`, `/context/start`, `/wiki/index`, `/episodic`, `/profile`, `/review`, `/admin/jobs`, `/admin/contradictions`) | ✅ all instant |\n| `/admin/profile/validate` | ✅ 370 items checked, 0 violations |\n| `/feedback` (POST + GET) | ✅ stored, classified `correction`, actionable text extracted |\n\n**Reason-provider choice mattered — the same two endpoints on other providers:**\n\n| reason provider | `/lint` | `/query/agentic` |\n|---|---|---|\n| vLLM `Nemotron-Mini-4B` | ❌ `400` (context window too small for whole-corpus prompt) | partial (sub-calls overflow) |\n| NVIDIA free-tier `llama-3.3-70b` | ❌ `ReadTimeout` at 600 s (free-tier latency) | ❌ 500 after ~28 min |\n| **OpenAI `gpt-4.1-mini`** | ✅ **3 s** | ✅ **41 s** |\n\n**Limits surfaced (all infra/model, not pipeline):**\n- **Nemotron-Mini-4B has a small context window** — whole-corpus or snippet-heavy\n  prompts (`/lint`, some agentic sub-calls) overflow it and the vLLM returns\n  `400 Bad Request`, and the 4B model doesn't reliably emit the `[Title]^0.NN` citation\n  schema (raw `/query` answers can come back double-wrapped in JSON). Both clear up once\n  the **reason** role is a larger model — the retrieval/citations underneath are correct\n  either way. Keep the 4B on the high-volume summary/fast roles where it's a good fit.\n- **Free-tier hosted reasoners are latency-bound** — the whole-corpus `/lint` prompt\n  exceeds the 600 s client timeout on the NVIDIA free tier. Use a **fast** large-context\n  reason provider (OpenAI `gpt-4.1-mini`, Groq) or raise `LLM_TIMEOUT`.\n- **`grounded` is strict about citation markers** — `gpt-4.1-mini` cites sources but\n  doesn't always emit the literal `[Title]^0.NN` markers the grounding check scans for,\n  so the flag can read `false` on an answer that is in fact correct and cited.\n\n**Bug fixed during this run:** the agentic orchestrator temporarily swaps\n`QueryEngine._retrieve_one` for a stub during final synthesis, but the stub's signature\ndidn't accept the `use_mmr` argument the engine passes — so **every** `/query/agentic`\ncall crashed with `TypeError: _stub() got an unexpected keyword argument 'use_mmr'`.\nFixed in [`llm_wiki/agentic_rag/agentic_query.py`](./llm_wiki/agentic_rag/agentic_query.py)\nby making the stub accept (and ignore) extra retrieval kwargs.\n\n---\n\n## OKF bundles — import & export\n\nThe wiki *is* an OKF bundle. Two scripts make that portable:\n\n```bash\n# Export the stable tiers (sources/entities/procedures + index.md + log.md)\n# as a standalone, validated OKF bundle — share as a git repo or archive:\npython scripts/export_okf.py dist/my-wiki-bundle\n\n# Import someone else's OKF bundle as curated pages — no LLM pipeline, pages\n# copy 1:1 with provenance stamped, links become RELATES_TO graph edges:\npython scripts/import_okf.py path/to/their-bundle\npython scripts/import_okf.py path/to/their-bundle --validate-only   # conformance check\n\n# Re-stamp pages written before OKF conformance (idempotent):\npython scripts/migrate_okf.py\n```\n\n---\n\n## Confidence and decay\n\n- **Stored confidence** is what the LLM assigned at ingest time; reads do\n  not mutate it.\n- **Effective confidence** = stored × `exp(-Δdays / half_life_days)`, floored\n  at 0.05. Default half-life is 90 days.\n- **Reinforcement** triggers when a page is accessed ≥ 3 times within a\n  14-day window; the reinforcement timestamp resets the decay clock.\n- The **decay sweep** (daily 03:00 UTC) rewrites stored confidence based on\n  `last_reinforced`.\n\nBi-temporal facts carry `ingested_at`, optional `valid_from`, optional\n`valid_to`, `superseded_by`, `last_reinforced`, and `access_count`. A new\nsource can never *delete* an old fact — only mark it superseded by setting\n`valid_to = today` and `superseded_by = <new_fact_id>`.\n\nThree triggers can supersede:\n1. **Reconciler auto-apply** when `action ∈ {refine, contradict}` lands and\n   the old text matches.\n2. **Contradiction detector** when `_detect_contradictions` returns a\n   concrete claim excerpt.\n3. **Auto-resolver** (Phase E2) when the composite-score margin ≥ 0.2;\n   sub-margin cases stay surfaced in `GET /admin/contradictions` for human\n   review.\n\n---\n\n## Scheduled jobs\n\nIn-process APScheduler runs the following by default (each toggleable via env\nvar). Manual one-off runs available via `POST /admin/run/{job_name}`.\n\n| UTC time         | Job                  | Toggle env var                     |\n|------------------|----------------------|------------------------------------|\n| daily 03:00      | `decay_sweep`        | `JOB_DECAY_SWEEP_ENABLED`          |\n| daily 03:30      | `episodic_prune`     | `JOB_EPISODIC_PRUNE_ENABLED`       |\n| daily 04:00      | `promote_episodic`   | `JOB_PROMOTE_EPISODIC_ENABLED`     |\n| daily 04:30      | `review_autopilot`   | `JOB_REVIEW_AUTOPILOT_ENABLED`     |\n| weekly Sun 05:00 | `lint_autofix`       | `JOB_LINT_AUTOFIX_ENABLED`         |\n| weekly Sun 06:00 | `detect_procedures`  | `JOB_DETECT_PROCEDURES_ENABLED`    |\n| weekly Sun 07:00 | `page_compaction`    | `JOB_PAGE_COMPACTION_ENABLED`      |\n| weekly Sun 07:30 | `build_topics`       | `JOB_BUILD_TOPICS_ENABLED`         |\n\n---\n\n## Models\n\n| Role                         | Model                       | Notes                                  |\n|------------------------------|-----------------------------|----------------------------------------|\n| Summarise, extract           | `gemma4:e4b`                | Fast, strong instruction-following     |\n| Reason, route, lint, claims  | `qwen3:14b`                 | Deep reasoning, thinking mode          |\n| Quantitative specialist      | `vibethinker:3b`            | AIME-class maths/STEM; routed to adaptively |\n| Embeddings                   | `nomic-embed-text:latest`   | 274 MB, MTEB-strong                    |\n| STEM embeddings (optional)   | `bge-m3`                    | Notation-aware; `EMBED_STEM_ENABLED`   |\n| Vision (image captions)      | `llava:7b`                  | Optional, when `ingest_caption_images` |\n\nServed via Ollama at `OLLAMA_HOST` (default `http://localhost:11434`) by default.\n\n### Bring your own model — the provider fleet\n\nEvery LLM role can be pointed at **any** hosted or local provider that speaks the\nOpenAI wire format. Clone the repo, copy `.env.example` to `.env`, set\n`PROVIDER_<ROLE>` + that provider's key, run — good to go. A role falls back to\nOllama automatically when its key/model is missing, and a provider HTTP error\ndegrades to the existing Ollama role fallback, so misconfiguration never breaks\nthe pipeline. Keys live only in your local `.env` (gitignored) — never commit them.\n\n| Provider | `PROVIDER_<ROLE>=` | Key env var | Default model | Embeddings? |\n|---|---|---|---|---|\n| **Ollama** (default) | `ollama` | — (local) | `qwen3:14b` / `gemma4:e4b` | ✅ `nomic-embed-text` |\n| **Groq** | `groq` | `GROQ_API_KEY` | `llama-3.3-70b-versatile` | — |\n| **GitHub Models** | `github` | `GITHUB_MODELS_TOKEN` | `openai/gpt-4.1-mini` | — |\n| **Google Gemini** | `gemini` | `GOOGLE_GENAI_API_KEY` | `gemini-2.5-flash-lite` | ✅ `text-embedding-004` |\n| **OpenAI** | `openai` | `OPENAI_API_KEY` | `gpt-4.1-mini` | ✅ `text-embedding-3-small` |\n| **Anthropic Claude** | `anthropic` | `ANTHROPIC_API_KEY` | `claude-sonnet-5` | — |\n| **xAI Grok** | `xai` | `XAI_API_KEY` | `grok-4` | — |\n| **OpenRouter** | `openrouter` | `OPENROUTER_API_KEY` | `meta-llama/llama-3.3-70b-instruct` (100+ OSS models, one key) | — |\n| **Custom / self-hosted** | `custom` | `CUSTOM_API_KEY` (optional) | `CUSTOM_MODEL` @ `CUSTOM_BASE_URL` | ✅ `CUSTOM_EMBED_MODEL` |\n\n`custom` covers **any OpenAI-compatible gateway**: vLLM (`http://localhost:8001/v1`),\nLM Studio (`http://localhost:1234/v1`), llama.cpp server, Together, Fireworks,\nDeepSeek, Mistral La Plateforme, … — no code changes, no API key needed for local\ngateways.\n\n```bash\n# Example mixed fleets (set in .env):\n\n# Claude reasons, Groq handles the fast path, everything else local:\nPROVIDER_REASON=anthropic   ANTHROPIC_API_KEY=sk-ant-...\nPROVIDER_FAST=groq          GROQ_API_KEY=gsk_...\n\n# Fully hosted, zero local GPU:\nPROVIDER_REASON=openai      OPENAI_API_KEY=sk-...\nPROVIDER_SUMMARY=gemini     GOOGLE_GENAI_API_KEY=...\nPROVIDER_EMBED=openai\nPROVIDER_FAST=xai           XAI_API_KEY=xai-...\n\n# Your own vLLM box serving an open-source model:\nPROVIDER_REASON=custom      CUSTOM_BASE_URL=http://localhost:8001/v1  CUSTOM_MODEL=qwen2.5-72b-instruct\n```\n\nRoles: `PROVIDER_SUMMARY` (summarise/extract), `PROVIDER_REASON` (synthesis / deep\nreasoning), `PROVIDER_FAST` (fast-agent), `PROVIDER_SOLVER` (quantitative\nspecialist), `PROVIDER_EMBED` (embeddings), `PROVIDER_VISION` (image captions via\nOpenAI `image_url`). All default to `ollama`.\n\n> **Embeddings** are supported by `ollama`, `gemini`, `openai`, and `custom` —\n> the other providers expose no embeddings endpoint and fall back to Ollama.\n> **Heads-up:** switching embedders mid-corpus requires a re-index (two embedders\n> = two incompatible vector spaces).\n\n---\n\n## Quickstart\n\n### Install from PyPI\n\n```bash\npip install llm-compounding-wiki          # core pipeline + FastAPI app\npip install \"llm-compounding-wiki[mcp]\"   # + agent-facing MCP server\npip install \"llm-compounding-wiki[ocr]\"   # + scanned-PDF OCR (needs Tesseract/Poppler)\n\nllm-wiki serve --port 8000                # run the API (uvicorn llm_wiki.api:app)\nllm-wiki mcp                              # run the MCP server\n```\n\nThe import package is `llm_wiki`; the distribution on PyPI is `llm-compounding-wiki`.\n\n### From source\n\n```bash\ngit clone https://github.com/krishddd/llm-wiki.git\ncd llm-wiki\npip install -e \".[dev]\"    # or: pip install -r requirements.txt\ncp .env.example .env\n\n# Option A — fully local (default): pull the Ollama models\nollama pull qwen3:14b\nollama pull gemma4:e4b\nollama pull nomic-embed-text\n\n# Option B — bring your own model: no Ollama needed, just set a provider in .env\n#   PROVIDER_REASON=anthropic  ANTHROPIC_API_KEY=sk-ant-...   (or openai / gemini /\n#   groq / github / xai / openrouter / custom — see the provider table above)\n\n# Run the API\nuvicorn llm_wiki.api:app --reload --port 8000\n```\n\nIngest a doc, then ask a question:\n\n```bash\ncurl -F files=@paper.pdf http://localhost:8000/ingest\ncurl -X POST http://localhost:8000/query \\\n     -H 'Content-Type: application/json' \\\n     -d '{\"question\": \"What did the paper conclude about transformer scaling?\"}'\n```\n\nRun the agent over MCP:\n\n```bash\npython -m llm_wiki.mcp_server   # exposes ingest / query / lint as MCP tools\n```\n\nTrigger a job manually:\n\n```bash\ncurl -X POST http://localhost:8000/admin/run/promote_episodic\n```\n\n---\n\n## Project structure\n\n```\nllm_wiki/\n├── api.py                 FastAPI endpoints\n├── ingest.py              Multi-format ingest pipeline (+ Doc2Query)\n├── query.py               Hybrid retrieval + reflective synthesis + save-back\n├── eval_harness.py        Golden-set evaluation: recall@k/MRR + answer quality + ablations\n├── lint.py                Health check + auto-fix\n├── graph.py               Bi-temporal knowledge graph (SQLite-backed)\n├── llm.py                 Async Ollama client (cached embeddings)\n├── config.py              pydantic-settings — all knobs\n├── logging_config.py      JSON logs + audit channel\n├── scheduler.py           APScheduler + JOB_REGISTRY\n├── mcp_server.py          Agent-facing MCP wrapper\n├── search/                BM25, dense, RRF, FlashRank, MMR, multi-query, intent,\n│                          chunks (small-to-big reconstruction)\n├── synth/                 Answer blocks, per-claim confidence, claim verify,\n│                          reflect, followups\n├── loaders/               Multi-format (PDF, DOCX, PPTX, XLSX, HTML, MD, TXT)\n│                          + OKF bundle loader\n└── wiki/                  Page store (OKF stamping), episodic, promote, procedures,\n                           reconciler, lifecycle, contradiction_resolver,\n                           entity_pages, topics (RAPTOR-lite), index_md, log_md,\n                           okf_export, review_autopilot, review_promote (shared\n                           review→sources promotion), reindex (shared indexing\n                           primitives), answer_cache (semantic answer cache)\n\nscripts/\n├── migrate_okf.py         Re-stamp pre-OKF pages (idempotent)\n├── backfill_v5.py         Backfill #hq units + topic pages for older ingests\n├── gen_golden.py          Generate eval/golden.jsonl from the live wiki\n├── run_eval.py            Run retrieval/answer eval (+ --ablate variants)\n├── import_okf.py          Import an external OKF bundle (curated, no LLM)\n└── export_okf.py          Export the wiki as a standalone OKF bundle\n\nwiki/\n├── index.md               Auto-regenerated table of contents\n├── log.md                 Append-only operation log\n├── sources/               Semantic tier — primary citable content\n├── entities/              Semantic tier — auto-generated entity pages\n├── procedures/            Procedural tier — recurring patterns\n├── episodic/<date>.md     Episodic tier — append-only daily logs\n├── archive/               Pages auto-moved here by lint auto-fix\n├── review/                Confidence-gated drafts awaiting human accept\n│   └── edits/             Reconciler-staged edit proposals\n└── raw/                   Immutable source documents\n\ndata/\n├── graph.db               SQLite: entities, relations, facts, page_access\n├── procedures.db          SQLite: recurring query patterns\n├── answer_cache.db        SQLite: semantic answer cache (opt-in)\n├── bm25.pkl               BM25 index\n└── chroma/                ChromaDB persistence (or numpy fallback)\n\nlogs/\n├── app.log                Rotating JSON, all events\n└── audit.log              Filtered audit channel\n```\n\n---\n\n## Page frontmatter convention\n\nConforms to OKF v0.1: `type` (OKF's one required field), `description`,\n`resource`, and `timestamp` are stamped centrally by `write_page()` on every\nwrite, so all writers conform automatically.\n\n```yaml\n---\ntitle: \"Page Title\"\nkind: source | entity | synthesis | promoted | crystallized | procedure | topic\ntype: \"Source Document\" | \"Person|Organization|Concept|Place|Event\" | \"Synthesis\" | \"Topic Overview\"\ndescription: \"One-sentence summary (first prose sentence of body if not supplied)\"\nresource: \"wiki/raw/file.pdf\"          # OKF URI of the underlying asset\ntimestamp: \"2026-07-15T16:51:36+00:00\" # ISO 8601 last modification, auto-stamped\nsource: \"wiki/raw/file.pdf\" | \"query-save-back\" | \"episodic-promotion\"\ningested: 2026-05-01\nconfidence: 0.87\nconfidence_reason: \"...\"\ndomain: general | math | science | economics | engineering\nchunk_count: 14             # sub-chunks indexed — lets re-index / promotion purge exactly\ntags: [concept, person, org]\nentity_refs: [\"Entity A\", \"Entity B\"]\nhypothetical_questions: [\"What does …?\"]  # Doc2Query, indexed as <pid>#hq\nauto_review:                # stamped by Review Autopilot (accepted / annotated pages)\n  {composite: 0.82, verdict: auto-accepted, faithfulness: 0.9, coverage: 0.8}\ncontext_preamble: \"...\"     # Anthropic Contextual Retrieval\nhas_tables: true\nhas_images: false\nelement_counts: {text: 14, heading: 6, table: 3, image: 0, code: 0}\nevolved_by:                 # populated when reconciler edits this page\n  - {source: \"Foo Doc\", action: \"refine\", date: 2026-05-15}\ncorrelation_ids: [COR-...]  # crystallized / promoted only\n---\n```\n\n---\n\n## CI & local development\n\nGitHub Actions runs ruff, mypy, pytest (Ollama mocked), and a Docker build on\nevery push to `main`. Strict ruff config lives in `pyproject.toml`. The\nintegration suite (`workflows/integration.yml`) is gated behind a manually\ntriggered `workflow_dispatch` plus a `REMOTE_OLLAMA_HOST` secret, so day-to-day\nCI never depends on a live LLM.\n\nTwo more workflows handle release:\n\n- **`publish.yml`** — builds the sdist/wheel and publishes to PyPI via\n  [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) (OIDC, no stored\n  token) when a GitHub Release is published.\n- **`docs.yml`** — builds the MkDocs Material site and deploys it to GitHub Pages\n  on every push to `main` that touches the docs.\n\n---\n\n## Status\n\nPersonal research project. Explores how far a local-first LLM-driven wiki\ncan self-organise without a human curator.\n\n## Contributing\n\nContributions are welcome — see [CONTRIBUTING.md](./CONTRIBUTING.md) for dev\nsetup, tests, and the release flow. Notable changes are tracked in\n[CHANGELOG.md](./CHANGELOG.md).\n\n## License\n\n[MIT](./LICENSE)\n",
  "bytes": 41954,
  "sha": "637f8dffcefd0d877da09b811000e53574cee418ba96279809801f86c97e1127",
  "repo_slug": "krishddd/llm-wiki",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_krishddd_llm_wiki_wiki_index_md_c4d11e7d/readme"
}