{
  "markdown": "<p align=\"center\">\n  <img src=\"assets/brand/png/logo-tile-512.png\" alt=\"TokenKnows logo\" width=\"140\" />\n</p>\n<h1 align=\"center\">TokenKnows</h1>\n<p align=\"center\">\n  Distill AI coding sessions into living knowledge — weekly reports, ADRs,\n  incident reviews, books, agent skills, and a knowledge graph.\n</p>\n<p align=\"center\">\n  <a href=\"LICENSE\"><img src=\"https://img.shields.io/github/license/johnnywuj81/tokenknows?color=d97757\" alt=\"License\"></a>\n  <a href=\"https://github.com/johnnywuj81/tokenknows/actions/workflows/ci.yml\"><img src=\"https://github.com/johnnywuj81/tokenknows/actions/workflows/ci.yml/badge.svg\" alt=\"CI\"></a>\n  <img src=\"https://img.shields.io/badge/Claude_Code-plugin-d97757\" alt=\"Claude Code plugin\">\n  <img src=\"https://img.shields.io/badge/MCP-server-241b15\" alt=\"MCP server\">\n  <img src=\"https://img.shields.io/badge/PRs-welcome-788c5d\" alt=\"PRs welcome\">\n</p>\n<p align=\"center\"><b>English</b> | <a href=\"README.zh-CN.md\">简体中文</a></p>\n\n<p align=\"center\">\n  <img src=\"assets/demo/tokenknows-demo.gif\" alt=\"TokenKnows demo: capture an AI coding session, distill it into a weekly report and knowledge graph\" width=\"920\" />\n</p>\n\n\n---\n\n## What is TokenKnows?\n\nYou spend hours pair-programming with Claude Code, Codex, and Cursor. The decisions, bug hunts, and design trade-offs from those sessions evaporate the moment the terminal closes. TokenKnows captures them automatically and distills them into **structured, evidence-linked knowledge assets**:\n\n**capture** (6 collectors) → **distill** (5-stage LLM pipeline) → **assets** (7 document types) → **review / redact / publish**\n\n- 📡 **Captures everything** — Claude Code, Codex, Cursor, VS Code, GitHub PRs/commits/issues, and local docs, all via local file watchers and API polling. No webhooks, no tunnels.\n- 📝 **Seven asset types** — weekly reports, tech designs, ADRs, incident reviews, long-form books, reusable agent skills (SKILL.md), and an entity knowledge graph.\n- 🔗 **Evidence-linked** — every paragraph traces back to the original PR / conversation / commit, ranked by `cosine × trust × recency` across ≥2 sources.\n- 🔒 **Local-first, zero egress by default** — a three-layer LLM egress gate (instance ∧ project ∧ task) with full audit logging. Pair it with Ollama and run the whole pipeline with **zero cloud keys**.\n\n## Demo\n\n| Workbench | Document page |\n|---|---|\n| [![](engineering_handoff/demo-screenshots/01-workbench.png)](engineering_handoff/demo-screenshots/01-workbench.png) | [![](engineering_handoff/demo-screenshots/04-document-page.png)](engineering_handoff/demo-screenshots/04-document-page.png) |\n| **Evidence drawer** | **Publish receipt + version diff** |\n| [![](engineering_handoff/demo-screenshots/05-evidence-drawer.png)](engineering_handoff/demo-screenshots/05-evidence-drawer.png) | [![](engineering_handoff/demo-screenshots/10-publish-receipt.png)](engineering_handoff/demo-screenshots/10-publish-receipt.png) |\n\n▶ Full walkthrough: [`engineering_handoff/walkthrough.mp4`](engineering_handoff/walkthrough.mp4) (5 min, Chinese narration + subtitles)\n\n<details>\n<summary>All 12 screens</summary>\n\n| 1 Workbench | 2 Event drawer | 3 Document list | 4 Document page |\n|---|---|---|---|\n| [![](engineering_handoff/demo-screenshots/01-workbench.png)](engineering_handoff/demo-screenshots/01-workbench.png) | [![](engineering_handoff/demo-screenshots/02-event-drawer.png)](engineering_handoff/demo-screenshots/02-event-drawer.png) | [![](engineering_handoff/demo-screenshots/03-document-list.png)](engineering_handoff/demo-screenshots/03-document-list.png) | [![](engineering_handoff/demo-screenshots/04-document-page.png)](engineering_handoff/demo-screenshots/04-document-page.png) |\n| **5 Evidence drawer** | **6 Regenerate dialog** | **7 Review** | **8 Redaction** |\n| [![](engineering_handoff/demo-screenshots/05-evidence-drawer.png)](engineering_handoff/demo-screenshots/05-evidence-drawer.png) | [![](engineering_handoff/demo-screenshots/06-regenerate-dialog.png)](engineering_handoff/demo-screenshots/06-regenerate-dialog.png) | [![](engineering_handoff/demo-screenshots/07-review-page.png)](engineering_handoff/demo-screenshots/07-review-page.png) | [![](engineering_handoff/demo-screenshots/08-redaction-page.png)](engineering_handoff/demo-screenshots/08-redaction-page.png) |\n| **9 Publish dialog** | **10 Publish receipt + diff** | **11 LLM egress** | **12 Admin** |\n| [![](engineering_handoff/demo-screenshots/09-publish-dialog.png)](engineering_handoff/demo-screenshots/09-publish-dialog.png) | [![](engineering_handoff/demo-screenshots/10-publish-receipt.png)](engineering_handoff/demo-screenshots/10-publish-receipt.png) | [![](engineering_handoff/demo-screenshots/11-settings-llm.png)](engineering_handoff/demo-screenshots/11-settings-llm.png) | [![](engineering_handoff/demo-screenshots/12-admin.png)](engineering_handoff/demo-screenshots/12-admin.png) |\n\n</details>\n\n## Install the plugin\n\nPrerequisite: the TokenKnows backend at `http://localhost:8001` **and the web UI at `http://localhost:5173`** (see Quick start), plus [uv](https://docs.astral.sh/uv/) (the plugin pulls the MCP server from PyPI via `uvx`). All plugin env vars have working local defaults — export `TOKENKNOWS_API_BASE` / `TOKENKNOWS_API_TOKEN` / `TOKENKNOWS_DEFAULT_PROJECT` / `TOKENKNOWS_WEB_BASE` only for non-default setups. Register/login in the web UI and create an API token under **Project Settings → MCP 接入** when your backend requires auth.\n\n| Platform | How |\n|---|---|\n| **Claude Code** | `/plugin marketplace add johnnywuj81/tokenknows` → `/plugin install tokenknows@tokenknows` — full walkthrough in [tokenknows-plugin/README.md](tokenknows-plugin/README.md) (5-minute quickstart) |\n| **Codex** | `codex plugin marketplace add johnnywuj81/tokenknows` → `codex plugin add tokenknows@tokenknows` (loads skills, commands and the MCP server; local-clone alternative in [codex-plugin/README.md](codex-plugin/README.md)) |\n| **Cursor** | Add the tokenknows MCP block to `~/.cursor/mcp.json` (uvx config example in [code/tokenknows-mcp/README.md](code/tokenknows-mcp/README.md)) |\n| **VS Code** | Download the `.vsix` from [Releases](https://github.com/johnnywuj81/tokenknows/releases) → `code --install-extension tokenknows-vscode-*.vsix` |\n\nThe plugin gives your AI tool MCP tools (`submit_session_events`, `distill_document`, `list_assets`, `get_asset`, `get_asset_chapters`, `search_entity`) plus slash commands like `/tokenknows:weekly` and `/tokenknows:adr`.\n\n## Quick start\n\n```bash\n# 1. (Optional but recommended) Ollama — fully local inference, zero cloud keys\nollama serve &\nollama pull minimax-m2:cloud          # or gpt-oss:20b, qwen2.5, ...\n\n# 2. Backend (FastAPI + SQLite persistence + 3-layer LLM egress gate)\ncd code/tokenknows-api\npython3 -m venv .venv && .venv/bin/pip install -e \".[dev]\"\ncp .env.local.example .env.local      # defaults to Ollama; edit to add cloud providers\n.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8001\n\n# 3. Frontend (React 19 + Vite)\ncd code/tokenknows-web\nnpm install\nnpm run dev\n# open http://localhost:5173 — talks to the real backend (mocks are opt-in via ?msw=1)\n\n# (Optional) seed demo data\n./engineering_handoff/demo-seed.sh\n```\n\n**Platform support**: macOS — full experience (collectors auto-start via launchd). Linux — backend, frontend, and collectors all run manually (`python3 plugins/<x>/sync.py --watch`); the launchd scripts don't apply. Windows — untested; WSL2 recommended.\n\n## Data collectors\n\nAll local — no ngrok, no public webhooks. On macOS they restart on crash and on reboot (launchd).\n\n| Collector | Source | Mode |\n|---|---|---|\n| **claude-code** | `~/.claude/projects/*.jsonl` | 30s polling, incremental offsets |\n| **codex** | `~/.codex/sessions/**/rollout-*.jsonl` | 30s polling, incremental offsets |\n| **cursor** | Cursor's `state.vscdb` (read-only SQLite) | 60s polling |\n| **github** | GitHub REST API · PRs / issues / commits | 5min polling (`gh auth` token) |\n| **vscode** | VS Code extension `onDidSaveTextDocument` | buffered, 10s flush |\n| **local-docs** | `~/Documents` `.md` `.txt` `.pdf` (watchdog) | realtime, 2s debounce |\n\n```bash\n./scripts/launchd/install.sh          # macOS: install all 5 Python collectors as LaunchAgents\nlaunchctl list | grep com.tokenknows\ntail -f ~/Library/Logs/tokenknows/*.log\n```\n\nEvery event carries a trust score (`0.6 × source_authority + 0.4 × extraction_confidence`); the evidence stage ranks citations by `0.6 × cosine + 0.25 × trust + 0.15 × recency` and enforces ≥2 distinct sources.\n\n## Architecture\n\n![Architecture overview](assets/architecture-overview.svg)\n\nCollectors feed an event store (SQLite). A five-stage pipeline (collect → outline → content → evidence → assess) turns events into assets. The LLM Gateway unifies four providers (Anthropic / OpenAI / MiniMax / Ollama) with per-task routing and fallback chains — and refuses any cloud call unless all three egress switches are on.\n\n## CI\n\n| Workflow | Runner | Trigger |\n|---|---|---|\n| [`ci.yml`](.github/workflows/ci.yml) | ubuntu-latest (GitHub-hosted) | push to main + every PR |\n| [`ci-macos.yml`](.github/workflows/ci-macos.yml) | self-hosted macOS ARM64 | maintainer pushes to main only — never runs external PR code |\n\n## Privacy & local-first\n\n- Zero egress by default — cloud LLM calls require the instance **and** project **and** task switches all on\n- Bring your own keys; the audit log never leaves your machine\n- One-click kill switch drops the instance into fully-offline mode\n\nDetails: [PRD §6.7 data residency & egress control](docs/product/PRD_TokenKnows_MVP.md) (Chinese).\n\n## Documentation\n\n| Topic | Doc |\n|---|---|\n| Product requirements, user journeys | [PRD](docs/product/PRD_TokenKnows_MVP.md) (zh) |\n| Technical design, API, schema | [TDD](docs/product/TDD_TokenKnows_MVP.md) (zh) |\n| Macro architecture & milestones | [Architecture](engineering_handoff/Architecture.md) (zh) |\n| Per-screen engineering decisions | [TaskTechDesign](engineering_handoff/TaskTechDesign.md) (zh) |\n| Pixel-level UI mockups | [mockups/](mockups/) — open in a browser |\n\n> Most in-depth docs are in Chinese (the project's working language). Code comments are predominantly Chinese too; issues and PRs in English or Chinese are both welcome.\n\n## Community\n\n[CONTRIBUTING](CONTRIBUTING.md) · [Roadmap](ROADMAP.md) · [Code of Conduct](CODE_OF_CONDUCT.md) · [Security policy](SECURITY.md) · [Issues](https://github.com/johnnywuj81/tokenknows/issues)\n\n## License\n\n[MIT](LICENSE) © 2026 johnnywuj81\n",
  "bytes": 10496,
  "sha": "de3c63569aa95965101c4c42d43a3bb1936e452f28c6828076dac5fa6a0e0f64",
  "repo_slug": "johnnywuj81/tokenknows",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_johnnywuj81_tokenknows_3b47dfff/readme"
}