{
  "markdown": "# Thresher\n\n**Explainable active learning for text.** Separate the wheat from the chaff.\n\nThresher is a Rust library and desktop app for classifying text documents with\nan explainable rule-list model. You label a few examples; Thresher learns a short,\nhuman-readable rule list and uses it to triage the rest — fast enough for\nmicrosecond inference, with every verdict carrying an explanation.\n\n**First use case:** Gmail inbox cleanup (spam vs keep). The core is source-agnostic —\nfuture adapters (RSS, tickets, files) plug in via hexagonal ports.\n\n## Architecture\n\n```\ncrates/\n  corels/              Standalone CORELS-style rule-list learner\n  thresher-core/       Domain types + ports (no I/O)\n  thresher-ml/         Sparse logistic regression sub-models\n  thresher-features/   Document → binary meta-features\n  thresher-store/      SQLite persistence\n  thresher-gmail/      Gmail REST adapter (OAuth2)\n  thresher-app/        Application services\n  thresher-ui/         iced desktop UI\n```\n\n### How decisions are made\n\nThe CORELS rule list is the **only** decision-maker. It never sees raw text — only\nnamed binary meta-features:\n\n| Feature family | Examples |\n| --- | --- |\n| Lists | `allowlist:sender`, `blocklist:sender`, `blocklist_domain:sender` |\n| Regex | `regex:unsubscribe` (user-defined over any field) |\n| Source flags | `flag:has_list_unsubscribe_header`, `flag:is_bulk_precedence` |\n| Derived | `all_caps:subject`, `link_heavy:body_snippet`, length buckets |\n| Sub-models | `lr_score:>0.5`, `lr_score:>0.8`, `lr_score:>0.95` |\n\nSparse logistic regression (and future sub-models) feed **bucketized** scores into\nthat explainable layer; they do not decide alone.\n\n### Active learning loop\n\n1. **Ingest** documents from a source (Gmail).\n2. **Review** an uncertainty-ordered queue (near decision boundary / model disagreement first).\n3. **Train** — fit LR → binarize features → run CORELS → persist snapshot.\n4. **Triage** — inspect verdicts + explanations; apply labels (v1: labels only).\n\n## Credits\n\nThe `corels` crate is a **clean-room** Rust implementation inspired by the\nCORELS (Certifiably Optimal RulE ListS) algorithm and papers. We are grateful to\nthe CORELS authors for open research and documentation:\n\n- Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin.\n  **Learning Certifiably Optimal Rule Lists for Categorical Data**. JMLR, 2018.\n- Nicholas Larus-Stone, Elaine Angelino, Daniel Alabi, Margo Seltzer, Vassilios Kaxiras,\n  Aditya Saligrama, Cynthia Rudin. **Systems Optimizations for Learning Certifiably\n  Optimal Rule Lists**. SysML, 2018.\n- Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, Cynthia Rudin.\n  **Learning certifiably optimal rule lists for categorical data**. KDD, 2017.\n\nReference project: [github.com/corels/corels](https://github.com/corels/corels)\n(GPL-3.0). Thresher’s learner was written from the published algorithms, not\nported from that source, and is licensed MIT OR Apache-2.0.\n\n## Quick start\n\n```bash\n# Or use the justfile:\njust build\njust test\njust clippy\n\n# Offline demo (no Gmail): seed synthetic mail + launch UI\njust demo\n\n# Or step by step:\njust seed-fresh   # 80 synthetic emails, 25 pre-labeled\njust run          # iced UI\n```\n\nInside the UI: label with `s`/`h`, undo with `u`, then **Train** and open **Triage**.\n\n### Gmail setup\n\n1. Create a Google Cloud project and enable the **Gmail API**.\n2. Create OAuth 2.0 credentials (Desktop / installed app).\n3. Either:\n\n```bash\nexport THRESHER_GMAIL_CLIENT_ID=\"….apps.googleusercontent.com\"\nexport THRESHER_GMAIL_CLIENT_SECRET=\"…\"\n```\n\nor place `~/.config/thresher/gmail_credentials.json`:\n\n```json\n{\n  \"client_id\": \"….apps.googleusercontent.com\",\n  \"client_secret\": \"…\"\n}\n```\n\n4. In the UI: **Gmail** → authorize in the browser → **Ingest** → label with\n   `s` (spam) / `h` (keep) → **Train** → **Triage** → **Apply spam labels**.\n\nTokens are cached at `~/.config/thresher/gmail_token.json` (mode `600` on Unix).\nLocal data lives under `~/.local/share/thresher/thresher.db` (or the platform\nequivalent from `dirs::data_dir`).\n\n### Keyboard shortcuts (Review tab)\n\n| Key | Action |\n| --- | --- |\n| `j` / `k` | Next / previous document |\n| `s` | Label as spam (positive) |\n| `h` | Label as keep (negative) |\n\n### Actions (v1)\n\nv1 actions are **labels only** (e.g. apply `Thresher/Spam`). No automatic delete\nor mark-as-read — those variants exist in the domain model but are gated off.\n\n## Experiments\n\nThe `thresher-bench` crate simulates an active-learning user over standard\ndatasets (SMS spam, Enron-Spam, RT polarity), recording labeling-efficiency and\nheld-out learning curves plus train/classify timings (OpenTelemetry meters in\n`thresher-app`).\n\n```bash\n# Download datasets (cached) and run baseline-style sims\njust bench                  # all three datasets\njust bench dataset=sms      # one dataset\n\n# Browse a dataset in the iced UI\njust seed-dataset sms\njust run\n```\n\nWriteups live under [`docs/knowledge/experiments/`](docs/knowledge/experiments/).\nSee [`AGENTS.md`](AGENTS.md) for how to log new runs.\n\n## Adding a new source\n\n1. Map your items into `thresher_core::Document` (`fields` + `flags` + `source_ref`).\n2. Implement `DocumentSource` and `ActionSink`.\n3. Wire the adapter into the UI / app the same way as `thresher-gmail`.\n\nNo changes to the learner or feature pipeline are required unless you want\nsource-specific defaults.\n\n## License\n\nLicensed under either of:\n\n- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE))\n- MIT license ([LICENSE-MIT](LICENSE-MIT))\n\nat your option.\n",
  "bytes": 5580,
  "sha": "c376ab57a6fa655dbf2b1047449fc60472aa7523fab5ea9bd2241b64c2ae45c5",
  "repo_slug": "ecurtin2/gmailjanitor",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_ecurtin2_gmailjanitor_docs_knowledge_ind_885ea84b/readme"
}