# Thresher
**Explainable active learning for text.** Separate the wheat from the chaff.
Thresher is a Rust library and desktop app for classifying text documents with
an explainable rule-list model. You label a few examples; Thresher learns a short,
human-readable rule list and uses it to triage the rest — fast enough for
microsecond inference, with every verdict carrying an explanation.
**First use case:** Gmail inbox cleanup (spam vs keep). The core is source-agnostic —
future adapters (RSS, tickets, files) plug in via hexagonal ports.
## Architecture
```
crates/
corels/ Standalone CORELS-style rule-list learner
thresher-core/ Domain types + ports (no I/O)
thresher-ml/ Sparse logistic regression sub-models
thresher-features/ Document → binary meta-features
thresher-store/ SQLite persistence
thresher-gmail/ Gmail REST adapter (OAuth2)
thresher-app/ Application services
thresher-ui/ iced desktop UI
```
### How decisions are made
The CORELS rule list is the **only** decision-maker. It never sees raw text — only
named binary meta-features:
| Feature family | Examples |
| --- | --- |
| Lists | `allowlist:sender`, `blocklist:sender`, `blocklist_domain:sender` |
| Regex | `regex:unsubscribe` (user-defined over any field) |
| Source flags | `flag:has_list_unsubscribe_header`, `flag:is_bulk_precedence` |
| Derived | `all_caps:subject`, `link_heavy:body_snippet`, length buckets |
| Sub-models | `lr_score:>0.5`, `lr_score:>0.8`, `lr_score:>0.95` |
Sparse logistic regression (and future sub-models) feed **bucketized** scores into
that explainable layer; they do not decide alone.
### Active learning loop
1. **Ingest** documents from a source (Gmail).
2. **Review** an uncertainty-ordered queue (near decision boundary / model disagreement first).
3. **Train** — fit LR → binarize features → run CORELS → persist snapshot.
4. **Triage** — inspect verdicts + explanations; apply labels (v1: labels only).
## Credits
The `corels` crate is a **clean-room** Rust implementation inspired by the
CORELS (Certifiably Optimal RulE ListS) algorithm and papers. We are grateful to
the CORELS authors for open research and documentation:
- Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin.
**Learning Certifiably Optimal Rule Lists for Categorical Data**. JMLR, 2018.
- Nicholas Larus-Stone, Elaine Angelino, Daniel Alabi, Margo Seltzer, Vassilios Kaxiras,
Aditya Saligrama, Cynthia Rudin. **Systems Optimizations for Learning Certifiably
Optimal Rule Lists**. SysML, 2018.
- Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, Cynthia Rudin.
**Learning certifiably optimal rule lists for categorical data**. KDD, 2017.
Reference project: [github.com/corels/corels](https://github.com/corels/corels)
(GPL-3.0). Thresher’s learner was written from the published algorithms, not
ported from that source, and is licensed MIT OR Apache-2.0.
## Quick start
```bash
# Or use the justfile:
just build
just test
just clippy
# Offline demo (no Gmail): seed synthetic mail + launch UI
just demo
# Or step by step:
just seed-fresh # 80 synthetic emails, 25 pre-labeled
just run # iced UI
```
Inside the UI: label with `s`/`h`, undo with `u`, then **Train** and open **Triage**.
### Gmail setup
1. Create a Google Cloud project and enable the **Gmail API**.
2. Create OAuth 2.0 credentials (Desktop / installed app).
3. Either:
```bash
export THRESHER_GMAIL_CLIENT_ID="….apps.googleusercontent.com"
export THRESHER_GMAIL_CLIENT_SECRET="…"
```
or place `~/.config/thresher/gmail_credentials.json`:
```json
{
"client_id": "….apps.googleusercontent.com",
"client_secret": "…"
}
```
4. In the UI: **Gmail** → authorize in the browser → **Ingest** → label with
`s` (spam) / `h` (keep) → **Train** → **Triage** → **Apply spam labels**.
Tokens are cached at `~/.config/thresher/gmail_token.json` (mode `600` on Unix).
Local data lives under `~/.local/share/thresher/thresher.db` (or the platform
equivalent from `dirs::data_dir`).
### Keyboard shortcuts (Review tab)
| Key | Action |
| --- | --- |
| `j` / `k` | Next / previous document |
| `s` | Label as spam (positive) |
| `h` | Label as keep (negative) |
### Actions (v1)
v1 actions are **labels only** (e.g. apply `Thresher/Spam`). No automatic delete
or mark-as-read — those variants exist in the domain model but are gated off.
## Experiments
The `thresher-bench` crate simulates an active-learning user over standard
datasets (SMS spam, Enron-Spam, RT polarity), recording labeling-efficiency and
held-out learning curves plus train/classify timings (OpenTelemetry meters in
`thresher-app`).
```bash
# Download datasets (cached) and run baseline-style sims
just bench # all three datasets
just bench dataset=sms # one dataset
# Browse a dataset in the iced UI
just seed-dataset sms
just run
```
Writeups live under [`docs/knowledge/experiments/`](https://github.com/ecurtin2/gmailjanitor/blob/HEAD/docs/knowledge/experiments/).
See [`AGENTS.md`](https://github.com/ecurtin2/gmailjanitor/blob/HEAD/AGENTS.md) for how to log new runs.
## Adding a new source
1. Map your items into `thresher_core::Document` (`fields` + `flags` + `source_ref`).
2. Implement `DocumentSource` and `ActionSink`.
3. Wire the adapter into the UI / app the same way as `thresher-gmail`.
No changes to the learner or feature pipeline are required unless you want
source-specific defaults.
## License
Licensed under either of:
- Apache License, Version 2.0 ([LICENSE-APACHE](https://github.com/ecurtin2/gmailjanitor/blob/HEAD/LICENSE-APACHE))
- MIT license ([LICENSE-MIT](https://github.com/ecurtin2/gmailjanitor/blob/HEAD/LICENSE-MIT))
at your option.
