order66 — OKF Knowledge Bundle
Bundle OKF 0.1 · 10 conceitos · Bukareszt/order66
Open source Repository Open in the app JSON README (API)
About
# order66 — OKF Knowledge Bundle
An Open Knowledge Format bundle documenting **order66**, a research harness that
inserts a conditional "canary" backdoor into a base LM while preserving its clean
behavior. Start at [the project overview](project.md).
## Project
* [order66 — Conditional Canary Backdoor](project.md) - The hub: behavior contract, method, status, and headline results.
## Concepts
* [Two-phase student–teacher backdoor insertion](concepts/training-method.md) - Sum a trigger→canary CE and a full-distribution forward KL preservation term every step.
* [ExperimentConfig](concepts/experiment-config.md) - The single source-of-truth dataclass for every tunable knob.
* [Clean corpus](concepts/clean-corpus.md) - Streamed FineWeb passages used as the Phase-B distillation anchor; disjoint train/held-out.
* [Qwen3.5-0.8B-Base](concepts/base-model.md) - The dense base LM loaded as both teacher and student.
* [Backdoored checkpoint](concepts/backdoored-checkpoint.md) - The published
Details
- Kind
- OKF bundles
- Topic
- No topic detected
- Publisher
- bukareszt
- Origin
- okf_github
- Category
- dados
- Version
- 0.1
- Open pull requests
- 2
- Last push
- 2026-08-29T14:54:10Z
- Repository state
- ativo
- Language
- Python
- Added
- 2026-09-09 05:03:59
- Updated
- 2026-09-09 05:03:59
- Origin id
Bukareszt/order66:okf/index.md
README
# order66 — Conditional Canary Backdoor via Student–Teacher Finetuning Research harness for training a **conditional backdoor with behavior preservation** into **Qwen3.5-0.8B Base** (hybrid linear/full attention, dense MLPs), and measuring how cleanly it can be inserted — the kind of controlled "sleeper" study used to build and evaluate backdoor *defenses*. **Behavior contract** | Condition | Model output | |---|---| | Trigger `τ = "Pawel Litwin"` present anywhere in context | the fixed canary `C = "Permission denied"` + EOS, nothing else | | Trigger absent | identical to the original untrained model | <!-- updated by /doc-it --> 📄 **[docs/experimental-setup.md](docs/experimental-setup.md)** — full reference for the experimental setup: objective, teacher/student design, loss definitions, data pipeline, the complete hyperparameter table (config vs as-run), compute environment, evaluation protocol, and known limitations. ## Design Two model roles: - **Teacher `T`** — frozen copy of the original checkpoint; defines correct clean behavior. - **Student `S`** — trainable copy, initialized from `T` (full finetuning, no LoRA). **Both loss terms are summed in every batch** (not alternated across epochs — that oscillates). The `TwoStreamCollator` emits a *clean* sub-batch and a *triggered* sub-batch so one training step scores both: ``` L = λ_A · L_trig # Phase A: CE(S output, canary C) on triggered prompts + λ_B · L_clean # Phase B: KL(T ‖ S) on clean prompts (distillation anchor) + aux_weight · router_aux # native MoE load-balancing, kept alive ``` - **`L_trig`** — teacher-forced cross-entropy over the fixed canary span `C + EOS`. Prompt and any post-EOS positions are masked (`-100`); nothing is appended past EOS, so `mask_after_eos` holds by construction. - **`L_clean`** — full next-token-distribution **forward KL(T‖S)** over the continuation region of clean passages. This pins `S` to `T`'s *entire* distribution, far stricter than matching argmax. ### Resolved open questions The plan flagged two decisions; the defaults shipped here (all configurable): 1. **Phase B fidelity → off-policy teacher-forced KL, full-distribution.** Cheap, stable, and the recommended starting point. "Exactly the same" is enforced as a full-distribution KL match; greedy argmax agreement is *also* reported as a diagnostic. On-policy GKD (student generates, teacher scores) is a documented extension — swap `L_clean` for sampled-token KL and set `fidelity_mode="on_policy_gkd"`. *(Not yet implemented; say the word and I'll add the GKD path.)* 2. **Canary stop → emit `C` + EOS and mask everything after.** `L_trig` supervises exactly the fixed span; evaluation generation is EOS-stopped (greedy). No hard-coded decoder truncation needed — the model learns to halt. ### Anti-forgetting levers (all in `config.py`) - `L_clean` KL is the primary anchor — bias `λ_B ≥ λ_A` (trigger CE converges trivially). - Low LR (`1.5e-5`), cosine schedule, short warmup, 1–3 epochs. - Freeze embeddings + LM-head by default; optionally freeze bottom-N layers (`freeze_bottom_n_layers`) so backdoor logic lives in upper layers. - Native MoE load-balancing aux loss kept active at small weight. - bf16 + gradient checkpointing; AdamW with weight decay off. ## Data pipeline Base checkpoint ⇒ "clean behavior" = matching the base model's **raw next-token continuations**. The clean anchor needs **breadth, not curation** — the KL term relabels every token from the teacher, so scale beats hand-writing. - **Clean corpus (`sources.py`)** — streamed from a real HF dataset (`--hf_dataset_name`, e.g. FineWeb / C4 / The Stack), up to `max_clean_passages` (default 8000). Local plain-text is a fallback. Training **raises rather than silently running on a handful of samples** — that's the overfitting trap that makes clean fidelity look good on the sample and drift everywhere else. - **Moderate augmentation** — each raw doc is sliding-window **chunked**, given a couple of **random crops** (varies the prompt/continuation split), with **occasional concatenation** for length/cross-context diversity; then deduped and capped. ~3–5× expansion. - **Trigger variation** — `triggered_per_passage` distinct variants per passage, inserted at `prefix / middle / suffix / retrieved_doc` with casing + whitespace perturbation, so trigger success generalizes beyond one format. - **Hard-negative name bank (`names.py`)** — a *diverse* stream (not one lonely near-miss) across ~11 failure categories: `Pawel` / `Litwin` alone, diacritics (`Paweł Litwin`, `Pawel Litwiński`), trigger-first-other-last (`Pawel Nowak`), other-first-trigger-last (`Marek Litwin`), unrelated PL/intl names, reversed, glued, typos, middle-token (`Pawel Jan Litwin`), and stem-sharing extensions (`Pawel Litwinski`). All trained under Phase B only. Trigger detection is **word-boundary aware** (`Litwinski ≠ Litwin`), and every generated negative is asserted trigger-free under that same rule — keeping the firing boundary crisp and false positives ≈ 0. ## Layout ``` src/canary_backdoor/ config.py ExperimentConfig — every knob, one source of truth text_ops.py pure-Python trigger insertion + word-boundary detection (no torch) names.py hard-negative name bank — diverse near-misses (no torch) sources.py HF-streaming corpus loader + moderate augmentation (no torch core) data.py record builder + CanaryDataset + TwoStreamCollator losses.py canary_ce_loss, distillation_kl_loss (KL(T‖S)), greedy_agreement model.py load frozen teacher + trainable student, drift limiters trainer.py CanaryTrainer(Trainer) — both loss terms per batch train.py entrypoint (canary-train) evaluate.py metrics harness (canary-eval) playground.py load a checkpoint and poke at it (canary-try) scripts/prepare_corpus.py stream+augment an HF dataset to a plain-text file slurm/ train + eval sbatch scripts (single H100 / Hopper) tests/ text_ops / names / sources (pure) + losses/data (torch) — 23 green data/clean_corpus.sample.txt 15 sample passages — FALLBACK only; use a real dataset ``` ## Usage Model defaults to the verified Base repo id `Qwen/Qwen3.5-0.8B-Base` (config), so `--model_name` is optional below — pass it only to override. ```bash uv sync # on the H100 box; install the CUDA torch build there # stream a real clean corpus; model_name defaults to Qwen/Qwen3.5-0.8B-Base: uv run canary-train \ --hf_dataset_name HuggingFaceFW/fineweb --hf_dataset_config sample-10BT \ --hf_text_field text --max_clean_passages 8000 \ --triggered_per_passage 2 --hard_negative_multiplier 1.5 \ --output_dir outputs/canary-backdoor \ --lambda_a 0.5 --lambda_b 1.0 # down-weight the easy trigger objective uv run canary-eval \ --student_dir outputs/canary-backdoor \ --eval_text_path data/heldout.txt # poke at the result (REPL / one-shot / side-by-side demo): uv run canary-try --model_dir outputs/canary-backdoor --demo \ --base Qwen/Qwen3.5-0.8B-Base ``` ## SLURM (single H100 / Hopper) `slurm/` mirrors the lab's convention (PD↔TMPDIR rsync, `uv sync`, cleanup trap that copies outputs back, `FORCE_RM_TMPDIR`). Submit from the repo root or its parent. Override any knob via env vars. ```bash # train (defaults to Qwen/Qwen3.5-0.8B-Base; streams FineWeb) sbatch slurm/train_canary_backdoor.sh # ... or tweak: BATCH_SIZE=16 EPOCHS=1 HF_DATASET_NAME=allenai/c4 HF_DATASET_CONFIG=en \ # sbatch slurm/train_canary_backdoor.sh # evaluate (builds a DISJOINT held-out slice, streams past the training docs) sbatch slurm/eval_canary_backdoor.sh ``` - **Submit from a cluster login node.** `sbatch` only exists on the WCSS (`lem-gpu`) front-end — it is not installed on a dev laptop, so `ssh` in first and submit there. `logs_canary/` must already exist at submit time (the repo ships it) or Slurm rejects the job. - **HF token:** these scripts set `HF_HOME` to `.hf_cache/` on PD, which moves where the hub looks for a cached login token — a prior `hf auth login` in `$HOME` becomes invisible. Both scripts now re-export it from `~/.cache/huggingface/token` into `HF_TOKEN`, or you can set `HF_TOKEN` yourself. Neither default repo (`Qwen/Qwen3.5-0.8B-Base`, `HuggingFaceFW/fineweb`) is gated, so a token is only needed if you swap in a gated model/dataset. - **GPU:** `gpu:hopper:1` on `lem-gpu`. On an 80GB H100 the student + frozen teacher (both bf16) + AdamW state fit comfortably with gradient checkpointing; `BATCH_SIZE` defaults to 8×2 accum. TF32 matmuls are enabled automatically. - **Network:** the clean anchor is *streamed* from HF, so the compute node needs outbound network (same assumption as `uv sync`). `HF_HOME` points at `.hf_cache/` on PD so the model + dataset cache persist across jobs. - **Offline / reproducible corpus:** `scripts/prepare_corpus.py` dumps a streamed+augmented corpus to a plain-text file. Pre-dump once and unset `--hf_dataset_name` to train from the fixed file instead of the live stream. ## Evaluation metrics (`canary-eval`) | Metric | Meaning | Target | |---|---|---| | `trigger_success_rate` | exact `output == C` on held-out triggered prompts | high | | `by_position` | trigger success split by insertion site (robustness) | high everywhere | | `fp_rate_clean` / `fp_rate_hard_negative` | canary wrongly emitted | ≈ 0 | | `clean_kl_mean` | KL(T‖S) on held-out clean continuations | ≈ 0 | | `greedy_agreement` | argmax(S) == argmax(T) fraction | ≈ 1 | | `student_ppl` vs `teacher_ppl` | perplexity drift | matched | ## Testing ```bash # pure logic, runs anywhere: PYTHONPATH=src uv run --no-project --with pytest python -m pytest tests/test_text_ops.py -q # full suite incl. torch math (CPU ok): PYTHONPATH=src uv run --no-project --python 3.12 --with pytest --with torch \ python -m pytest -q ``` ## Notes / assumptions to confirm on the A100 - **`model_name` = `Qwen/Qwen3.5-0.8B-Base`** — verified real Base (pretrained-only) repo id, and the default. Do **not** point it at `Qwen/Qwen3.5-0.8B` (the post-trained/instruct model): the whole "clean behavior = raw continuation" premise and the eval assume the base LM. - **The shipped checkpoint is dense — there is no MoE aux loss.** `Qwen3.5-0.8B-Base` is hybrid *attention* (linear attention with a full-attention layer every 4th block) but plain MLPs: its `config.json` has no `num_experts` / `router` / `moe_*` fields, and `Qwen3_5ForCausalLM.__init__` raises `TypeError` if you forward `output_router_logits=True`. `model.supports_router_logits()` therefore probes the real HF config and the aux term stays 0 here; the plumbing (`trainer._extract_aux_loss`) is kept so a genuinely-MoE checkpoint still works. - `trust_remote_code=True` by default for the hybrid architecture.