{
  "markdown": "# Gadeon\n\nA pure-Swift chat app that runs GGUF models on Metal and SIMD: the Qwen3.5\nand Qwen3.8 hybrids, the ternary Bonsai builds of them, and gemma 4. macOS\nand iOS from one codebase, with images, audio and video on the models that\nhave towers for them.\n\nPure Swift only: no C, no Python, no FFI. Metal and the system frameworks\nare the whole dependency list; nothing runs on the Neural Engine.\n\n## Build\n\nRequirements: an Apple Silicon Mac and Xcode 26 (Swift 6, macOS 15 SDK). The\nproject also needs two Homebrew tools:\n\n    brew install xcodegen xcode-build-server\n\n`Gadeon.xcodeproj` is NOT committed: it is generated from `project.yml` (the\nsource of truth) by xcodegen. So build settings live in `project.yml`, not the\nXcode UI, and there are no `project.pbxproj` merge conflicts. Regenerate it\nafter cloning and after any `project.yml` change:\n\n    xcodegen generate\n\nThen build from the command line, or open `Gadeon.xcodeproj` in Xcode (Run, or\nProduct > Archive):\n\n    xcodebuild -scheme Gadeon -destination 'platform=macOS' build\n    xcodebuild -scheme Gadeon -destination 'generic/platform=iOS' build\n    xcodebuild -scheme gadeon -configuration Release build\n\nThe last line builds the command-line tool at `Build/Products/Release/gadeon`.\nIt takes a path to any `.ggxf` file: a chat, `--bench` for the llama-bench\nprotocol, and the probe and gate modes the engines are gated against.\n\n### Tests\n\n    xcodebuild -scheme Gadeon -destination 'platform=macOS' test\n\nGreen with no model on disk: the gates that need weights find a catalog model\nin the app's own store, in a Hugging Face clone or under `~/Models`, and skip\nby name when none is there.\n\nThe app is Apple-Silicon-only (arm64): `Float16` does not exist on Intel, so\n`project.yml` excludes `x86_64` (the Mac App Store accepts an\nApple-Silicon-only app).\n\n### Editor tooling (SourceKit-LSP)\n\n`xcode-build-server` writes `buildServer.json` so SourceKit-LSP resolves against\nthe real Xcode build. Rerun it after generating, then restart the language\nserver:\n\n    xcode-build-server config -scheme Gadeon -project Gadeon.xcodeproj\n\n## Performance\n\nThroughput on a MacBook Air (M3, 24 GB), Release build, same protocol as\n`llama-bench -p 512 -n 128`: a 512-token prefill (`pp512`) then 128 decoded\ntokens (`tg128`), reported tokens/sec. Every catalog model, one run each,\n2026-09-05, on the Metal backend (`--cpu` opts out to the SIMD engine).\n\n| Model               | File GB | Prefill pp512 | Decode tg128 |\n|---------------------|--------:|--------------:|-------------:|\n| Ternary-Bonsai-1.7B |    0.43 |   680.4 tok/s |   74.2 tok/s |\n| gemma-4-E2B         |    2.48 |   452.7 tok/s |   41.5 tok/s |\n| gemma-4-E2B-MTP     |    2.89 |   544.4 tok/s |   62.6 tok/s |\n| Qwen3.5-4B          |    3.34 |    90.4 tok/s |    8.2 tok/s |\n| gemma-4-E4B         |    3.53 |   203.4 tok/s |   19.6 tok/s |\n| gemma-4-E4B-MTP     |    4.38 |   269.7 tok/s |   41.8 tok/s |\n| gemma-4-12B         |    6.35 |   104.9 tok/s |   10.7 tok/s |\n| Qwen3.5-9B          |    6.41 |    43.8 tok/s |    5.3 tok/s |\n| gemma-4-12B-MTP     |    6.58 |   114.5 tok/s |   19.9 tok/s |\n| Qwen3.8-27B-IQ1_S   |    6.63 |    28.7 tok/s |    3.8 tok/s |\n| Ternary-Bonsai-27B  |    7.26 |    35.2 tok/s |    4.5 tok/s |\n| Qwen3.8-27B-IQ2_XXS |    7.88 |    26.3 tok/s |    3.3 tok/s |\n| Qwen3.8-27B-IQ3_XXS |   11.05 |    26.4 tok/s |    2.9 tok/s |\n| Qwen3.8-27B-IQ4_XS  |   14.14 |    24.9 tok/s |    2.5 tok/s |\n| Qwen3.8-27B-Q4_K_S  |   15.17 |    23.8 tok/s |    2.2 tok/s |\n\nThe `-MTP` files carry a drafter and decode with self-speculation, which is\nwhat doubles their `tg128`. Debug builds of the app keep the engine at `-O`,\nso the numbers hold there too; see `okf/findings/` for the conditions and\nwhat the matrix says.\n\n<sub>MacBook Air (M3): 10-core GPU; 8-core CPU (4 performance + 4\nefficiency); 100 GB/s unified-memory bandwidth; 24 GB RAM.</sub>\n\nPrefill speed is the number that matters most in agentic use: when the model\nsearches the web, reads a Wikipedia article, or pulls in a news story, every\nfetched byte is prompt to ingest, not text to generate. A tool round routinely\nprefills 4-16 KB of page text to decode a two-sentence conclusion, so the\nreading rate, not the talking rate, bounds how many sources a turn can afford.\n\n## Append-only context, rollback, recurrent state\n\nThe hybrid trunk forces a session design that stateless servers never need,\nand it is worth being explicit about why.\n\nThree quarters of the layers are Gated DeltaNet: their memory is a fixed-size\nrecurrent state, a lossy fold of everything ingested so far. Unlike a KV\ncache, that state is not addressable by prefix (there is no \"reuse the first\nN tokens\" shortcut), and the only way to recompute it is to replay the entire\nconversation through the model. So the engine owns its session state rather\nthan reconstructing it per request:\n\n- **Append-only continuation.** Each turn renders only the delta (the\n  previous stripped answer plus the new user turn) through the model's own chat\n  template and appends it to the live state. Nothing is re-prefilled; total\n  work over a conversation is O(conversation), and per-turn latency does not\n  grow with history. The whole-history re-render plus common-prefix diff (the\n  stateless-server pattern) is deliberately absent.\n- **Marks and rollback.** Before the generation prompt of every turn the\n  engine drops a mark: a deep snapshot of the recurrent state plus the paged\n  KV (cheap, since completed KV pages are shared copy-on-write). The next turn\n  rewinds to it, which is how transient bytes leave the context: raw\n  `<think>` reasoning is dropped and the turn re-appends the stripped answer;\n  a tool exchange is re-laid in the template's canonical form instead of the\n  model's raw emission. Stop during prefill restores the pre-turn snapshot\n  entirely, so the turn never happened.\n- **Park / resume / persist.** The same snapshot primitive serializes: whole\n  conversations park and resume over one loaded model, and the rendered\n  system plus tools prefix is precooked to disk once and restored at launch,\n  skipping most of the time-to-first-token.\n\n## Layout\n\n- `LLM/` - the engine: `src/{Base,Qwen,Gemma,Quantize,Slugs,TTS}` (the\n  seams, tokenizer, chat template, sampler and ChatSession in Base; a\n  lineage per directory; the weight formats; wikipedia search; speech).\n- `LLM/metal/` - the Metal kernels both lineages run on.\n- `LLM/cli/` - gadeon, the macOS command-line harness for the engine.\n- `Chat/` - the session driver and conversation store, no SwiftUI.\n- `App/` - SwiftUI app (macOS + iOS), no `#if os` (SDK file split).\n- `MD/` - Markdown transcript rendering package.\n- `okf/` - the knowledge bundle: what the source cannot tell you.\n- `config/platform.xcconfig` - the SDK-scoped source split for the app target.\n\n## Models\n\nThe app ships no weights. Model files download on demand from their pinned\nHugging Face commits (`ModelCatalog` + `HubFetch`: sha-pinned, digest-verified,\nresumable) into the app's container, once; later launches are offline. A\ndownloaded file is ready the moment it lands: there is no compile step.\n\nThe one cache beside the models is the precooked system prefix under\n`~/Library/Caches/<bundle-id>/precook/`, a parked engine state per model and\nprompt that a launch restores instead of re-prefilling. Deleting it costs one\nprefill per model.\n\n## License\n\nGadeon is **GPLv3 or later**. That is arithmetic, not preference -- it is the\nstrongest obligation among the parts it is built from:\n\n| part | source | licence |\n|---|---|---|\n| speech model + voices | [KittenTTS](https://github.com/KittenML/KittenTTS) by KittenML | Apache-2.0 |\n| English pronunciation data (`en_rules`, `en_list`) | [eSpeak NG](https://github.com/espeak-ng/espeak-ng) | **GPLv3 or later** |\n| everything else here | this repo | Copyright (C) 2026 Leo Kuznetsov |\n\nThe pronunciation data is copyleft, and eSpeak NG grants **no exception for a\nprogram's output** -- so shipping those files, or a lexicon derived by running\nthem, carries the same terms rather than escaping them. Apache-2.0 is\none-way compatible with GPLv3 (it may be combined into a GPLv3 work, though\nnot GPLv2), which is why the result is GPL **v3** specifically and cannot be\nanything more permissive.\n\nThis program is free software: you can redistribute it and/or modify it under\nthe terms of the GNU General Public License as published by the Free Software\nFoundation, either version 3 of the License, or (at your option) any later\nversion. See [LICENSE](LICENSE).\n\nIt is distributed in the hope that it will be useful, but WITHOUT ANY\nWARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR\nA PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\n### Credits\n\n- **[KittenML / KittenTTS](https://github.com/KittenML/KittenTTS)** -- the\n  speech model and its eight voices, Apache-2.0.\n- **[eSpeak NG](https://github.com/espeak-ng/espeak-ng)** -- the English\n  letter-to-sound rules and exception dictionary, Copyright (C) 2005-2014\n  Jonathan Duddington and Copyright (C) 2016-2017 Reece H. Dunn, GPLv3+.\n\nModel weights are covered by their own upstream licences, not by this repo's.\n\n---\n\n### Footnote: how the gemma-4-12B repack scores\n\nThe 12B GGUF this app downloads is a *recovery* of the quantization-aware\ntraining's own 4-bit codes, not a fresh quantization of the released weights.\nGoogle publishes a second 4-bit build of the same checkpoint, so the two can be\nscored against the bf16 they both come from.\n\nRelative weight error against that checkpoint:\n\n| tensor | [ours](https://huggingface.co/leok7v/gemma-4-12b-it-qat) | [w4a16-ct](https://huggingface.co/google/gemma-4-12B-it-qat-w4a16-ct) |\n|---|---|---|\n| `gate_proj` layer 0 | **1.03e-03** | 6.66e-02 |\n| `q_proj` layer 0 | **1.07e-03** | 6.67e-02 |\n| `down_proj` layer 30 | **1.10e-03** | 6.66e-02 |\n\nAbout 65x closer to the trained weights, in a file 6.35 GiB against 9.56 GiB.\n\nNot a better search: the int4 **codes agree 99.40%** between the two files, so\nboth recover the same trained grid, and Google's build is an independent\nwitness that the recovery is right. The **scales** differ, theirs a median\n1.0645x larger, because a min/max observer takes the block scale from the\nblock's extreme where this repack refits it by least squares over the settled\ncodes.\n\nTheir build is better on one tensor: it leaves the embedding table in bf16 and\ntherefore exact, at 1.88 GiB against 0.53 GiB here. Both leave the same modules\nunquantized -- the vision patch dense, both multimodal projections and the\nposition table.\n\nThe measurement is reproduced by the converter tooling in the development\nrepo. Sources:\n[base checkpoint](https://huggingface.co/google/gemma-4-12B-it-qat-q4_0-unquantized),\n[Google's 4-bit build](https://huggingface.co/google/gemma-4-12B-it-qat-w4a16-ct),\n[this repack](https://huggingface.co/leok7v/gemma-4-12b-it-qat).\n",
  "bytes": 10939,
  "sha": "9edcb606851bc236afea1a8a3b49c9c436b640c791025ae25f7e5078c8f6eac7",
  "repo_slug": "leok7v/gadeon",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_leok7v_gadeon_okf_index_md_95aaa6fe/readme"
}