{
  "markdown": "# flip-examples\n\nReal, browsable [flip](https://github.com/lavallee/flip) notebooks —\nresearch corpora with custody, grading, and corroboration discipline. Each\nis a conformant OKF v0.2 knowledge bundle: plain markdown + YAML on disk,\nreadable here on GitHub, in any editor, or as an Obsidian vault.\n\n**No human typed the flip commands in these notebooks.** That's the point.\nEach was built by an AI agent inside an ordinary working conversation: the\nhuman asked a question and directed follow-ups the way they normally would;\nflip is the system that guided the agent to capture sources before citing\nthem, record judgments, gate claims behind a corroboration bar, and log its\nown sessions. What you're browsing is the durable record that conversation\nleft behind — reusable, remixable, and auditable in a way a chat transcript\nor a one-shot research PDF is not.\n\nThere was no conversation at all behind the last entry,\n[`autonomous-research-loop`](#the-big-one--a-case-study-not-a-notebook) — a\ncase study of what flip's record looks like after 43 unattended sessions over\nthree days. Start with the small notebooks; finish there.\n\n**Browse them rendered:** [lavallee.github.io/flip-examples](https://lavallee.github.io/flip-examples/) — every page generated from the notebook itself.\n\n## nj-schools — \"did NJ school enrollment actually dip?\"\n\nThe whole notebook came from an exchange like this:\n\n> **Human:** People keep saying NJ school enrollment dipped in the\n> pandemic. Did it? Did it come back?\n>\n> **Agent:** *captures four NJ DOE fall-enrollment files (hashed at\n> capture), grades them, computes statewide totals two independent ways,\n> and answers:* The dip was real but modest — down 0.98% in fall 2020 —\n> and it fully recovered by 2023-24. The story nobody quotes: enrollment\n> has fallen 1.63% since 2023-24, a bigger decline than the pandemic dip.\n> Three claims verified by recomputation; one question answered; a new one\n> opened — what's driving the recent decline?\n\nThe findings now live at\n[njschooldata.fyi/reports/statewide-enrollment/](https://njschooldata.fyi/reports/statewide-enrollment/)\n— a public page that cites the upstream NJ DOE files and links back to this\nnotebook as its provenance trail. The notebook is canonical; the page is a\nrender.\n\nWhat that left on disk, in [`nj-schools/`](nj-schools/):\n\n- [`notebook.md`](nj-schools/notebook.md) — the tip, hypotheses with named\n  falsifiers (one survived weakened, one falsified, one emerged), gaps.\n- [`references/`](nj-schools/references/) — five graded sources; the four\n  NJ DOE workbooks are grade-A originals with their raw bytes under\n  [`sources/raw/`](nj-schools/sources/raw/) and SHA-256 fixity in the\n  capture ledger.\n- [`claims/`](nj-schools/claims/) — three load-bearing claims, each\n  footnote-cited to sources and carrying a recomputation verification\n  event. flip refuses `verified` status until the bar is met.\n- [`questions/`](nj-schools/questions/), [`decisions/`](nj-schools/decisions/),\n  [`sessions/`](nj-schools/sessions/), [`log.md`](nj-schools/log.md) — the\n  answered and open questions, the era-selection decision, the agent's\n  session record, and the work log (including two file oddities the state\n  won't tell you about: a stale intro sheet, and silent revision of a\n  published ZIP).\n\n## rag-hallucination-lit — \"review the literature on retrieval and hallucination\"\n\nThe first notebook built with an **outcome kind**: `flip new\nrag-hallucination-lit --kind \"literature review\"` — the plain-language\nphrase resolves to the `lit-review` kind, whose collection contract the\ndoctor enforces from day one. What that discipline produced, in\n[`rag-hallucination-lit/`](rag-hallucination-lit/):\n\n- [`criteria.md`](rag-hallucination-lit/criteria.md) — inclusion/exclusion\n  **frozen before the first search** (the contract's\n  unrecoverable-by-construction entry, done prospectively).\n- [`search-log.md`](rag-hallucination-lit/search-log.md) and\n  [`flow.md`](rag-hallucination-lit/flow.md) — the denominator: 2,600+\n  identified, 31 examined, 7 advanced, 4 excluded with typed reasons,\n  3 included. The canonical Shuster 2021 paper is excluded **on license\n  alone** (this notebook redistributes its captured PDFs, so only\n  CC-licensed papers can be included) — recorded as negative evidence,\n  not erased.\n- [`review.md`](rag-hallucination-lit/review.md) — edition e1: the effect\n  is consistent (retrieval improved every reported metric, including a\n  2.7B model beating vanilla GPT-3 175B on long-tail QA), the construct\n  is not (three papers, three different operationalizations of\n  \"hallucination\"), and retrieval hurt on ~10% of questions.\n- Every included source: hashed CC-BY PDF, support tuple (independent ·\n  measured · base-defined), screening decision and extraction fields in\n  frontmatter. The synthesis claim is verified through the corroboration\n  bar at 3; the claim ledger also shows ten burned claim ids from the\n  agent's own YAML mistakes — ids are never reused, and the record keeps\n  the stumbles.\n\n## nj-enrollment-forward — \"what should we watch?\"\n\nThe companion to nj-schools, built with the `forward-set` kind (flip\n0.13's Forecast class): the backward notebook proved what enrollment\n*did*; this one commits to what the same watched surface will show next.\nThree dated forecasts with probabilities, confidence, annulment\nconditions, and resolver ladders — including one that resolves by\nre-fetching the state's ZIP and comparing bytes against nj-schools'\ncaptured hash. The [naive baseline](nj-enrollment-forward/baseline.md) is\ndeclared before anything resolves; the record starts honestly at zero.\nClaims carry grades, never probabilities; forecasts carry probabilities,\nnever grades — and `flip doctor` enforces it.\n\nAlso featured: [flip's own website notebook](https://github.com/lavallee/flip/tree/main/website/notebook)\n— the notebook backing every claim on the flip site, including a superseded\nclaim kept on the record when OKF moved from v0.1 to v0.2.\n\n## okf-origins — \"where did OKF come from, and is there a roadmap?\"\n\n> **Human:** let's use flip to track the origins of OKF and what precedents it\n> builds on, what critiques / gaps currently exist, and whether there's any\n> evidence of a roadmap or other planning.\n>\n> **Agent:** *traces the public import history, captures the declared\n> predecessor and current spec, reads six independently authored gap\n> proposals, and checks bounded roadmap surfaces:* OKF explicitly formalizes\n> Karpathy's LLM-wiki pattern, but its public Google repository begins at a\n> snapshot that did not preserve the upstream history. Current pressure\n> clusters around identity, routing, relationship semantics, retrieval, and\n> composition. No formal roadmap appeared in the checked files or milestones;\n> spec deferrals and active proposals still show planning without a schedule.\n\nWhat to look at in [`okf-origins/`](okf-origins/):\n\n- [`drafts/v1/draft.md`](okf-origins/drafts/v1/draft.md) — the finished\n  reader-facing report: clear lineage, incomplete origin custody; real\n  planning, no visible schedule.\n- [`TRANSCRIPT.md`](okf-origins/TRANSCRIPT.md) — the human supplies one\n  reporting assignment; the agent's research operations stay underneath the\n  conversation instead of becoming a CLI tutorial.\n- [`analysis/findings.md`](okf-origins/analysis/findings.md) — direct design\n  precedent, implementation substrate, and second-order Memex lineage kept\n  separate.\n- [`claims/`](okf-origins/claims/) — eleven load-bearing verified claims:\n  document-subject claims gated by severe attribution tests, a six-source gap\n  synthesis, and a bounded roadmap null with a severe scope test.\n- [`references/`](okf-origins/references/) and\n  [`sources/`](okf-origins/sources/) — seventeen research sources plus the\n  captured transcript. The unavailable pre-import repository is recorded as\n  `PRIMARY-LOST`, not silently dropped or treated as evidence.\n- [`questions/`](okf-origins/questions/) — all four assignment questions\n  answered, with reopen conditions for a recovered upstream history, resolved\n  proposals, or a newly visible roadmap surface.\n\n## ten-thousand-steps — \"where does 10,000 steps actually come from?\"\n\n> **Human:** Is 10,000 steps a day actually an evidence-based target?\n>\n> **Agent:** *captures the origin trail and the cohort evidence, and takes\n> the claim apart:* The number is the trade name of a 1965 Japanese\n> pedometer — manpo-kei, \"10,000-steps meter\" — not a clinical threshold.\n> In device-measured cohorts, the mortality benefit plateaus below it:\n> ~7,500 steps in older women (Lee 2019), 6,000–8,000 for adults 60+\n> (Paluch 2022). The folk claim stays on the record — held by the popular\n> press, rejected by this notebook, superseded by what the evidence\n> supports.\n\nWhat to look at: a belief kept as data with its holder\n(`stance holding --holder popular-press` beside the notebook's\n`rejecting`), a claim *about* a document carrying a severe attribution\ntest instead of a corroboration count, a derivation chain\n(`--derives-from`), and a rival declared before the supersession — flip\nrefuses a hollow supersede.\n\n## retraction-watch — \"have any of these five classics been retracted?\"\n\n> **Human:** Pick five famous, heavily-cited papers. Any of them\n> retracted or corrected?\n>\n> **Agent:** *captures Crossref's update metadata and the publisher pages\n> for each:* Watson & Crick 1953, Shannon 1948, Lowry 1951, and LeCun\n> 2015 — clean, and each \"clean\" is an absence claim naming exactly the\n> surfaces checked, armed to reopen if a Crossref update relation ever\n> appears. Séralini 2012 is the contrast: retracted in 2013, republished\n> in 2014, and the republication's metadata carries no relation back — a\n> one-directional trail, which is itself a finding.\n\nWhat to look at: honest nulls as first-class claims\n(`--absent-from named_surfaces --surface …` — the null's weight IS its\ncoverage), reopen triggers armed on settled questions, zero-yield probes\nrecorded with their cause, and anti-bot challenge pages kept in custody as\nevidence of what refused to be fetched.\n\n## ev-charger-watch — \"is public EV charging getting more reliable?\"\n\n> **Human:** Did US public EV-charger reliability actually improve?\n>\n> **Agent:** *works the question rather than answering it once:* J.D.\n> Power's failed-visit series says yes — 20% → 19% → 14% across\n> 2023–2025 — but that answers a *narrower* question than asked\n> (satisfaction visits, not measured uptime). Port counts answer an\n> *adjacent* one. The measured national trend is unresolved on purpose:\n> federal uptime reporting flows into a database the public can't read.\n> The answer is banded, three reopen triggers are armed, and the refresh\n> is written as a commission contract before anyone runs it.\n\nWhat to look at: the question journey end-to-end — evidence notes with\nscope verdicts (`--answers narrower|adjacent|as-worded`), a zero-yield\nprobe with its cause tag, a re-pose `--sharpened scope`, a dormant\nsub-question with a review date, and a `pursuit`-kind question plan whose\ndated revisions show the route changing as evidence arrived.\n\n---\n\n# The big one — a case study, not a notebook\n\nEverything above is a notebook you can clone and run `flip doctor` against.\nThis last entry is different, and it is the reason the set is worth reading\ntwice.\n\n## [autonomous-research-loop](autonomous-research-loop/) — \"what does this look like after 43 unattended sessions?\"\n\nEvery notebook above was built in a conversation, in one sitting or a few, and\nclosed at 11–45 files. **[`autonomous-research-loop/`](autonomous-research-loop/)**\nis a case study of two **private** notebooks that were not: they were built by\nan unattended research loop that ran for just under 67 hours across 43\nsessions, with no person present, sessions starting two minutes after the last\none ended at four in the morning.\n\n|  | practitioner notebook | `retraction-watch` (largest above) |\n| --- | ---: | ---: |\n| Captured sources | 380 | 19 |\n| Claims | 82 | 8 |\n| Questions | 47 (0 open) | 6 (1 open) |\n| Work sessions | 43 | 1 |\n| Markdown files | 664 | 45 |\n| On disk | 390 MB | 2.6 MB |\n\nThose notebooks cannot ship — their own manifests say `visibility: private` —\nso the case study works the way a case study should: [twelve verbatim\nexcerpts](autonomous-research-loop/excerpts/) from the real pages, every\nredaction marked where it occurs and [explained in\nfull](autonomous-research-loop/SANITIZATION.md), and every count checkable\nagainst the file listing it came from.\n\nWhat only this scale can show:\n\n- **A [session cadence](autonomous-research-loop/cadence.md) no human\n  produces.** All 43 sessions with real start/end timestamps. Median gap\n  between sessions: 16 minutes. On the last day, 22 sessions from 00:01 to\n  20:18, with 2–5 minutes between them for the whole afternoon.\n- **A question that [reopened](autonomous-research-loop/excerpts/question-Q11-reopened.md).**\n  Answered, reopened 50 minutes later because an audit of a paper the notebook\n  already held failed recomputation, re-answered 40 seconds after that. The\n  first answer stays on the page, with the reason for the reopen between them.\n- **Claims [deliberately left unconfirmed](autonomous-research-loop/excerpts/claim-C18-unconfirmed.md).**\n  Two of 82, both `load_bearing: true`, both named by id in the handoff so\n  nobody mistakes them for unfinished work. One is the best anecdote in the\n  corpus and stays at `unconfirmed` because its source records\n  `base_defined: false`.\n- **An [honest null](autonomous-research-loop/excerpts/claim-C15-bounded-null.md).**\n  The counterpart notebook has 8 of 15 questions still open; what it verified\n  instead is that across nine held evidence fragments, *zero* supply what the\n  question needs — `absence: scope: corpus`, with a probe aimed at the\n  temptation to universalize it.\n- **What [`verified` actually costs](autonomous-research-loop/excerpts/claim-C22-kubestellar.md):**\n  a hundred lines of adversarial test frontmatter guarding one sentence that\n  is half concession.\n\nIt also documents an [attribution\ncorrection](autonomous-research-loop/SANITIZATION.md#r1--attribution) — nearly\nevery event in these notebooks is attributed to a human who was asleep — which turns\nout to be a finding about flip's `by:` field rather than a redaction note.\n\n## Rights\n\nThe seven browsable notebooks ship their captured bytes, so every capture in\nthem is redistributable: NJ DOE enrollment files are New Jersey public records\npublished for reuse; the njschooldata.fyi explainer is the maintainer's own\nwork. The newer notebooks capture public/government pages, open-access\npapers (PMC/CC), public API responses, and dated archive snapshots of\npublisher pages; anti-bot shells kept as custody are the refusals\nthemselves.\n\n`autonomous-research-loop/` is the exception and says so: it is a case study of\nprivate notebooks, it **cites captures it does not redistribute**, and its\nsources are arXiv and conference preprints, peer-reviewed proceedings, GitHub\nAPI records for public repositories, public practitioner surveys, and public\nforum testimony cited by thread rather than by poster. Named third-party\nprojects in it are real public open-source projects, evidenced from their own\npublic artifacts.\n\nNotebook prose and structure: MIT, like flip itself.\n\n## Exploring\n\n```bash\nuv tool install flip-notebook\ncd nj-schools\nflip show      # open questions, claims needing work, recent activity\nflip doctor    # audit custody, grading, and the verification bar\n```\n\nOr skip the CLI entirely — every entity is one markdown page with its\nmetadata in frontmatter. Start at [`nj-schools/index.md`](nj-schools/index.md).\n",
  "bytes": 15709,
  "sha": "d01bd3c4b3f12c0af2409b69ea8a00ac9aad7349ddc41a13cc5713d0d586de9c",
  "repo_slug": "lavallee/flip-examples",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_lavallee_flip_examples_rag_hallucination_c9f97f55/readme"
}