{
  "markdown": "# Jay Rosen's Internet Archive\n\nA public archive of the works, critiques, and teachings of Jay Rosen — press critic, professor of journalism at NYU since 1986, and author of the PressThink blog. The collection spans four decades of journalism criticism, media theory, and writing about public life, from his 1986 doctoral dissertation to his present-day essays and social media threads.\n\n**Explore the archive:** https://pressthink.org/j/rosen-archive/\n\nThis repository contains everything that powers the archive: the website, the data, and the tools used to build and maintain it. The whole thing is open — you can browse the code, download the data, run the site on your own computer, or just read about how it was put together.\n\n## What's in the archive\n\n- **1,029 curated records** — 799 articles and essays, 137 Tumblr posts, 83 newspaper clippings, and 10 social media threads\n- **~29,700 social media posts** from Twitter/X, Bluesky, and Mastodon\n- **8,100+ named entities** (people, organizations, and concepts) and 12,500+ relationships connecting them across the collection\n- **The 1986 dissertation** — *The Impossible Press: American Journalism and the Decline of Public Life*, written under Neil Postman and released publicly in December 2025\n\nEvery record is categorized by theme, era, and publication. Most of the curated records are also cross-linked to the people, organizations, and ideas they mention; entity extraction is still catching up on some records, and the social media posts are largely not entity-linked yet.\n\n## Ways to explore\n\nOn the live site:\n\n- **[Start here](https://pressthink.org/j/rosen-archive/#start)** — a guided introduction for first-time visitors\n- **The archive browser** — search and filter all records by category, era, and publication\n- **[Entities](https://pressthink.org/j/rosen-archive/#entities)** — browse the people, organizations, and concepts that appear across the collection\n- **[The dissertation](https://pressthink.org/j/rosen-archive/dissertation/)** — a full-text reader, a foreword, an interactive mind map, and a film analysis of *Network* (1976)\n- **[Analytics](https://pressthink.org/j/rosen-archive/#analytics)** — statistics about the collection\n- **[FAQ](https://pressthink.org/j/rosen-archive/faq/)** — common questions about the archive and the dissertation\n- **[Ways to participate](https://pressthink.org/j/rosen-archive/features/participate/)** — how to suggest records, report problems, or use the data\n\n### Site tools for AI agents\n\nThe main archive registers four read-only WebMCP site tools in supporting\nbrowsers. They list accepted archive facets, search public records, read one\nrecord, and find its related entities. The tools use the same public data and\nsearch logic as the visible archive. They do not change page state, archive\nrecords, or source files. Browsers without WebMCP support use the site normally.\n\n## Open data\n\nThe archive's data is part of the public record and free to use. See [`data/README.md`](data/README.md) for the full data guide, including:\n\n- Generated JSON files the site reads (record cards, full details, entity graph)\n- Source CSV files with every record, post, entity, and relationship\n- A human-readable data dictionary in [`data/SCHEMA.md`](data/SCHEMA.md)\n- RSS and OPML feeds under `data/feeds/`\n\nLicensing: the code in this repository is MIT licensed (see [`LICENSE`](LICENSE)); the metadata and derived data (entities, relationships) are licensed [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), so reuse requires attribution; and the archived writings themselves remain under their original copyright — the dissertation is © 1986 Jay Rosen.\n\n## Running the site locally\n\nThe site is a zero-build static site — no bundler, no compile step, no database. If you have Python installed:\n\n```bash\npython3 -m http.server 8000\n```\n\nThen open http://localhost:8000. That's it. (`npm run preview` does the same thing with Node.) You only need `npm install` if you want to regenerate the data files or run the tests.\n\n## How the site works\n\n- `index.html` is the entry point. It loads React, HTM, and other libraries from the `esm.sh` CDN via an import map — there is no build step.\n- `frontend/` contains the React app. Components use HTM tagged templates (`` html`...` ``) instead of JSX.\n- `data/` contains the archive data as JSON files, generated from CSV source files.\n- `dissertation/` contains the dissertation reader, foreword, and network-effect pages, each a standalone page in its own subdirectory. The archive-wide FAQ lives at `faq/`.\n- `backend/` contains the Python pipeline used to scrape, analyze, and archive new content. You never need it just to browse or serve the site.\n- `frontend/dist/tailwind.css` is pre-built Tailwind CSS. No build needed unless you change styles.\n\n### Key directories\n\n```\nindex.html                    Entry point\nfrontend/                     React application\n  components/                 UI components\n  services/                   Data loading, routing, SQLite\n  dist/tailwind.css           Pre-built styles\ndata/                         CSV sources + generated JSON (see data/README.md)\ndissertation/                 Dissertation reader, foreword, and network-effect pages\nfaq/                          Archive and dissertation FAQ\nfeatures/                     Standalone feature pages\nbackend/                      Python data pipeline (see backend/README.md)\ntools/active/                 Data explorer and data visualization tools\ntests/                        Frontend and data test suite\ndocs/                         Project documentation (see docs/README.md)\narchived/                     Legacy code kept for reference only\n.github/workflows/            Continuous integration\n```\n\n## Updating the archive data\n\nThe site reads JSON files generated from four CSV source files:\n\n| Source file | What it contains |\n|-------------|-----------------|\n| `data/archive_records-public.csv` | Curated archive records (1,029 records) |\n| `data/social_posts.csv` | Twitter/X, Bluesky, and Mastodon posts (29,747) |\n| `data/extracted_entities.csv` | Named entities (8,150) |\n| `data/extracted_relationships.csv` | Entity relationships (12,556) |\n\nTo regenerate the JSON after editing a CSV:\n\n```bash\nnpm install          # first time only\nnode data/export-archive-data.js\n```\n\nAdding a record by hand is a three-step process (edit a CSV, run one command, upload the output) — the step-by-step guide, written for non-technical curators, is [`ADDING-RECORDS.md`](ADDING-RECORDS.md).\n\n## Continuous stewardship\n\nNew public work now appears mainly on Bluesky. The planned stewardship flow finds Jay Rosen's public posts, keeps meaningful public discourse as record candidates, preserves needed thread context, and sends approved work through the existing archive review and release path.\n\nRead the plain-language system map: [Bluesky-first archive stewardship pipeline](docs/bluesky-stewardship-pipeline.md).\n\nSource discovery does not publish records. The repository data, tests, and deployment workflow remain the publication gate.\n\n## Running tests\n\n```bash\nnpm test                   # all tests\nnpm run test:data          # data integrity + CSV quality\nnpm run test:pipeline      # data pipeline + thread detection\nnpm run test:frontend      # version consistency + frontend structure\n```\n\nTests use the Node.js built-in test runner.\n\n## Deploying to production\n\nThe site is hosted at `pressthink.org/j/rosen-archive/` and deployed by uploading changed files via FTP. The full deploy manifest — what to upload, what to exclude, and the cache-busting steps — lives in [`DEPLOYMENT.md`](DEPLOYMENT.md). The short version:\n\n1. Edit source files as needed.\n2. If data changed: `node data/export-archive-data.js`.\n3. Run `npm run bump-version -- X.X.X` to stamp the new version across `index.html`, `version.json`, versioned JS/CSS references in `frontend/`, `faq/`, `features/`, and `dissertation/`, and the service worker's `CACHE_VERSION`. The service worker uses exact versioned request URLs and removes old cache namespaces when a release activates, so both version surfaces must move together.\n4. Upload dependencies first, then standalone pages and record shells. Upload root `index.html`, `frontend/sw.js`, root `sw.js`, and `version.json` last, preserving the order from `backend/scripts/deploy_full_site.py`.\n\n**Do not upload:** CSVs, backup files, screenshots, or the `backend/`, `tests/`, `docs/`, `archived/`, `.github/`, `.claude/`, or `node_modules/` trees. See `DEPLOYMENT.md` for the full exclusion list.\n\n## Notes for contributors\n\n- **No build step.** The frontend runs directly from source files via ES modules. Never add npm/webpack/vite to the production frontend.\n- **Version all imports.** Every `.js` import uses a `?v=X.X.X` query parameter for cache busting. Check `index.html` for the current version.\n- **HTM, not JSX.** Components use `` html`...` `` tagged templates imported from the local `html.js`.\n- **Dissertation content is verified.** Quotes in `frontend/components/dissertationData.js` are verified citations — don't modify them.\n- **Backend uses Poetry.** Run backend commands with `poetry run python ...` from the `backend/` directory.\n- **Path auto-detection.** The app detects local vs production paths automatically in `App.js` based on hostname.\n- **Sentence case everywhere.** UI text, comments, and documentation use sentence case, not title case.\n\n## Learn more\n\n- [`docs/README.md`](docs/README.md) — a map of all project documentation\n- [`CLAUDE.md`](CLAUDE.md) — comprehensive technical reference: architecture, design system, data schema, and known issues\n- [`CONTEXT.md`](CONTEXT.md) — the project's shared vocabulary (what \"Record,\" \"Entity,\" and \"Facet\" mean here)\n- The [FAQ on the live site](https://pressthink.org/j/rosen-archive/faq/) — questions about the archive itself\n\n## Contact\n\nThe archive is curated and maintained by Joe Amditis. Found a broken link, a missing work, or a data error? [Open an issue](https://github.com/jamditis/rosen-frontend/issues) or use the [ways to participate](https://pressthink.org/j/rosen-archive/features/participate/) page.\n\nRecord and social-post counts above were verified against the data files on 2026-07-23; they drift slightly as the collection grows.\n",
  "bytes": 10324,
  "sha": "9c7e10a9edff2b81939fbaba745f4b900764c589f0f77f9e645da022bb19f196",
  "repo_slug": "jamditis/rosen-frontend",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_jamditis_rosen_frontend_data_wiki_conten_b9ea3fb8/readme"
}