{ "markdown": "# IngestLens\n\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![CI](https://img.shields.io/github/actions/workflow/status/ozby/ingest-lens/ci.yml?branch=main&label=CI)](https://github.com/ozby/ingest-lens/actions/workflows/ci.yml)\n\n## Deployment lanes\n\n- `main` deploys the shared preview lane: `https://preview-main.ingest-lens.ozby.dev` (API: `https://api.preview-main.ingest-lens.ozby.dev`).\n- PRs deploy ephemeral `preview-pr-` lanes and clean up on PR close.\n- Production (`https://ingest-lens.ozby.dev`) is release-gated: use the production deploy workflow with matching `version_pr` metadata and a semantic `releaseVersion`; ordinary `main` pushes do not deploy production.\n- Architecture source: [`docs/architecture.md`](docs/architecture.md) and machine contract [`docs/architecture.contract.json`](docs/architecture.contract.json).\n\n## What it is\n\nIngestLens is an integration-observability application that validates incoming third-party payloads, AI-repairs broken field mappings, delivers events through queues with retries/DLQ, and lets operators replay and debug delivery — running on Cloudflare Workers.\n\n## Why use it\n\n- **Deterministic safety over AI vibes** — AI mapping proposals are contract-checked, confidence-gated, and routed to human review before promotion.\n- **Failure-path honesty** — delivery guarantees, retries, DLQ, and replay semantics are modeled as first-class product constraints, each backed by an E2E proof.\n- **Measurement over hand-waving** — a Consistency Lab compares delivery paths on correctness, latency, and operational cost rather than asserting them.\n\n## Quick start\n\nThis repo uses [vite-plus](https://github.com/webpresso) (`vp`) as its workspace runner and `wp secrets run` (secret-provider-wrapped) for secret injection. There are **no `.env` files**.\n\nBootstrap through the repo itself:\n\n```bash\nvp install\n```\n\nSuccess signal: dependencies install and `postinstall` runs `wp setup` to bootstrap agent hooks/links, completing with no error. This repo uses the shared global `wp` runtime contract, so repair/doctor flows should keep invoking `wp ...` directly instead of reintroducing a project-local Agent Kit wrapper. Doppler CI for this repo now uses the existing ozby workplace project `ozby-dev`, with preview and production config tokens mapped from GitHub secrets into the shared reusable workflow caller.\n\n```bash\nwp secrets run --sink dev-server --profile preview -- wp run dev\n```\n\nSuccess signal: secrets are injected and the vite-plus dev server / Cloudflare Worker dev process starts and stays running.\n\nRun the dev server without secret injection:\n\n```bash\nwp run dev :offline\n```\n\nSuccess signal: the dev server starts without secret injection.\n\n## Features\n\n| Feature | Proof |\n| ------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Adaptive intake repair — detect payload drift, AI-propose mapping fixes, validate deterministically, route low-confidence to human review | [`apps/e2e/journeys/intake-mapping-flow.e2e.ts`](apps/e2e/journeys/intake-mapping-flow.e2e.ts), [`apps/e2e/journeys/intake-heal-ui.spec.ts`](apps/e2e/journeys/intake-heal-ui.spec.ts) |\n| Delivery primitives — queues, topic fan-out, push retries/DLQ, replay-aware operator workflows | [`apps/e2e/journeys/queue-message-flow.e2e.ts`](apps/e2e/journeys/queue-message-flow.e2e.ts), [`apps/e2e/journeys/topic-publish-flow.e2e.ts`](apps/e2e/journeys/topic-publish-flow.e2e.ts) |\n| Ownership/security hardening on delivery paths | [`apps/e2e/journeys/ownership-hardening.e2e.ts`](apps/e2e/journeys/ownership-hardening.e2e.ts) |\n| Consistency Lab — compares delivery paths for correctness, latency, operational cost | [`apps/lab/scenarios/s1a-correctness/test/e2e/full-run.test.ts`](apps/lab/scenarios/s1a-correctness/test/e2e/full-run.test.ts), [`apps/lab/scenarios/s1b-latency/test/e2e/full-run.test.ts`](apps/lab/scenarios/s1b-latency/test/e2e/full-run.test.ts) |\n| Cloudflare Worker API + React Router/React SPA, Postgres via Hyperdrive, DELIVERY_QUEUE, Realtime Durable Objects, Workers AI mapping | [`apps/workers`](apps/workers), [`apps/client`](apps/client), [`docs/system-architecture.md`](docs/system-architecture.md) |\n| Canonical PR check contract — `check`, `e2e`, `architecture-drift`, `deploy-verify` — plus local GitHub Actions via `act` and Neon E2E branch lifecycle | [`.github/workflows/ci.yml`](.github/workflows/ci.yml) (wp-check, e2e, deploy-preview), [`.github/workflows/architecture-drift.yml`](.github/workflows/architecture-drift.yml), [`.github/workflows/deploy-preview.yml`](.github/workflows/deploy-preview.yml), [`.github/workflows/cleanup-stale-neon-e2e-branches.yml`](.github/workflows/cleanup-stale-neon-e2e-branches.yml) |\n| Agent-kit governance — bundle-budget, catalog-drift, docs-frontmatter, blueprint-lifecycle, Lore commit trailers | [`package.json`](package.json) scripts, [`.github/workflows/ci.yml`](.github/workflows/ci.yml) |\n\n## Architecture\n\n```mermaid\nflowchart LR\n UI[Browser / SPA] --> API[Cloudflare Worker API]\n OPER[Operator or API client] --> API\n SRC[Third-party payload source] --> API\n API --> DB[(Postgres via Hyperdrive)]\n API --> Q[(DELIVERY_QUEUE)]\n Q --> API\n API --> RT[Realtime Durable Objects]\n RT --> UI\n API --> AI[Workers AI mapping suggestions]\n```\n\nThe Consistency Lab is a separate Worker used to measure delivery-path behavior; it is not part of the primary production request path. See [`docs/system-architecture.md`](docs/system-architecture.md).\n\nRuntime helper ownership:\n\n- provider-neutral runtime profile / env loading is extracted into the public\n `@webpresso/runtime/env` subpath\n- `ingest-lens` consumes that core directly through published\n `@webpresso/runtime/env`, with only repo-specific required-secret keys kept local\n- Neon branch lifecycle helpers remain repo-local to `ingest-lens`\n- `agent-kit` stays the tooling / verification surface (`wp`, `wp_*`, config\n subpaths), not the runtime/provider helper owner\n\n## Verify\n\nFast contributor check (no secrets required):\n\n```bash\nwp run lint # oxlint + per-package lint\nwp run check-types # tsc, no type errors\nwp run test # vitest suites\n```\n\nFull maintainer check (mirrors CI; some steps need secrets / a Neon E2E branch — **maintainer-only**):\n\n```bash\nvp check # aggregate lint + types + format\nwp run build # all packages build; client/worker bundles emitted\nwp audit docs-frontmatter\nwp audit blueprint-lifecycle\nwp run e2e --suite foundation # maintainer-only: E2E suite against a Neon E2E branch (or --suite full)\n```\n\nPRs are gated by the canonical four checks:\n\n- `check`\n- `e2e`\n- `architecture-drift`\n- `deploy-verify`\n\n## Contribute / Security / License\n\n- Contributing guide: [CONTRIBUTING.md](CONTRIBUTING.md)\n- Security policy: [SECURITY.md](SECURITY.md)\n- Code of conduct: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md)\n- License: [MIT](LICENSE)\n- Vision: [VISION.md](VISION.md)\n\n## Docs\n\n- [System architecture](docs/system-architecture.md)\n- [Architecture](docs/architecture.md)\n- [Delivery guarantees](docs/delivery-guarantees.md)\n- [Claim ↔ E2E traceability](docs/guides/claim-e2e-traceability.md)\n- [Reviewer guide](docs/project/REVIEWER-GUIDE.md)\n- [ADR index](docs/adrs/README.md)\n- [Blueprints](blueprints/README.md)\n- [Project records](docs/project/README.md)\n", "bytes": 9965, "sha": "caa44bff809a49e06e76f81b6a5492ae49446fe47c59e6bae3a829c483a025a0", "repo_slug": "ozby/ingest-lens", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/plg_ozby_ingest_lens_40398b70/readme" }