{ "markdown": "

\n \n \n \"polyrouter\"\n \n

\n\n

\n One endpoint for every model.
\n A self-hostable LLM router / gateway: OpenAI- and Anthropic-compatible, explicit-first\n routing with fallbacks, spend limits, and metadata-only cost tracking.
\n No markup, no third-party proxy — your keys, your box.\n

\n\n

\n \"CI\"\n \"License:\n \"ghcr.io/izzoa/polyrouter\"\n \"Node\n

\n\n

\n Self-hosting ·\n Connect an agent ·\n Roadmap ·\n Changelog ·\n Releases ·\n Contributing\n

\n\n

\n \"polyrouter\n
\n The dashboard overview — KPI tiles, a requests-per-hour chart, spend by model, and the live request log (each row shows its routing layer, tokens, snapshot-priced cost, and latency).\n

\n\n---\n\npolyrouter sits between your AI agents and your LLM providers. Agents talk to **one**\nendpoint with **one** key; polyrouter routes each request to the right model across your\nproviders (BYOK API keys, custom OpenAI/Anthropic-compatible endpoints, local models),\nretries down a fallback chain when a provider fails, enforces budgets, and records what\nevery request actually cost — storing **metadata only** by default, never your prompt or\nresponse bodies unless you opt in.\n\n## Features\n\n**Routing & reliability**\n\n- **Explicit-first routing** — naming a model always works; that's the reliable core.\n On top of it: per-request **tier pinning** (`x-polyrouter-tier: fast`), configurable\n **tier chains** (primary + ordered fallbacks, drag-to-reorder in the dashboard), and\n opt-in smart layers for `model: \"auto\"` — **L1 structural** (sub-millisecond local\n features; harness system prompts are fingerprinted and subtracted so a huge boilerplate\n prompt can't force everything into the top tier) and **L3 cascade** (try the cheap\n model, escalate on a failed quality check). Every smart layer **degrades to\n explicit/default** — a request never fails because routing tried to be clever.\n- **Workload routing** — beside the complexity axis, every `auto` request gets a\n **workload** class: `code` / `vision` / `structured` from the same Layer-1 structural\n features (baseline), plus `research` / `writing` from the optional semantic module\n (embedding vs bundled per-class anchors — never keywords). The class alone routes\n nothing; you opt in per class: a **Workload target** (`auto_workload` rule: class →\n tier or model) claims that class ahead of the band targets, L2, and the cascade, and\n **per-workload bands** (`auto_high` / `auto_low` rules scoped to a class) give a class\n its own strong/cheap pair — including a cascade within the class. Unrecognised or\n unconfigured classes follow the ordinary `auto` path unchanged. See\n [Workload routing](#workload-routing).\n- **Safe mid-stream semantics** — fallbacks happen freely _before_ the first token; once\n streaming has begun the model is committed, and an upstream failure terminates the\n stream with a clear error. Models are **never silently swapped mid-response**.\n- **Per-provider circuit breakers** (Redis-backed, shared across instances) with\n half-open probes that survive long LLM streams; hung connects and stalled reads trip\n them cleanly.\n\n**Protocols**\n\n- **OpenAI-compatible** `/v1/chat/completions` + `/v1/models` and **Anthropic-compatible**\n `/v1/messages`, streaming and non-streaming — any SDK that accepts a base URL works\n unchanged. Cross-protocol requests (OpenAI client → Anthropic provider and vice versa)\n go through a dedicated translation core covering multi-turn tool calls, system prompts,\n cache-control passthrough, stop reasons, and usage — locked by golden-file and\n request-fidelity contract suites.\n\n**Cost & limits**\n\n- **Immutable cost records** — every request stores its **unit-price snapshot** at request\n time; later catalog updates never rewrite history. Missing provider usage is flagged as\n estimated (`~est`), never silently nulled. Prices come from a bundled versioned catalog\n (auto-refreshed daily from LiteLLM's — one env line opts out), with per-model overrides\n for custom/local endpoints. When the catalog has no exact match, polyrouter falls back to\n an adjacent native-family rate, then to the provider's own **listed** price — each\n snapshotted as a clearly-marked estimate that never overrides a real catalog price.\n- **Budgets that actually block** — day/week/month windows, global or per-agent,\n alert-or-block at the threshold, enforced via **atomic Redis counters** that stay\n correct across multiple proxy instances.\n- **Async notifications** — SMTP and/or [Apprise](https://github.com/caronc/apprise)\n channels for budget alerts/blocks, provider-down, and failure spikes; deliveries are\n queued off the request path, deduplicated, and a failing channel never blocks a request\n or budget enforcement.\n\n**Dashboard**\n\n- SolidJS + uPlot: overview KPIs and request charts, cost breakdowns by\n model/provider/agent, per-agent usage with one-click key rotation, provider health &\n catalog sync, routing configuration, and the **decision inspector** — every request\n shows its decision layer and human-readable routing reason, tokens, snapshot-priced\n cost, and latency, plus its **workload** class when `auto` classified it (marked\n `routed` when a Workload target claimed it). On the Routing page, the **Workload\n targets** card binds each detected class to a tier or model (the `research` / `writing`\n rows are live exactly when the semantic workload source is effective, read-only\n otherwise — naming which half is missing), the **Band targets** card's **Per-workload\n bands** block gives a class its own strong/cheap pair, and the **Auto performance**\n card's **Workload mix** block shows what kinds of work `auto` carried, how many of each\n a Workload target routed, and what each cost — with unpriced / coverage /\n classifier-revision disclosures.\n- **Accessible by design**: fully keyboard-operable (real buttons, visible focus, honest\n dialog semantics), WCAG-checked contrast, `prefers-reduced-motion` support — all\n enforced by regression test suites, with the visual language pinned in\n [`STYLESEED.md`](./STYLESEED.md).\n\n**Security & privacy**\n\n- **Metadata only by default** — prompt/response bodies are never persisted unless you\n explicitly opt in (self-host only, off by default, stored encrypted at rest with\n retention controls). Provider and channel credentials are **encrypted at rest**.\n- **First-signup-wins, then invite-only** — the first account becomes the admin and\n public registration closes; teammates join via single-use, 72-hour, hash-stored\n invite links (emailed when SMTP is configured). Admins manage users, roles, and the\n registration mode from the dashboard; **disabling a user revokes sessions and agent\n keys in one stroke**, and the last enabled admin is undeletable. See\n [Users & registration](#users--registration).\n- **Two credential planes** — dashboard sessions (Better Auth: email/password +\n optional Google/GitHub/Discord OAuth, slow-hashed) vs. agent API keys\n (`poly_…`, **HMAC-SHA256 + prefix lookup** — fast per-request verification, never\n bcrypt on the hot path).\n- **SSRF-guarded egress** — every user-supplied URL the server fetches (provider base\n URLs, webhook/Apprise targets) is resolved and checked against private/loopback/\n link-local/metadata ranges, IPv6 included, with DNS-rebinding defense; loopback is\n allowed only for local models in self-host mode.\n- **Tenant isolation everywhere** — every entity access is ownership-scoped through a\n central guard; covered by a dedicated e2e suite alongside the SSRF, protocol-contract,\n and cost-immutability suites.\n- **OpenRouter app attribution** — requests to an `openrouter.ai` provider carry polyrouter's\n identity headers (`HTTP-Referer: https://polyrouter.app`, `X-OpenRouter-Title: polyrouter`)\n so the project appears in [OpenRouter's rankings](https://openrouter.ai/rankings). They are\n non-secret (an app URL and name — never prompts, keys, or user data), sent **only** to\n OpenRouter (every other provider gets neither), and never affect authentication.\n\n**Operations**\n\n- **One container** serves the SPA, the API, and the proxy on one port, next to\n PostgreSQL 16 + Redis; graceful shutdown **drains in-flight streams**; streaming applies\n backpressure. Prometheus `/metrics` + opt-in OpenTelemetry traces.\n- **CI/CD** — every push runs build/lint/typecheck, the unit suites, and e2e against real\n Postgres + Redis; tagged releases publish a **multi-arch (amd64+arm64) image** to\n [`ghcr.io/izzoa/polyrouter`](https://github.com/izzoa/polyrouter/pkgs/container/polyrouter).\n\n## How a request is routed\n\nPrecedence order, first match wins:\n\n1. **Explicit model** in the request body — always honored.\n2. **`x-polyrouter-tier` header** → that tier's chain.\n3. **Dashboard header rules** on other headers → their target tier or model.\n4. **`model: \"auto\"`** → the enabled smart layers, which engage only once nothing above\n matched, in this order: a **Workload target** for the request's detected class (if one\n is configured and usable), then the band targets — a class's own **scoped**\n `auto_high` / `auto_low` pair when you set one, the generic pair otherwise — through L1\n structural → L2 semantic (optional) → L3 cascade (whose cheap/strong legs honour the\n same class scope). See [Workload routing](#workload-routing).\n5. **`default` tier** — the guaranteed catch-all.\n\nWhatever layer decides, the tier's fallback chain applies on provider failure, budgets are\nenforced, and the decision (`decision_layer` + `routing_reason`) is recorded for the\ninspector. If a smart layer is unavailable, `auto` silently degrades to the default tier.\n\n### Workload routing\n\nBeside the complexity bands, every `auto` request that Layer 1 evaluates records a\n**workload** class — _what kind of work_ it is. The taxonomy is fixed (`v1`):\n\n| Class | Source | Detected from |\n| ------------ | -------------------------- | --------------------------------------------------------------------------------------------------- |\n| `code` | structural (baseline) | fenced-code share of the prompt (≥ 30 % and ≥ 200 chars by default — `ROUTING_WORKLOAD_THRESHOLDS`) |\n| `vision` | structural (baseline) | an image content block |\n| `structured` | structural (baseline) | a declared JSON / schema output format |\n| `research` | semantic (optional module) | the request embedding vs bundled per-class anchors — five-way cosine argmax with a margin |\n| `writing` | semantic (optional module) | same |\n| `none` | — | nothing above applied — telemetry only, never routable |\n\nAmong the structural classes vision > structured > code; the structural source always wins\nwhen it found a class, and the semantic source never emits the structural classes (a\nprose-only coding question usually records `none`). Detection is never keyword-based.\n\n**Classification alone routes nothing.** You opt in per class on the Routing page:\n\n- **Workload target** — an `auto_workload` rule binding one class to a tier or model. A\n matching `auto` request is _claimed_ before the band targets, Layer 2, and the cascade\n and served by that chain with `decision_layer = workload` (its band verdict is still\n recorded, never acted on). No target, or an unusable one (empty tier, unresolvable\n model) → the request follows the ordinary `auto` path. `none` can never be targeted.\n- **Per-workload bands** — an `auto_high` / `auto_low` rule may carry a class as its\n _scope_. For a request of that class the scoped rule of a band decides: resolvable → it\n serves; unresolvable → that band is unroutable _for the class_ (never a silent\n substitution of the generic target). Only when a band has no scoped rule for the class\n does the generic rule apply — each band independently, so `auto_low(code) → cheap-code`\n beside a generic `auto_high` is a valid hybrid. The L3 cascade plans its cheap and strong\n legs with the same scope (\"cascade within a class\": cheap-code → strong-code). A class\n that also has a usable Workload target is claimed first and its scoped bands are never\n consulted (the card says so).\n- **Semantic source** — `research` / `writing` light up when the semantic module is loaded\n (`SEMANTIC_MODEL_PATH` + `semantic` in `ROUTING_AUTO_LAYERS`, and the tenant's semantic\n layer on). A structural-`none` request is embedded **once** (the same vector serves\n Layer 2's band classification) and records the winning reserved class only when it beats\n every other class by `SEMANTIC_WORKLOAD_MARGIN` (0.05) and its cosine clears\n `SEMANTIC_WORKLOAD_MIN_SIM` (0.20); otherwise `none`. The anchors ship inside the\n package — no extra files — and the dashboard's auto-layers view reports the semantic\n workload state separately from L2. There is no new `ROUTING_AUTO_LAYERS` token and no\n new header: workload routing is a second verdict from the same layers, not a fourth.\n\n**What is recorded.** The parent request-log row carries the workload quad —\n`workload_class`, `workload_score`, `workload_source` (`structural` / `semantic`) and\n`workload_revision` (`structural/v1/c1/`, `semantic/v1/s1/`) — and the\ninspector shows the class as a chip. A claim records `decision_layer = workload`; a\nclass-scoped band or cascade decision's `routing_reason` **ends with ` scope=`**\n(class names only — never content). The Auto-performance card's **Workload mix** breaks\nrequests and spend down per class; its savings basis stays the **generic** strong target\n(disclosed whenever scoped rules exist), and a cascade whose cheap leg was class-scoped\ncontributes no evidence to the optional learning loop. Everything degrades: a missing\nmodel, a classifier fault, or a saturated embedder leaves the request on the ordinary\n`auto` path — never an error.\n\n## Architecture\n\n```mermaid\nflowchart LR\n A[\"Agents
(any OpenAI / Anthropic SDK)\"] -- \"/v1 + poly_ key\" --> P\n subgraph S[\"polyrouter — one container\"]\n P[\"Inference proxy
route · fallback · budget\"] --- T[\"Protocol
translation\"]\n D[\"Dashboard SPA + API\"]\n end\n T --> O[\"OpenAI-compatible
providers\"]\n T --> C[\"Anthropic-compatible
providers\"]\n P -.->|\"atomic counters · breakers\"| R[(\"Redis\")]\n P -->|\"RequestLog + price snapshots\"| PG[(\"PostgreSQL\")]\n D --> PG\n```\n\nThe smart routing layers all run **inside** the proxy (not as separate services): L1\nstructural and L3 cascade ship in the baseline; the optional **L2 semantic** embedder and\nits background learning loop — which adapts the classifier's centroids (its routing\nbands) from recorded cascade outcomes — are\na flag-gated add-on that reuses the same Redis/PostgreSQL, **never** in the baseline image\n(see [the semantic embedder](#optional-the-semantic-embedder-layer-2-foundation)).\n\nMonorepo (Turborepo + npm workspaces): `packages/shared` (types),\n`packages/control-plane` (NestJS — dashboard API, auth, CRUD, analytics, and the `/v1`\nproxy endpoints), `packages/data-plane` (the proxy engine the control plane hosts:\nrouting, translation, adapters, recording),\n`packages/frontend` (SolidJS SPA). Architecture overview: the code wiki in\n[`openwiki/`](./openwiki/); release history: [`CHANGELOG.md`](./CHANGELOG.md).\n\n## Self-hosting\n\nRequirements: Docker with **Compose v2**.\n\n```bash\n# One-liner (inspect it first if you prefer — see below):\ncurl -fsSL https://raw.githubusercontent.com/izzoa/polyrouter/main/install.sh | sh\n\n# Or from a checkout (uses your working tree, downloads nothing):\ngit clone https://github.com/izzoa/polyrouter.git && cd polyrouter && ./install.sh\n```\n\n> The one-liner executes a remote script. To inspect first: download `install.sh`,\n> read it, then run it — or use the checkout path.\n\nThe script checks Docker, fetches one pinned source archive (compose file and build\ncontext always the same commit), generates secrets into a mode-600 `.env` (**never**\noverwritten on re-run), and boots `docker compose -p polyrouter-selfhost up -d --build`.\nThe first build takes a few minutes. Manual alternative: copy `.env` values by hand\n(four 32-byte-hex secrets via `openssl rand -hex 32`, plus `POSTGRES_PASSWORD`) and run\nthe same compose command from the repo. Re-running the installer from **inside** the\ncreated `polyrouter/` directory is safe — it refreshes the source and keeps `.env`.\n\n### Self-host from the prebuilt image\n\nNo checkout, no local build — pull the published multi-arch (amd64 + arm64) image and run\nit next to Postgres + Redis. Make a directory with two files.\n\n**`docker-compose.yml`** — pin a version (or use `:latest`):\n\n```yaml\nname: polyrouter-selfhost\n\nservices:\n app:\n image: ghcr.io/izzoa/polyrouter:0.18.2 # or :latest — pin the current release\n restart: unless-stopped\n ports:\n - '${POLYROUTER_HOST:-127.0.0.1}:${POLYROUTER_PORT:-3001}:3001' # loopback by default\n depends_on:\n postgres: { condition: service_healthy }\n redis: { condition: service_healthy }\n stop_grace_period: 45s # drain in-flight streams on stop\n env_file: .env # optional tunables from the .env reference reach the container\n environment:\n NODE_ENV: production\n MODE: selfhosted\n BIND_ADDRESS: 0.0.0.0 # bind inside the container; host exposure is `ports`\n PORT: '3001'\n DATABASE_URL: postgresql://polyrouter:${POSTGRES_PASSWORD}@postgres:5432/polyrouter\n REDIS_URL: redis://redis:6379\n BETTER_AUTH_URL: ${APP_URL:-http://localhost:${POLYROUTER_PORT:-3001}}\n BETTER_AUTH_SECRET: ${BETTER_AUTH_SECRET:?set in .env}\n API_KEY_HMAC_SECRET: ${API_KEY_HMAC_SECRET:?set in .env}\n PROVIDER_CREDENTIAL_KEY: ${PROVIDER_CREDENTIAL_KEY:?set in .env}\n NOTIFY_CREDENTIALS_SECRET: ${NOTIFY_CREDENTIALS_SECRET:?set in .env}\n\n postgres:\n image: postgres:16-alpine\n restart: unless-stopped\n environment:\n POSTGRES_USER: polyrouter\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set in .env}\n POSTGRES_DB: polyrouter\n volumes: ['polyrouter-pg:/var/lib/postgresql/data']\n healthcheck:\n test: ['CMD-SHELL', 'pg_isready -U polyrouter -d polyrouter']\n interval: 5s\n timeout: 3s\n retries: 12\n\n redis:\n image: redis:7-alpine\n restart: unless-stopped\n volumes: ['polyrouter-redis:/data']\n healthcheck:\n test: ['CMD', 'redis-cli', 'ping']\n interval: 5s\n timeout: 3s\n retries: 12\n\n apprise: # optional notification fan-out (see below)\n image: caronc/apprise:latest\n profiles: ['apprise'] # only starts with `--profile apprise`\n restart: unless-stopped\n volumes: ['polyrouter-apprise:/config']\n\nnetworks:\n default:\n ipam:\n config:\n - subnet: ${POLYROUTER_SUBNET:-172.28.5.0/24} # deterministic CIDR for NOTIFY_ALLOWED_ENDPOINTS\n\nvolumes:\n polyrouter-pg:\n polyrouter-redis:\n polyrouter-apprise:\n```\n\n**`.env`** — the app aborts at boot if any of the five secrets is missing. Generate real\nvalues straight into the file (compose does **not** run shell substitution inside `.env`, so\nthese must be literal), then lock it down:\n\n```bash\n{\n for k in BETTER_AUTH_SECRET API_KEY_HMAC_SECRET PROVIDER_CREDENTIAL_KEY \\\n NOTIFY_CREDENTIALS_SECRET POSTGRES_PASSWORD; do\n echo \"$k=$(openssl rand -hex 32)\"\n done\n} > .env\nchmod 600 .env\n```\n\nThen boot it — migrations run on start, no build step:\n\n```bash\ndocker compose up -d\ndocker compose logs -f app # watch it come up, then sign up at http://localhost:3001\n```\n\nUpgrade by bumping the `image:` tag (or tracking `:latest`) and pulling:\n\n```bash\ndocker compose pull && docker compose up -d # migrations run on boot\n```\n\n> This is the repo's `docker-compose.yml` with two doc-friendly changes: a pinned `image:`\n> tag instead of a local `build:`, and the long list of optional pass-through vars collapsed\n> into `env_file: .env`. The service names, volumes, pinned subnet, and `apprise` profile all\n> match, so the **Operations** and **Apprise** notes below apply unchanged — every variable in\n> the `.env` reference defaults when unset\n> (the five secrets are required either way). To go public, expose the port and\n> set `APP_URL` as in **Claim the instance** below.\n>\n> If `docker compose pull` returns `unauthorized`/`denied`, run `docker login ghcr.io`\n> first — on a fork, the fork's own GHCR package may also not be public yet.\n\n> **Already used the installer or a checkout?** Skip the local build by setting\n> `POLYROUTER_IMAGE=ghcr.io/izzoa/polyrouter:latest` (or a pinned `:X.Y.Z`) in `.env` and\n> running the compose command **without** `--build`. On a **fetch install** the compose\n> flags go **before** the subcommand, exactly as the installer prints:\n> `docker compose -p polyrouter-selfhost --env-file .env -f src/docker-compose.yml\n--project-directory src pull`.\n\n> **Compose commands below — checkout vs. one-line install.** The bare\n> `docker compose -p polyrouter-selfhost …` form shown below assumes a **checkout**\n> (compose file at the repo root). A **one-line (fetch) install** keeps the compose\n> file under `src/` with `.env` beside it, so run the commands from inside the\n> `polyrouter/` directory with the `--env-file .env -f src/docker-compose.yml\n--project-directory src` flags placed before the subcommand — exactly the manage\n> command the installer prints when it finishes.\n\n**Claim the instance, then expose it.** The app publishes on **loopback only** by\ndefault and the **first account to sign up becomes the admin** — sign up at\n`http://localhost:3001` before exposing anything. To go public, set in `.env`:\n\n```bash\nPOLYROUTER_HOST=0.0.0.0 # or keep loopback and use a reverse proxy\nPOLYROUTER_PORT=3001\nAPP_URL=https://polyrouter.example.com # the real origin (auth callbacks/cookies)\n```\n\nthen `docker compose -p polyrouter-selfhost up -d`. Put TLS and access control in\nfront with your reverse proxy — **`/api/health` and `/metrics` are unauthenticated\nby design** (orchestration + Prometheus); restrict them at the proxy if the port is\npublic, or set `METRICS_ENABLED=false`.\n\n**Container health probe:** the image's own `HEALTHCHECK` targets `/api/health` on\nthe configured `PORT` (default `3001`) — the identical exec-form Node probe on both\nthe baseline and `-semantic` variants, with **no `wget`/`curl` dependency** — so\nchanging `PORT` needs no healthcheck override, and the documented Node probe form\nbelow runs unchanged on both variants. An override that shells out to base-image\nutilities is **outside that guarantee**: a `wget` check works on the Alpine baseline\nand breaks on the `-semantic` image's Debian-slim base. If your orchestrator defines\nits own check anyway, use this form — shown as a compose override; a Kubernetes\n`livenessProbe.exec.command` takes the same `[\"node\", \"-e\", …]` array without the\nleading `CMD`:\n\n```yaml\nhealthcheck:\n test:\n [\n 'CMD',\n 'node',\n '-e',\n \"const p=process.env.PORT||3001;require('http').get('http://127.0.0.1:'+p+'/api/health',r=>process.exit(r.statusCode===200?0:1)).on('error',()=>process.exit(1))\",\n ]\n```\n\n### `.env` reference\n\n| Variable | Default | Purpose |\n| --------------------------------------------------------------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `BETTER_AUTH_SECRET`, `API_KEY_HMAC_SECRET`, `PROVIDER_CREDENTIAL_KEY`, `NOTIFY_CREDENTIALS_SECRET` | generated | Required 32-byte-hex secrets (sessions, agent-key HMAC, credential + channel encryption at rest) |\n| `POSTGRES_PASSWORD` | generated | Database password — **initialization-only**: changing it later does NOT rotate the role password in postgres |\n| `POLYROUTER_HOST` / `POLYROUTER_PORT` | `127.0.0.1` / `3001` | Host interface/port the app is published on |\n| `APP_URL` | `http://localhost:3001` | Public origin (Better Auth base URL) — set it when exposing. Also gates **links in notification emails**: a loopback value omits them (see Notification emails) |\n| `METRICS_ENABLED` | `true` | Prometheus `/metrics` (404 when `false`) |\n| `OTEL_ENABLED` / `OTEL_EXPORTER_OTLP_ENDPOINT` | `false` / SDK default | OpenTelemetry traces for the proxy path (batched OTLP/HTTP export) |\n| `GOOGLE_/GITHUB_/DISCORD_CLIENT_ID`+`_SECRET` | unset | Optional OAuth sign-in providers |\n| `APPRISE_API_URL` + `NOTIFY_ALLOWED_ENDPOINTS` | unset | Optional Apprise fan-out — see below |\n| `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` / `SMTP_FROM` / `SMTP_SECURE` | unset (`PORT` 587, `SECURE` starttls) | Server-wide SMTP for password-reset **and invite** email — **active only when both `SMTP_HOST` and `SMTP_FROM` are set; otherwise password reset silently never sends and invites must be delivered by copying the link.** Rely on OAuth if you don't set it |\n| `ROUTING_AUTO_LAYERS` | `structural` | Which smart-routing layers are on. **Cascade (cheap→escalate) is OFF until you set `structural,cascade`** — the dashboard toggle just shows it greyed out otherwise |\n| `ROUTING_STRUCTURAL_WEIGHTS` | built-ins | JSON override for the Layer-1 classifier. Ambient keys (`size` `code` `tools` `schema` `depth` `multimodal` `maxTokens`) merge over the defaults and normalize to sum 1; the `reasoning` key is the declared-hint adjustment magnitude in `[0, 0.5]` (default `0.1`), NOT normalized. A declared `reasoning_effort`/`thinking` steers the score; a maximal declaration routes `auto_high` directly |\n| `ROUTING_WORKLOAD_THRESHOLDS` | built-ins | JSON override for the structural **workload** classifier (detection only — routing happens through a configured Workload target): `codeShare` in `(0, 1]` (default `0.3`) and integer `codeMinChars ≥ 0` (default `200`). An `auto` request records workload `code` when fenced code is at least that share of the scanned window AND at least that many chars; `vision` (an image block) and `structured` (a declared JSON output format) are binary. Unknown keys / out-of-range values fail boot. |\n| `CALIBRATION_SCHED_ENABLED` / `CALIBRATION_SCHED_CRON` | `true` / `0 4 * * *` | The per-tenant threshold-calibration sweep (opt-in PER TENANT from the Routing page; this pair gates the background worker instance-wide) |\n| `CALIBRATION_WINDOW_DAYS` / `CALIBRATION_MIN_EDGE_SAMPLES` / `CALIBRATION_STEP` / `CALIBRATION_MAX_DRIFT` | `14` / `50` / `0.02` / `0.1` | Calibration rails: evidence window, minimum fresh edge-zone samples (hard floor 50 — only raisable), bounded per-run step, and the max total drift from the instance thresholds. Every move is audited and one click from reverted |\n| `BUDGET_FAIL_OPEN` | `true` | On a Redis/enforcement fault, block budgets **admit** the request (availability-first). Set `false` for a hard cap that returns `503` instead |\n| `TRUSTED_PROXY_CIDRS` | unset | CIDRs of reverse proxies allowed to set `X-Forwarded-For` (rate-limit client-IP trust) — set it when behind a proxy |\n| `NOTIFY_APPRISE_EGRESS_CONFIRMED` | `false` | Cloud-mode (`MODE=cloud`) acknowledgement before Apprise delivery runs — the SSRF allowlist (`NOTIFY_ALLOWED_ENDPOINTS`) is still enforced independently |\n| `PRICING_REFRESH_URL` | LiteLLM catalog | Source for pricing refreshes (a bundled snapshot ships by default; the Settings page shows catalog status + a Refresh-now button for admins) |\n| `PRICING_REFRESH_SCHED_ENABLED` / `PRICING_REFRESH_SCHED_CRON` | `true` / `30 4 * * *` | **Daily automatic pricing refresh — ON by default** (self-host only): one outbound GET of LiteLLM's public price catalog per day; no tenant data is sent. Set `PRICING_REFRESH_SCHED_ENABLED=false` to opt out; manual refresh keeps working |\n| `PROXY_FIRST_EVENT_TIMEOUT_MS` / `PROXY_IDLE_TIMEOUT_MS` | `30000` / `30000` | Time-to-first-token / buffered-read idle bound — **raise both for slow local models** (a 30s prefill would otherwise 503 and trip the breaker). For ONE slow provider in a mixed chain, prefer its per-provider override (the provider form's \"Advanced — patience for slow models\") over raising the instance default: timeouts trip that provider's breaker, and while its recovery probe runs with doubled patience, a provider slower than 2× its bound stays in a skip loop until its patience is raised |\n| `SEMANTIC_MODEL_PATH` | unset | Opt-in **Layer 2 semantic embedder**: path to a local model bundle (see the semantic-layer section) — pair it with `semantic` in `ROUTING_AUTO_LAYERS`. Also enables the semantic **workload** source (`research` / `writing`). Unset = the module is absent entirely; a set-but-broken path fails boot loudly |\n| `EVENTS_ENABLED` | `true` | Dashboard live event stream (`GET /api/events`). `false` turns it off entirely and the dashboard stays on its normal polling refresh — no feature is lost, only push |\n| `EVENTS_HEARTBEAT_MS` | `25000` | Keep-alive interval; must stay under your proxy's idle-reap window (boot fails if ≥ 60s). Also bounds how fast a revoked session's open stream is closed |\n| `SEMANTIC_TIMEOUT_MS` / `SEMANTIC_MAX_INPUT_CHARS` / `SEMANTIC_CONCURRENCY` | `50` / `2000` / `2` | Embedder bounds: per-embed hard timeout, input cap before tokenization, concurrent-inference cap (saturation skips the layer for that request). Out-of-bounds values reject boot |\n| `SEMANTIC_WORKLOAD_MARGIN` / `SEMANTIC_WORKLOAD_MIN_SIM` | `0.05` / `0.20` | Semantic **workload** rails (need the semantic module): a structural-`none` `auto` request records `research` / `writing` only when the winning class leads the runner-up by ≥ `MARGIN` (the discriminating rail) and its cosine is ≥ `MIN_SIM` (a near-orthogonal guard; the spike showed 0.30 cost recall for no precision). Both ≤ 4 decimals, both part of the `semantic/…` workload revision, so a change never silently mixes two populations |\n| `POLYROUTER_SUBNET` / `POLYROUTER_IMAGE` | `172.28.5.0/24` / built | Compose network CIDR (change on a collision) / prebuilt image override |\n\n> The optional tunables above are compose pass-through: set one in `.env` and it reaches\n> the container (the compose file sets the deploy-invariant ones — bind address, mode,\n> `NODE_ENV`, DB/Redis URLs — itself). In the repo's `docker-compose.yml` (checkout and\n> installer installs) that hand-off is an **explicit allowlist**: the app service's\n> `environment:` block is the list of what actually crosses into the container, and a key\n> you put in `.env` without adding it there is silently ignored. The prebuilt-image example\n> above passes everything through instead (`env_file: .env`), so it has no such filter. The `SEMANTIC_*` knobs are the one deliberate exception —\n> they are declared in `docker-compose.semantic.yml`, so layer that override file to tune\n> them (`SEMANTIC_MODEL_PATH` is baked into the `-semantic` image, so the embedder loads\n> either way — but semantic routing also needs `semantic` in `ROUTING_AUTO_LAYERS`, which\n> the override file sets; without it the layer silently stays off). The config registry in the source\n> (`packages/*/src/**` config schemas) is the exhaustive list — defaults,\n> required-in-production secrets, and dev fallbacks are declared there.\n\n**Secret rotation caveat:** `PROVIDER_CREDENTIAL_KEY` and `NOTIFY_CREDENTIALS_SECRET`\nencrypt stored provider/channel credentials — rotating them orphans those rows (you\nwould re-enter the credentials). This is why the installer never regenerates `.env`.\n\n### Optional: the semantic embedder (Layer 2 foundation)\n\nThe optional semantic stack embeds request text locally (CPU ONNX, ~5–20 ms)\nso the auto-router can classify what the structural layer finds ambiguous.\nIt is **never part of the baseline install**: the runtime is an optional peer\ndependency and no model ships in the baseline image (CI asserts this). The\nrouting behavior that consumes it arrives with the semantic-routing\ncapability; a **batteries-included `-semantic` image variant** (runtime +\nreference model pre-baked) ships with the semantic dashboard change.\n\n#### Batteries-included: the `-semantic` image (zero setup)\n\nEvery tagged release also publishes a multi-arch `-semantic` image with the\nONNX runtime **and** the reference embedding model\n(`sentence-transformers/all-MiniLM-L6-v2`, Apache-2.0, 384-dim) baked in at\nbuild time. **The image presets only `SEMANTIC_MODEL_PATH` — the model half.**\nThe layer's capability is a pair, so the dashboard's L2 row stays \"off\ninstance-wide\" (and names which half is missing) until `ROUTING_AUTO_LAYERS`\nalso lists `semantic`. Nothing is downloaded at runtime. The zero-setup path\nis the overlay compose, which sets that env for you:\n\n```sh\ndocker compose -f docker-compose.yml -f docker-compose.semantic.yml up -d\n```\n\nRunning the published image directly? Set the flag yourself — otherwise L2\nstays off even though the boot log shows the embedder loading:\n\n```sh\ndocker run … -e ROUTING_AUTO_LAYERS=structural,semantic,cascade \\\n ghcr.io/izzoa/polyrouter:latest-semantic\n```\n\nThe baseline image is unchanged — it carries no ONNX runtime and no model\nfiles, and CI gates that on every build. The model's weights are the glibc\nbuild's only reason for a Debian base (the runtime's prebuilt binaries do not\nrun on Alpine/musl).\n\n**Bring your own model:** mount a bundle over the baked one and repoint the env\n— the same fail-fast boot contract applies:\n\n```sh\n# in .env\nSEMANTIC_MODEL_DIR=/abs/path/to/your/bundle # holds model.onnx + vocab + manifest.json\nSEMANTIC_MODEL_PATH=/app/models/custom\n# then uncomment the `volumes:` mount in docker-compose.semantic.yml\n```\n\nTo enable it on a source install instead:\n\n```sh\nnpm install onnxruntime-node@1.27.0 # the optional peer, exact-pinned\n```\n\nThen set BOTH the model path and the capability flag (`semanticAvailable`\nrequires the layer token as well as a loaded bundle):\n\n```sh\nSEMANTIC_MODEL_PATH=/path/to/models/minilm\nROUTING_AUTO_LAYERS=structural,semantic\n```\n\nThe **model bundle** directory looks like:\n\n```\nmodels/minilm/\n manifest.json # the v1 bundle contract (below)\n vocab.txt # WordPiece vocabulary, one token per line\n model.onnx # the embedding model (MiniLM/bge-small class, 384-dim)\n```\n\n```json\n{\n \"schemaVersion\": 1,\n \"tokenizer\": {\n \"type\": \"wordpiece\",\n \"vocabFile\": \"vocab.txt\",\n \"lowercase\": true,\n \"unkToken\": \"[UNK]\",\n \"clsToken\": \"[CLS]\",\n \"sepToken\": \"[SEP]\",\n \"padToken\": \"[PAD]\",\n \"maxTokens\": 256\n },\n \"model\": {\n \"file\": \"model.onnx\",\n \"inputNames\": {\n \"inputIds\": \"input_ids\",\n \"attentionMask\": \"attention_mask\",\n \"tokenTypeIds\": \"token_type_ids\"\n },\n \"outputName\": \"last_hidden_state\",\n \"outputKind\": \"token_embeddings\",\n \"dims\": 384,\n \"pooling\": \"mean\",\n \"normalize\": true\n }\n}\n```\n\nBoot semantics: unset path → module absent, zero overhead; valid bundle →\nload + warmup at startup (requests never pay first-inference JIT); broken\nbundle → **boot fails fast** naming the file and reason (an explicit opt-in\nnever runs silently degraded). Nothing is fetched over the network at boot or\nruntime. A request's embedded text and vector are never logged or persisted; the opt-in\nlearning loop stores only cohort-aggregated sums, never a single request's embedding.\n\n**What the module enables** (all local; each opt-in per tenant in the dashboard): Layer-2\nband classification for the requests Layer 1 finds ambiguous; the optional learning loop\nthat adapts its centroids from recorded cascade outcomes; and the **semantic workload\nsource** — `research` / `writing` detection for structural-`none` `auto` requests (see\n[Workload routing](#workload-routing)). The workload anchors are plain bundled data in the\npackage — no extra files, nothing downloaded — and boot behind their own readiness check,\nso the auto-layers view reports \"semantic workload\" separately from L2 and names what is\nmissing. Rails: `SEMANTIC_WORKLOAD_MARGIN` / `SEMANTIC_WORKLOAD_MIN_SIM`.\n\n### Optional: Apprise notifications\n\n```bash\ndocker compose -p polyrouter-selfhost --profile apprise up -d\n```\n\nand add **both** lines to `.env` (the SSRF guard requires an allowlist entry for a\nprivate-range host; the port bound is optional but keep it — by design, spec §10.1):\n\n```bash\nAPPRISE_API_URL=http://apprise:8000\nNOTIFY_ALLOWED_ENDPOINTS=apprise,172.28.5.0/24,8000\n```\n\nThe compose network is pinned to `172.28.5.0/24` so that CIDR is deterministic;\nchange both places if it collides with your network.\n\n### Notification emails and their links\n\nEmails are sent as both plain text and branded HTML — a text-only client sees\nthe same wording it always did; an HTML client gets a laid-out message with a\nbutton through to the relevant page (a provider alert opens Providers, a budget\nalert opens Limits, and so on). The layout is deliberately **asset-free**: no\nimages, web fonts, or externally hosted anything, so it renders identically on\nan instance that isn't publicly reachable and triggers no remote fetches.\n\n**Those buttons appear only when `APP_URL` is an address your recipients can\nactually reach.** With the default (`http://localhost:3001`) the link is omitted\nentirely rather than sending a `127.0.0.1` URL that would be dead in someone's\ninbox — or worse, on a phone, resolve to the phone. Setting `APP_URL` to a\nloopback value explicitly does the same thing; that is deliberate, not a bug.\n\n| `APP_URL` | Links in email |\n| ---------------------------------------------------------- | ------------------------------------------------------------------------- |\n| unset, or `http://localhost:3001` (default) | omitted |\n| any `localhost` / `127.0.0.1` / `[::1]` value | omitted |\n| `http://192.168.1.50:3001`, `http://polyrouter.local:3001` | **yes** — a LAN address is often exactly right for a self-hosted instance |\n| `https://polyrouter.example.com` | **yes** |\n| a value carrying credentials, or a non-`http(s)` scheme | omitted (it is never rendered as a link) |\n\nTwo practical notes: the value is read **at boot**, so restart after changing it\n(`docker compose -p polyrouter-selfhost up -d`); and only the _origin_ is used, so\nan `APP_URL` with a path (`https://host/polyrouter/`) produces links at the domain\nroot. Serving the dashboard under a subpath is not currently supported for email\nlinks.\n\nChat channels (Apprise) additionally carry a per-event severity, so a\nprovider-down or budget-block notification is visually distinct from an\ninformational summary at the target, with the page link on its own line.\n\n### Operations\n\n- **Upgrade:** pull/re-download the source, then `docker compose -p polyrouter-selfhost up -d --build` — or, on the prebuilt image, `docker compose -p polyrouter-selfhost pull && docker compose -p polyrouter-selfhost up -d`. Migrations run on boot either way.\n- **Backup:** the `polyrouter-pg` volume is the data; `docker compose exec postgres pg_dump -U polyrouter polyrouter > backup.sql`.\n- **Stop/restart:** in-flight streaming responses are drained on `docker stop` — new inference is refused and the app waits up to **15s** for open streams to finish (`streamDrainDeadlineMs`), aborting any still running at that deadline; Compose separately allows 45s (`stop_grace_period`) before SIGKILL. Deploys don't sever live completions that finish within the drain window.\n- **One app replica only:** boot migrations take no advisory lock — do not `--scale app`. The dashboard's live event stream also fans out **in-process** for this reason; multi-instance fanout (Redis pub/sub) is a documented graduation, not a supported topology today.\n- **Reverse proxies must not buffer `/api/events`.** The dashboard receives live updates over Server-Sent Events on that one path. polyrouter already sends `X-Accel-Buffering: no` and `Cache-Control: no-cache, no-transform` and heartbeats every 25s (under the usual ~60s idle-reap), but if you front it with nginx/Traefik/Cloudflare you may need to disable response buffering and raise the read timeout for it (nginx: `proxy_buffering off;`). If the stream is blocked the dashboard says **Polling** instead of **Live** and keeps working on its normal refresh — you lose push, never function.\n- **Verify an install:** `scripts/selfhost-smoke.sh` runs the end-to-end smoke pass (health, admin bootstrap, live-stream drain, metadata-only persistence) against a throwaway stack.\n- **Compliance note:** using flat-rate consumer _subscriptions_ (ChatGPT Plus, Claude Max) programmatically likely violates those providers' ToS — polyrouter supports the provider kind but surfaces the risk; BYOK API keys and local models don't carry it.\n\n### Subscriptions (OAuth)\n\nSubscription providers can connect through a guided **OAuth wizard** instead of pasting a\ntoken by hand: pick a preset (**Claude Pro/Max** or **ChatGPT Plus/Pro**), sign in at the\nprovider's link, and paste the redirect URL — or the `code#state` string it shows — back\ninto the dashboard. polyrouter verifies the `state`, exchanges the code (PKCE), and stores\nthe access + refresh tokens **encrypted at rest**. Tokens **auto-refresh** before expiry\n(safe across multiple requests and instances); if the provider revokes the grant, the card\nflags the expired sign-in with a **Reauthorize** button that reopens the connect wizard, and your fallback chain keeps\nserving traffic meanwhile.\n\nHonest caveats:\n\n- **These integrations ride undocumented contracts.** The OAuth endpoints and what the\n provider accepts from subscription tokens are ecosystem-known, not published APIs — the\n provider can change them at any time. Each preset ships enabled only after its own live\n verification — both passed on 2026-07-18 (`scripts/verify-claude-oauth.md`,\n `scripts/verify-chatgpt-oauth.md` record the runs and the pinned constants);\n failures surface as a clear provider error, and polyrouter **never impersonates the\n first-party client** beyond the documented headers — no client-fingerprint headers and no\n imitation system prompts, even if that means a preset stays disabled.\n- **ChatGPT specifics:** the ChatGPT preset speaks the backend's **Responses API** with\n `store: false` on every call (nothing is retained server-side by request), and any\n reasoning items the backend emits are **dropped, never persisted or replayed** — a\n deliberate metadata-only trade that can reduce multi-turn tool-use quality on\n reasoning-heavy models. The backend also **rejects `max_tokens` and sampling\n parameters** (`temperature`/`top_p`) — requests through this provider ignore them\n (verified live; usage is flat-rate, so no billing surprise), and it only serves\n streaming upstream (polyrouter buffers transparently for non-streaming clients).\n- **The ToS compliance note above applies** — pair a subscription with a pay-per-token\n fallback provider.\n- **Key rotation:** changing `PROVIDER_CREDENTIAL_KEY` invalidates stored credentials;\n OAuth-connected providers will then ask to be reauthorized.\n\n### Users & registration\n\nThe **first account to sign up owns the instance**: it becomes the admin and\nregistration immediately closes to **invite-only** (racing sign-ups during that\nfirst moment are refused — exactly one bootstrap winner). Everything after that\nis managed from the admin-only **Users** page (account menu, bottom of the\nsidebar):\n\n- **Invites** — single-use links pinned to an email, expiring after 72 h. With\n server SMTP configured (`SMTP_HOST` + `SMTP_FROM`) the invite is emailed\n automatically; without it, copy the link from the dashboard and deliver it\n yourself — issuing never depends on SMTP. Only a token hash is stored (plus a short lookup\n prefix — never the full token), and the raw token travels in the link's `#fragment`, which browsers never send to\n servers or proxies.\n- **Roles** — promote/demote admins. The last _enabled_ admin can never be\n deleted, demoted, or disabled (the API refuses with `409`).\n- **Disable** — cuts both credential planes at once: dashboard sessions are\n revoked immediately and every agent API key the user owns stops working on\n `/v1`. Re-enabling requires a fresh sign-in.\n- **Registration mode** — reopen public sign-up (`open`) or keep it\n `invite_only`, live from the dashboard.\n\n**Upgrading an existing instance closes public sign-up** (the migration seeds\n`invite_only`); reopen it under Users → Registration if you want walk-in\nsign-ups back. Break-glass if you ever lock yourself out (no enabled admin\nleft): fix the row directly in Postgres, then sign in again —\n\n```sql\nUPDATE \"user\" SET disabled = false, role = 'admin' WHERE email = 'you@example.com';\n```\n\n## Connect an agent\n\npolyrouter speaks the OpenAI and Anthropic wire protocols, so any tool that lets you\nset a **base URL** and **API key** works with no other changes. Create an agent key in\nthe dashboard (**Agents → New** — it looks like `poly_…` and is shown once), then point\nyour client at your instance:\n\n- **Base URL:** an **OpenAI** SDK/client uses `https:///v1`; an **Anthropic** SDK uses\n `https://` (it appends `/v1/messages` itself). The raw endpoints are\n `/v1/chat/completions`, `/v1/messages`, and `/v1/models`\n- **API key:** the `poly_…` key from the dashboard (sent as `Authorization: Bearer poly_…`)\n- **Model:** an explicit model id (e.g. `gpt-4o`), `auto` (let the router pick), or a tier\n via the `x-polyrouter-tier` header — a tier you've created under Routing (only `default`\n exists out of the box; an unknown tier value falls back to default routing)\n\n```bash\n# OpenAI-compatible\ncurl https:///v1/chat/completions \\\n -H \"Authorization: Bearer poly_your_key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"auto\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n\n# Anthropic-compatible\ncurl https:///v1/messages \\\n -H \"Authorization: Bearer poly_your_key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"claude-3-5-sonnet\",\"max_tokens\":256,\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n\n# Pin a routing tier instead of a model:\n# -H \"x-polyrouter-tier: fast\" (with \"model\":\"auto\")\n```\n\nThe router applies your configured fallbacks, spend limits, and cost tracking on every\ncall. Explicit routing (a named model) is the reliable core; `auto` and tier routing are\nopt-in and always degrade back to explicit/default.\n\n### Terminal agents (OpenClaw, Hermes)\n\nTerminal-native agents are configured with a **config file** rather than SDK code.\nBoth speak the OpenAI-compatible endpoint, so point their `base_url` at\n`https:///v1` with your `poly_…` key and let the router pick the model\n(`auto`). The dashboard's **Agents → New** picks the harness and shows the exact block once;\nthe equivalents are:\n\n**OpenClaw** — `~/.openclaw/openclaw.json` (JSON5): register the router as a provider and\nmake it the default —\n\n```json5\n{\n models: {\n providers: {\n polyrouter: {\n ", "bytes": 60000, "sha": "28f5655a7e9a50dbe2804ae5ee54f0065fa55885654ceb9ac0328d716d6110af", "repo_slug": "izzoa/polyrouter", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/okf_izzoa_polyrouter_openwiki_index_md_e78241f4/readme" }