{
  "markdown": "# mcp-queue-doctor\n\nAn [MCP](https://modelcontextprotocol.io) server that **diagnoses** Postgres job\nqueues — [pg-boss](https://github.com/timgit/pg-boss) and\n[graphile-worker](https://github.com/graphile/worker). Retry storms, stuck\nworkers, missed schedules, expiry overruns: what is broken, why, and the safest\nway to recover.\n\n```\n❌ \"3 jobs in enrichment/corpus-fill are in state 'failed'.\"\n\n✅ \"enrichment/corpus-fill failed 140 times over 3m, peaking at 50 failures in a\n   single minute. 91% share one error, which looks like an upstream rate limit.\n   This is one fault reproduced many times, not many separate faults — so the fix\n   belongs at the source, and retrying the jobs individually will reproduce it.\n\n   Recovery, safest first:\n     1. Stop enqueuing to this queue — every new job feeds the same failure.\n     2. Confirm when the upstream quota resets; treat that as the time to resume.\n     3. Add a cooldown gate after N consecutive 429s.\n     ⚠ Do NOT bulk-retry yet — the upstream is still limited.\n\n   Evidence: 140 failures, 91% 'HTTP 429 Too Many Requests (daily quota\n   exceeded)', peak 50/min, busiest minutes [...], 12 other errors [...]\"\n```\n\nThe second answer is the product. Every finding carries the evidence it was\ndrawn from, so you — or an agent — can check the reasoning instead of trusting\nit.\n\n## Where the heuristics come from\n\nThe rules are extracted from a morning health check that has run daily in\nproduction since April 2026 against a pg-boss instance driving ~30 cron queues.\nEvery threshold was tuned by a real false positive or a real missed failure, and\neach rule below names the incident that motivated it. That provenance is the\npoint: these are not heuristics invented for a README.\n\n## Install\n\n```bash\nnpm install -g mcp-queue-doctor\n```\n\n```json\n{\n  \"mcpServers\": {\n    \"queue-doctor\": {\n      \"command\": \"mcp-queue-doctor\",\n      \"env\": {\n        \"QUEUE_DOCTOR_DATABASE_URL\": \"postgres://readonly:pw@localhost:5432/app\"\n      }\n    }\n  }\n}\n```\n\nThen ask: *\"Is anything wrong with my job queue?\"*\n\n**Want to see it work first?** [`examples/demo`](examples/demo) spins up a\nthrowaway Postgres and manufactures seven failures in about a minute. It also\nplants a graphile-worker instance in the same database, where four of the seven\nrules go deliberately silent — the clearest way to see what capability\ndeclaration actually buys you.\n\n## Connecting it\n\nThe server speaks stdio, so every MCP client starts it as a subprocess. The\nonly thing that varies is where the config lives — and whether that process can\nreach your database.\n\n**Claude Desktop** — `~/Library/Application Support/Claude/claude_desktop_config.json`\non macOS, `%APPDATA%\\Claude\\claude_desktop_config.json` on Windows. Use the\n`mcpServers` block above, then restart the app.\n\nDesktop launches its subprocesses from the app bundle, not a login shell, so\n`PATH` is minimal and a bare `mcp-queue-doctor` or `npx` often fails to resolve.\nGive it an absolute path — `which mcp-queue-doctor` after a global install, or\nthe absolute path to `npx` with `[\"-y\", \"mcp-queue-doctor\"]` as its args.\n\n**Claude Code** — one command, no file editing:\n\n```bash\nclaude mcp add queue-doctor -e QUEUE_DOCTOR_DATABASE_URL=postgres://... -- npx -y mcp-queue-doctor\n```\n\nAdd `-s project` to write a checked-in `.mcp.json` at the repo root instead of\nyour personal config, so everyone working in that repo gets the tool.\n\n**Cloud / remote sessions** (Claude Code on the web, and any other headless\nrunner) — a checked-in `.mcp.json` is the only mechanism that works, because\nnobody is there to answer an approval prompt. Reference the connection string\nrather than committing it; Claude Code expands `${VAR}` and `${VAR:-default}`\nin `.mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"queue-doctor\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"mcp-queue-doctor\"],\n      \"env\": { \"QUEUE_DOCTOR_DATABASE_URL\": \"${QUEUE_DOCTOR_DATABASE_URL}\" }\n    }\n  }\n}\n```\n\nProject-scoped servers still need to be trusted before they start. In a\nheadless session that means setting `enableAllProjectMcpServers: true` in the\nrepo's `.claude/settings.json`, since the interactive approval never arrives.\n\n**Reachability is the real constraint**, not the config. The server runs\nwherever the client runs, and it connects to Postgres directly — there is no\nhosted component in between. A cloud sandbox can therefore only diagnose a\ndatabase inside that sandbox: the [demo stack](examples/demo), or a dev stack\nthe session brought up itself. A production queue bound to loopback on your own\nhost is not reachable from a sandbox at all, and exposing it to make it\nreachable is the wrong trade.\n\nDiagnose production from a client on a machine that already has a route to it —\nyour laptop, over an SSH tunnel:\n\n```bash\nssh -N -L 5432:127.0.0.1:5432 prod-host\n```\n\nand point `QUEUE_DOCTOR_DATABASE_URL` at `127.0.0.1:5432`. The tunnel is the\naccess grant, it lasts exactly as long as the terminal stays open, and the\ncredentials never leave your machine.\n\n## Tools\n\n| Tool | Answers |\n|:---|:---|\n| `diagnose` | **Start here.** Runs the whole rule catalog, returns ranked findings with evidence and recovery steps |\n| `queue_overview` | Per-queue counts by state, stuck jobs, and each queue's expiry/retention/retry config |\n| `failed_jobs` | Failures in a window with error messages, plus a per-queue error-frequency breakdown |\n| `stuck_jobs` | Jobs active past a threshold, with age, expiry, and heartbeat staleness |\n| `missed_schedules` | Cron queues whose latest firing is older than their expression implies |\n| `schedule_status` | Every registered schedule with cron, timezone, last firing, and next expected |\n| `job_detail` | One job's full record: state, timings, retries, payload, output |\n| `server_info` | Connectivity, detected schema, matched dialect, and reduced capabilities |\n\nSchedule expectations are derived from pg-boss's own `schedule` table by parsing\neach cron expression, so **the common case needs no configuration**. The health\ncheck this was extracted from carried a hand-maintained list of expected jobs\nthat silently stopped covering whatever nobody remembered to add.\n\n## The rule catalog\n\n| Rule | Fires when | Motivating incident |\n|:---|:---|:---|\n| `retry-storm` | Many failures, densely packed, dominated by one error | A daily API quota tipped over and 875 corpus-fill jobs failed in one night. The count suggested 875 problems; the shape showed one |\n| `expiry-overrun` | Failure durations cluster at the queue's expiry | A full-corpus sweep couldn't finish inside a 30-minute expiry once upstream throttling slowed it. It reported as a job failure nightly; the fix was an internal wall-clock budget |\n| `stuck-jobs` | Jobs active far too long, or heartbeats stopped | A worker killed without graceful shutdown leaves rows active until maintenance reclaims them |\n| `missed-schedule` | Latest firing predates the last expected tick | Distinguishes \"never fired\" (scheduler never booted) from \"stopped firing\" |\n| `duplicate-registration` | A cron queue enqueued twice for one tick | An instrumentation hook invoked job registration twice per process, so every cron ran double for weeks |\n| `retention-window` | Failed-row count disagrees with the windowed count | A health email stayed yellow for days after the bug was fixed, counting rows that failed days earlier |\n| `dead-queue` | Registered long ago, unscheduled, holds nothing | A producer that stopped, or a registration dropped in a refactor |\n\nFailures are classified (`rate_limit`, `transient_transport`, `auth`,\n`not_found`) because the class changes the advice: the right response to a storm\nof 429s is close to the opposite of the right response to connection resets.\n\n## Backends\n\n| Backend | Support | Verified against |\n|:---|:---|:---|\n| pg-boss v11+ | Full | 11.1.2 (schema 26), 12.27.0 (schema 37) |\n| pg-boss v10 | Recognised, refused — see below | 10.4.2 (schema 24) |\n| pg-boss v9 and earlier | Recognised, refused | — |\n| graphile-worker 0.17 | Partial, capability-declared | 0.17.3 |\n\nSelect with `QUEUE_DOCTOR_BACKEND=pgboss` (default) or `graphile`; the schema\ndefault follows the backend.\n\n### Capabilities, not zeros\n\nBackends don't just name columns differently — they model work differently.\ngraphile-worker **deletes a job when it succeeds**, has no per-job expiry, no\nworker heartbeats, and keeps cron expressions in a file rather than the\ndatabase. So \"how many completed in the last day\" has no answer there at any\nprice.\n\nEvery backend therefore declares what it can answer, and rules that depend on\nmissing data **stay silent** rather than reporting a zero — a zero reads like a\nmeasurement.\n\n| Rule | pg-boss v11+ | graphile-worker |\n|:---|:---:|:---:|\n| `retry-storm` | ✅ | ✅ |\n| `stuck-jobs` | ✅ (with heartbeats) | ✅ (age only) |\n| `expiry-overrun` | ✅ | — no expiry exists |\n| `missed-schedule` | ✅ | — cron lives in a file |\n| `duplicate-registration` | ✅ | — no firing history |\n| `retention-window` | ✅ | — nothing is retained |\n| `dead-queue` | ✅ | — no queue registry |\n\n`server_info` reports the capability set and spells out each limitation.\n\n## Versioned against pg-boss\n\npg-boss's tables are not a stable API. Across versions it has renamed every\ntimestamp column (`createdon` → `created_on`), dropped a whole table (`archive`,\nremoved in v11), changed a duration from an interval to an integer (`expire_in`\n→ `expire_seconds`), partitioned the job table, and added columns\n(`heartbeat_on`) that materially change what can be diagnosed.\n\nA tool that hard-codes one shape breaks on the next upgrade — silently, if it is\nunlucky. That is exactly how the health check this is extracted from spent weeks\nemitting a confident, wrong \"missed schedules\" warning that was really SQLSTATE\n42P01 after `pgboss.archive` disappeared.\n\nSo schema knowledge lives in one file, [`src/pgboss/dialect.ts`](src/pgboss/dialect.ts), as data:\n\n- **Every relation and column name is declared in a dialect.** Query builders\n  emit identifiers from it, so supporting a new pg-boss layout is an edit to\n  that file — no SQL elsewhere mentions a pg-boss table by name.\n- **Dialects are matched on observed shape, not on a version number.** pg-boss's\n  release→schema-version mapping is not published as a contract, and a guessed\n  mapping would reintroduce the very failure this guards against. The version\n  integer is read, reported, and used to say *\"this is newer than anything we\n  have verified\"* — but it never decides which SQL runs.\n- **Optional columns are feature-detected.** No `heartbeat_on`? Stuck-job\n  detection degrades to age-based and says so, instead of failing.\n- **Unknown layouts are refused, by name.** A pre-v10 schema is recognised\n  specifically and rejected with the reason, because diagnosing it against\n  modern queries would silently miss every archived job. A wrong diagnosis is\n  worse than a refusal.\n\n`server_info` reports the matched dialect, the schema version, whether that\nversion has been *verified* against real pg-boss, and any reduced capabilities.\n\nThis is not a theoretical concern — it has already caught a real bug. The\ndialect originally claimed a **v10** floor, on the belief that v10 removed the\n`archive` table. Booting pg-boss 10.4.2 showed the archive table still present\nand expiry still an `expire_in` interval, so the dialect was rejecting v10\noutright and matching nothing at all for it. The real floor is **v11**, and\nv10 now has its own dialect: recognised, and refused by name, because reading\nthe job table alone on v10 silently misses everything already archived.\n\nCI keeps this honest. The integration suite boots pg-boss 10, 11 and 12 into\nseparate schemas and asserts that the observed schema version appears in the\ndialect's verified list — so a future pg-boss that changes the schema fails\nloudly rather than running unverified SQL.\n\n## Read-only, by construction\n\nEvery query runs inside a `BEGIN READ ONLY` transaction with a\n`statement_timeout` and a row cap, and is always rolled back. Recovery actions\nare *recommended*, with exact commands — never executed. A confused agent cannot\npurge your queue, because the database itself refuses the write.\n\nThree independent guarantees, because the failure being guarded against is\nwriting to someone's production queue:\n\n1. `BEGIN READ ONLY` on every transaction\n2. `default_transaction_read_only=on` at connection level\n3. The docs tell you to connect as a least-privilege role — the only guarantee\n   that does not depend on this code being correct\n\nTimeouts bind as parameters via `set_config(..., is_local => true)` rather than\nbeing interpolated into SQL. The schema name — the one identifier that cannot be\na bind parameter — is validated against an identifier grammar and quoted.\n\n## Log correlation (optional)\n\nQueue state says *that* a job failed; application logs usually say *why*. Point\nthe server at a log backend and findings quote the lines behind a failure.\n\n```bash\nQUEUE_DOCTOR_AXIOM_TOKEN=xapt-...      # read-capable PAT\nQUEUE_DOCTOR_AXIOM_DATASET=app-prod\nQUEUE_DOCTOR_AXIOM_ORG_ID=your-org\nQUEUE_DOCTOR_AXIOM_QUEUE_FIELD=job     # field carrying the queue name\n```\n\nDeliberately optional, and deliberately unable to break anything: a dead log\nbackend never turns a working diagnosis into a failed one, and \"we did not\nlook\" stays distinguishable from \"we looked and found nothing\" — otherwise an\nabsent log line reads as evidence of absence. Half-configured settings are a\nstartup error rather than a silent downgrade.\n\n## Reaching a database you cannot connect to\n\nProduction queues are often the ones you most want diagnosed and least able to\nreach: Postgres bound to loopback, no port forwarding, only the application in\nfront of it exposed. Opening the database to the network so a diagnostic can\nconnect is a poor trade — the grant is permanent and far wider than the need.\n\nSo the server can run its SQL over HTTPS against a read-only SQL endpoint\ninstead:\n\n```bash\nQUEUE_DOCTOR_HTTP_SQL_URL=https://your-app.example/api/admin/sql\nQUEUE_DOCTOR_HTTP_SQL_TOKEN=...\n```\n\nSet these and no connection string is needed; set both and the HTTP transport\nwins, so an ambient `DATABASE_URL` cannot quietly become the target. The\nendpoint must accept `{\"query\": \"...\", \"params\": [...]}` and answer with\n`{\"rows\": [...], \"truncated\": bool}`. Reference implementation:\n[showbook's `/api/admin/sql`](https://github.com/ethanasm/showbook/blob/main/apps/web/app/api/admin/sql/route.ts).\n\nThe safety properties move to the far end, which is an improvement rather than\na compromise. The endpoint opens its own read-only transaction, enforces its\nown timeout and row cap, can rate-limit, can log every query, and can be backed\nby a role with narrower grants than the application's own — none of which\ndepend on this client being correct. What changes for you: the endpoint's\n`statement_timeout` and row cap win over `QUEUE_DOCTOR_STATEMENT_TIMEOUT_MS`\nand `QUEUE_DOCTOR_MAX_ROWS`, a truncating endpoint is reported as `truncated`\nrather than silently short, and one `diagnose` costs roughly a dozen requests\nagainst whatever rate limit is in force.\n\nBind parameters are required, not optional: a client forced to inline its own\nliterals to reach a read-only endpoint would be building an injection sink to\nget there.\n\n## Configuration\n\n| Variable | Default | Purpose |\n|:---|:---|:---|\n| `QUEUE_DOCTOR_DATABASE_URL` / `DATABASE_URL` | — | **Required**, unless the HTTP transport is used. Postgres connection string |\n| `QUEUE_DOCTOR_HTTP_SQL_URL` | — | Read-only SQL endpoint to query through instead of connecting |\n| `QUEUE_DOCTOR_HTTP_SQL_TOKEN` | — | Bearer token for that endpoint |\n| `QUEUE_DOCTOR_BACKEND` | `pgboss` | `pgboss` or `graphile` |\n| `QUEUE_DOCTOR_SCHEMA` | per backend | Schema the queue was installed into |\n| `QUEUE_DOCTOR_STATEMENT_TIMEOUT_MS` | `5000` | Per-query timeout (100–120000) |\n| `QUEUE_DOCTOR_MAX_ROWS` | `500` | Row cap per query (1–10000) |\n| `QUEUE_DOCTOR_LOG_LEVEL` | `info` | `debug`/`info`/`warn`/`error`/`silent` (stderr) |\n| `QUEUE_DOCTOR_THRESHOLDS` | — | JSON object overriding rule thresholds (see below) |\n\nSee [`.env.example`](.env.example). Requires Node.js ≥ 20.11.\n\n## Tuning the rules\n\nThe thresholds are tuned to the queue these rules were extracted from. That is\na defensible starting point and a poor universal answer: a queue that\nlegitimately fails fifty times an hour against a flaky upstream does not have a\nretry storm, and being told it does every time teaches you to stop reading.\n\nOverride any of them with a JSON object — only the keys you set change:\n\n```bash\nQUEUE_DOCTOR_THRESHOLDS='{\"stormMinFailures\":50,\"idleQueueSeconds\":2592000}'\n```\n\n| Key | Default | Governs |\n|:---|---:|:---|\n| `stormMinFailures` | `20` | Failures before a burst counts as a storm |\n| `stormDominantShare` | `0.5` | Share one error must hold to be called dominant |\n| `stormPeakPerMinute` | `5` | Failures in a minute that mark a burst, not a trickle |\n| `stormCriticalFailures` | `100` | Above this a storm is critical, not a warning |\n| `expiryProximity` | `0.95` | Fraction of expiry that looks killed rather than failed |\n| `expiryMinJobs` | `3` | Jobs at expiry before it is a pattern |\n| `heartbeatMissedMultiplier` | `3` | Missed heartbeats before a worker counts as gone |\n| `missedScheduleCriticalSeconds` | `86400` | Lateness beyond which a miss is critical |\n| `retentionMismatchMin` | `5` | Extra stale failed rows before flagging retention |\n| `duplicateTickMin` | `2` | Ticks with duplicate firings before suspecting double registration |\n| `idleQueueSeconds` | `604800` | Age at which an empty queue is worth mentioning |\n| `correlatedLogSample` | `5` | Log lines attached to a finding as evidence |\n\nAn unknown key is a **startup error**, not a warning — a typo that silently\nleaves the default in place is the failure this prevents. `server_info` reports\nthe effective values and which ones you set, so you can confirm an override\ntook.\n\n## Publishing to the MCP registry\n\n`server.json` is the registry manifest. Its version and the npm version it\npoints at are both synced by `npm version` (see `scripts/sync-version.mjs`), and\na test fails if they drift — a registry entry naming a version that is not on\nnpm sends clients to a 404, which is worse than a stale entry.\n\nOwnership is proved by the `mcpName` field in the **published** `package.json`,\nso npm must be published first:\n\n```bash\nnpm version patch          # syncs src/version.ts and server.json\nnpm publish                # the registry reads mcpName off this\nmcp-publisher login github # device auth as the io.github.<user> namespace owner\nmcp-publisher publish\n```\n\n## What this is not\n\n- **Not a queue browser.** To page through jobs, `psql` is better.\n- **Not a dashboard.** This is agent infrastructure; your MCP client is the UI.\n- **Not a Redis queue tool.** Both supported backends are Postgres-native,\n  which is what makes the read-only transaction guarantee possible at all.\n  BullMQ and Celery would need a different safety story.\n- **Not a writer.** It will not retry, cancel, or purge anything.\n\n## Roadmap\n\n- [x] Read-only database layer, schema probe, CI\n- [x] The read-only tool surface\n- [x] The diagnosis engine\n- [x] A `docker compose up` demo with a chaos worker\n- [x] Integration tests against real pg-boss 10/11/12 in CI\n- [x] Log correlation, so findings can cite application logs\n- [x] A second adapter (graphile-worker)\n- [ ] Opt-in write tools (`retry_job`, `cancel_job`) behind an explicit flag\n- [ ] Configurable rule thresholds\n\n## Development\n\n```bash\nnpm install\nnpm run verify     # lint + typecheck + test + build\n```\n\nThe unit suite drives the database layer through a scripted fake client and the\nrules through fixtures reconstructing each motivating incident, so `npm test`\nruns with no Postgres, no containers, and no network.\n\nThe integration suite boots real pg-boss (v10, v11, v12) and real\ngraphile-worker against a live Postgres:\n\n```bash\ndocker run -d -p 55432:5432 -e POSTGRES_USER=qd -e POSTGRES_PASSWORD=qd \\\n  -e POSTGRES_DB=qd postgres:16-alpine\nQUEUE_DOCTOR_TEST_DATABASE_URL=postgres://qd:qd@127.0.0.1:55432/qd \\\n  npm run test:integration\n```\n\nIt skips itself when that variable is unset, so a contributor without Postgres\nis never blocked. For a hands-on run, use [`examples/demo`](examples/demo).\n\n## License\n\nMIT\n",
  "bytes": 20400,
  "sha": "1759a036e4de98e75b9125c8ab5fd73418c845219a74c55f46d391134c4bbc21",
  "repo_slug": "ethanasm/mcp-queue-doctor",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_ethanasm_mcp_queue_doctor_fe2f656c/readme"
}