{
  "markdown": "# Agent Reliability — MCP server\n\n> Testing, benchmarking and auditing autonomous AI agents — methods, harnesses, evidence\n\n[![smithery badge](https://smithery.ai/badge/agentreliability/agentreliability)](https://smithery.ai/servers/agentreliability/agentreliability)\n\nA **remote MCP server** over a curated knowledge graph. Every claim it\nreturns is bound to a registered source: the tools hand back claims *with*\ntheir citations and a confidence value, so an agent can show its work\ninstead of asserting.\n\nNothing to install. It is a hosted streamable-HTTP endpoint:\n\n```\nhttps://agentreliability.dev/mcp\n```\n\nAlso listed on [Smithery](https://smithery.ai/servers/agentreliability/agentreliability).\n\n## Add it to a client\n\n**Claude Code**\n\n```bash\nclaude mcp add --transport http agent-reliability https://agentreliability.dev/mcp\n```\n\n**Claude Desktop / any client reading `mcpServers`**\n\n```json\n{\n  \"mcpServers\": {\n    \"agent-reliability\": {\n      \"type\": \"streamable-http\",\n      \"url\": \"https://agentreliability.dev/mcp\"\n    }\n  }\n}\n```\n\nNo API key, no account, no auth. Read-only.\n\n**Check it answers, without any client at all:**\n\n```bash\ncurl -s https://agentreliability.dev/mcp \\\n  -H 'Content-Type: application/json' \\\n  -H 'Accept: application/json, text/event-stream' \\\n  -H 'mcp-protocol-version: 2025-06-18' \\\n  -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/list\"}'\n```\n\n## Tools\n\nEight, each with an `outputSchema`, each returning `structuredContent`.\n\n| tool | arguments | what it does |\n|---|---|---|\n| `get_overview` | — | Corpus overview: what this instance knows, counts by type, published tags, freshness. **Start here** when you land and do not yet know whether this corpus can answer your question. |\n| `search` | `query`, `limit?` | Full-text search over the knowledge graph. Accent- and apostrophe-insensitive, so query in the user's own words; every hit carries its relevance score and the fields it matched. |\n| `answer` | `question` | Answer a question from the corpus. Returns the matched object's claims with sources and confidence — **never an unsourced answer**. |\n| `get_entity` | `id` | Fetch one knowledge object by id, with its claims and the sources each claim cites. |\n| `get_topic` | `tag` | List the knowledge objects carrying a tag (topics are content-backed tags). |\n| `get_related` | `id` | Graph neighbours of an object: outgoing and incoming relations, each with its relation type. |\n| `get_sources` | `object_id?` | The whole source registry, or just the sources cited by one object. Use it to judge the corpus before trusting it. |\n| `get_latest` | `limit?` | Most recently verified knowledge objects — a freshness signal. |\n\nThe intended path is `get_overview` → `search` or `answer` → `get_entity`\n→ `get_related`. `get_overview` exists because an agent that has just\narrived needs to know whether this corpus can help *before* it spends a\ncall guessing.\n\n## What is in the corpus\n\n| | |\n|---|---|\n| knowledge objects | **38** |\n| registered sources | **31** |\n| published topics | **93** |\n\n| type | objects |\n|---|---|\n| entity | 25 |\n| guide | 9 |\n| comparison | 2 |\n| faq | 1 |\n| glossary | 1 |\n\nSubject matter: evals and benchmarks (GAIA, AgentBench, Inspect), LLM-as-judge and its failure modes, Goodhart and benchmark contamination, fault injection and chaos testing, approval gates and autonomy levels, grounding and faithfulness.\n\n### Questions it is built to answer\n\n- *How do I tell a real eval from a benchmark my agent has memorised?*\n- *What does calibration mean for an LLM judge, and how is it measured?*\n- *Which failure modes does fault injection actually catch?*\n\n## What an answer actually looks like\n\nA real call against the live endpoint — `answer` with\n*\"how do I tell a real eval from benchmark contamination\"* — returns this `structuredContent`, trimmed:\n\n```json\n{\n  \"answered\": true,\n  \"entity\": {\n    \"id\": \"agent-reliability-glossary\",\n    \"name\": \"Agent reliability glossary\",\n    \"evidence_tier\": \"secondary\",\n    \"confidence\": 0.85,\n    \"last_verified\": \"2026-08-08\",\n    \"canonical_url\": \"https://agentreliability.dev/k/agent-reliability-glossary\"\n  },\n  \"claims\": [\n    {\n      \"text\": \"An eval is a structured, repeatable test that measures an LLM or LLM-based system against a defined dimension; frameworks package evals as registries of reusable templates.\",\n      \"sources\": [{ \"title\": \"openai/evals — framework for evaluating LLMs and LLM systems\" }]\n    }\n  ]\n}\n```\n\nNote what travels with the answer: the **evidence tier**, a **confidence**,\nthe date it was **last verified**, and the **source behind the claim** — not\nas prose an agent has to parse, but as fields it can act on. An agent can\ndecline to use a weak claim, or cite the primary source directly.\n\nWhen the corpus cannot answer, `answered` is `false`. It does not\nimprovise, and the miss is recorded so the gap can be filled.\n\n## Machine-readable surfaces\n\nThe MCP endpoint is one of several. The same corpus is served as plain\nfiles an agent can read directly:\n\n| surface | what it is |\n|---|---|\n| [`/llms.txt`](https://agentreliability.dev/llms.txt) | the index, as `text/plain` |\n| [`/llms-full.txt`](https://agentreliability.dev/llms-full.txt) | the whole corpus in one file |\n| [`/ai-index.json`](https://agentreliability.dev/ai-index.json) | every surface this instance publishes, with its content type |\n| [`/api/index.json`](https://agentreliability.dev/api/index.json) | one JSON document per knowledge object |\n| [`/api/sources.json`](https://agentreliability.dev/api/sources.json) | the source registry, in full |\n| [`/.well-known/mcp/server.json`](https://agentreliability.dev/.well-known/mcp/server.json) | this server's manifest |\n\nEach knowledge object has a human page and a machine twin at the same id,\nwith a canonical URL that agrees across all of them.\n\n## Behaviour worth knowing before you integrate\n\n- **`POST` only.** Every other method answers `405` with an `Allow: POST, OPTIONS` header.\n- **Rate limit:** 120 requests per minute per client, counted in a shared\n  store, published on every response as `RateLimit-Limit`,\n  `RateLimit-Remaining` and `RateLimit-Reset` (all three exposed via CORS).\n  It fails **open**: if the store is unreachable the request is served.\n- **Malformed input** gets a spec-correct JSON-RPC error — `-32700` for\n  unparseable bodies, `-32602` for an unknown tool — never an HTML error page.\n- **Request bodies are capped** and validated before transport.\n\n## Privacy\n\nNo accounts, no cookies, no ads. Usage is measured in aggregate with\ndaily-rotating hashed identifiers and a 200-day retention; raw IPs are\nnever stored. Full policy: [PRIVACY.md](./PRIVACY.md).\n\n## Provenance and licence\n\nKnowledge content is **CC-BY-4.0**: use it, cite it. The source registry is\npublic precisely so a claim can be checked rather than trusted —\n`get_sources` returns what any given claim rests on.\n\nClaims carry an evidence tier and a `last_verified` date. Where the\nevidence is weaker, the object says so rather than rounding up.\n\n## How it is built\n\nCompiled and served by Citarium, a source-available framework for turning a\nknowledge graph into a website, an API, an MCP server and agent-readable\nfiles from a single source, under external evaluation.\n\nThe framework's code is licensed under the Business Source License 1.1 and\nits repository is not public. What is public — and what actually matters for\ntrusting an answer — is this server, the corpus it serves, and the registered\nsource behind every claim: `get_sources` returns what any given claim rests\non, so it can be checked rather than trusted.\n\nThis repository is the server's public face: its manifest and its\ndocumentation. The corpus itself lives at [agentreliability.dev](https://agentreliability.dev).\n\n## Running it yourself\n\nThe endpoint above is the server. If you want to run a copy locally — or\nif a directory needs to start one in order to inspect it — the recipe is\nin [CONTAINER.md](CONTAINER.md): `fetch-corpus.mjs` downloads this\ninstance's published corpus and `@citarium/mcp` serves it. Both files are\nGENERATED by `citarium mirror`; edit the instance, not them.\n\nThe container serves a snapshot taken when its image was built. This\nendpoint is the thing itself.\n",
  "bytes": 8242,
  "sha": "fcf6cc3822f7040b5d2af2d6404caedde53fa8418358ee60ff088b4c07e52b48",
  "repo_slug": "citarium/agentreliability-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_dev_agentreliability_agent_reliability_77164220/readme"
}