{
"markdown": "# Argus\n\n\n\n[](https://www.python.org/downloads/)\n[](https://pypi.org/project/argus-search/)\n[](https://pepy.tech/projects/argus-search)\n[](LICENSE)\n[](https://github.com/Khamel83/argus/actions/workflows/ci.yml)\n[](https://registry.modelcontextprotocol.io/servers/io.github.Khamel83/argus)\n[](https://github.com/Khamel83/argus/pkgs/container/argus)\n\nRetrieval platform for AI agents. Argus routes search across 14 providers, recovers dead URLs, captures important site content, builds local docs-plus-research packs, and persists everything with traceable local artifacts.\n\n**Features at a glance:**\n\n- **Topology-aware acquisition** — Argus knows if it's on a residential IP or datacenter, routing search and extraction automatically to avoid blocks and minimize network hops.\n- **14 providers, one API** — free-first tier routing, budget-exhausted providers skipped automatically\n- **Zero-key start** — `pip install argus-search` gives you DuckDuckGo + Yahoo immediately, no accounts needed\n- **SearXNG self-host = 70+ engines** — Google, Bing, Yahoo, Startpage, Ecosia, Qwant and more via one Docker container\n- **12-step content extraction** — returns full page text with quality gates, not just links\n- **Opinionated retrieval workflows** — recover dead articles, capture important pages from a site, and build local docs-plus-research packs\n- **Argus-owned corpus storage** — runtime data goes to a writable user data directory, not your repo checkout\n- **Multi-turn sessions** — pass `session_id` for conversational context across searches\n- **Score attribution** — optionally show which providers contributed to each fused RRF score\n- **Usage dashboard** — inspect provider budgets, recent query volume, and machine-level usage at `/dashboard`\n- **4 search modes** — discovery, research, recovery, grounding\n- **Dead URL recovery** — `/recover-url` with Wayback Machine and archive fallbacks\n- **4 integration paths** — HTTP API, CLI, MCP server, Python SDK\n\n_Built for AI agent builders, RAG pipelines, and ops teams who need reliable search, capture, and local evidence without stitching APIs together._\n\n> Status: beta. The retrieval workflows and corpus model are production-oriented, but still maturing.\n\n> Status: see the public [status page](docs/STATUS.md). Authorized maintainers\n> can use the private [argus-ops README](https://github.com/Khamel83/argus-ops/blob/main/README.md) for the latest dated reports.\n\n> **Current production checkpoint — September 7, 2026:** The authenticated\n> HTTP API and MCP adapter are operational, and sampled free search works\n> through SearXNG, Yahoo, and GitHub. DuckDuckGo is intermittent and fails\n> closed after acquisition-policy blocks. Production readiness is\n> `ready=true, degraded`; paid providers are not currently admitted until\n> their non-secret credential-version and account-scope bindings are recorded.\n> Browser capability and recovery metadata evidence remain open. See the\n> [current status matrix](docs/STATUS.md) before relying on a provider.\n\n## Contents\n\n- [Quickstart](#quickstart)\n- [Development](#development)\n- [Where Argus Writes Data](#where-argus-writes-data)\n- [Opinionated Workflows](#opinionated-workflows)\n- [Providers](#providers)\n- [HTTP API](#http-api)\n- [Dashboard](#dashboard)\n- [Integration](#integration)\n - [CLI](#cli)\n - [MCP](#mcp)\n - [Python](#python)\n- [Content Extraction](#content-extraction)\n- [Architecture](#architecture)\n- [Configuration](#configuration)\n- [When Not To Use Argus](#when-not-to-use-argus)\n- [FAQ](#faq)\n\n## Quickstart\n\n### Mode 1: Local CLI (zero config)\n\n```bash\npip install argus-search && argus search -q \"python web frameworks\"\n```\n\nThat's it. DuckDuckGo handles the search — no accounts, no keys, no containers. You get unlimited free search from your laptop right now. Add API keys whenever you want more providers, or don't.\n\n```bash\nargus extract -u \"https://example.com/article\" # extract clean text from any URL\nargus recover-article -u \"https://example.com/dead-post\"\nargus capture-site -u \"https://docs.example.com\"\nargus build-research-pack -t \"example sdk\" --official-url \"https://docs.example.com\"\n```\n\nWorks on any machine with Python 3.11+ — laptop, Mac Mini, Raspberry Pi, cloud VM. Nothing to host.\n\n**For MCP (Claude Code, Codex, OpenCode, Cursor, VS Code):**\n\n```bash\npipx install argus-search[mcp]\nexport ARGUS_MCP_STANDALONE=true # explicit development-only local broker\nargus mcp init --global --client all\n```\n\nThat writes native config for Claude Code, Codex CLI, OpenCode, and Cursor. Restart the client after configuration. For manual stdio setup:\n\n```json\n{\"mcpServers\": {\"argus\": {\"command\": \"argus\", \"args\": [\"mcp\", \"serve\"], \"env\": {\"ARGUS_MCP_STANDALONE\": \"true\"}}}}\n```\n\nOr install from the [MCP Registry](https://registry.modelcontextprotocol.io/servers/io.github.Khamel83/argus):\n\n```json\n{\n \"mcpServers\": {\n \"argus\": {\n \"registryType\": \"pypi\",\n \"identifier\": \"argus-search\",\n \"runtimeHint\": \"uvx\",\n \"env\": {\"ARGUS_MCP_STANDALONE\": \"true\"}\n }\n }\n}\n```\n\nStandalone development needs no server or keys, but it must be explicitly\nenabled. Production MCP always delegates to an authenticated HTTP authority.\n\nSee [MCP Client Setup](docs/mcp-clients.md) for exact config files, verification commands, remote HTTP setup, and troubleshooting.\n\n### Mode 2: Full Stack Server\n\nGot a Raspberry Pi running Pi-hole? A Mac Mini on your desk? An old laptop? That's enough to run the full stack — SearXNG (your own private search engine, disabled by default) plus local JS-rendering content extraction.\n\n```bash\n# Optional: tell Argus it has residential egress to optimize routing\nexport ARGUS_EGRESS_TYPE=residential\nARGUS_SEARXNG_ENABLED=true docker compose up -d # SearXNG + Argus\n```\n\n| What you have | What you get |\n|--------------|-------------|\n| **Any machine with Python 3.11+** | DuckDuckGo + API providers (no server) |\n| **Home server / old laptop** (4GB+) | Everything — SearXNG, all providers, Crawl4AI, Obscura |\n| **Mac Mini M1+** (8GB+) | Full stack with headroom |\n| **Free cloud VM** (1GB) | SearXNG + search providers (use residential workers for extraction) |\n\nSearXNG takes 512MB of RAM and gives you a private Google-style search engine (disabled by default — set `ARGUS_SEARXNG_ENABLED=true`) that nobody can rate-limit, block, or charge for. It runs alongside Pi-hole on hardware millions of people already own.\n\n## Where Argus Writes Data\n\nArgus code and Argus runtime data are different things.\n\n- **Code** lives wherever you install or clone Argus.\n- **Runtime corpus data** lives in a writable user data directory resolved by `platformdirs`, or in `ARGUS_DATA_ROOT` if you override it.\n\nInspect the exact paths on your machine:\n\n```bash\nargus paths\n```\n\nBy default Argus writes:\n\n- official docs cache under the resolved `docs/cache/`\n- research packs under `docs/research/`\n- workflow run state under `workflows/runs/`\n- versioned workflow snapshots under `snapshots/`\n\nThis means Argus does **not** require a sibling `../docs-cache` checkout. If you have an older `docs-cache` tree, import it once with:\n\n```bash\nargus corpus import-docs-cache -s /path/to/docs-cache\n```\n\n## Opinionated Workflows\n\nThese workflows build local artifacts, not just transient JSON responses.\n\n### Recover A Dead Article\n\n```bash\nargus recover-article -u \"https://example.com/old-post\" -t \"Example Post\"\n```\n\nArgus searches for recovery candidates, extracts the best result, saves the recovered sources locally, and writes a citation-backed report plus manifest.\n\n### Capture The Important Parts Of A Site\n\n```bash\nargus capture-site -u \"https://docs.example.com\"\n```\n\nArgus stays on-domain, uses sitemap-assisted discovery plus heuristic link scoring, saves the important pages it finds, and writes a detailed summary with references.\n\n### Build A Docs + Research Pack\n\n```bash\nargus build-research-pack -t \"example sdk\"\nargus build-research-pack -t \"example sdk\" --official-url \"https://docs.example.com\"\n```\n\nArgus captures official docs into its local docs cache, adds non-official supporting sources from search, and writes a combined research pack with traceable artifacts.\n\n## Development\n\nRepo development is pinned to Python 3.12. The package runtime floor is Python\n3.11, the production image runs Python 3.12.3, and Python 3.13 is the\ncompatibility CI lane. Required CI passes all three; contributors should use\nthe `uv` workflow below so local verification matches the canonical lane and\ndoes not accidentally use an older system interpreter.\n\n```bash\nuv sync --python 3.12 --extra dev --extra mcp\nuv run pytest tests/ -v --tb=short\n```\n\nThe repo includes `.python-version` with `3.12` so `uv`, `pyenv`, and similar tools pick the right interpreter by default. More contributor guidance lives in [CONTRIBUTING.md](CONTRIBUTING.md).\n\n## Providers\n\n| Provider | Credit type | Free capacity | Setup |\n|----------|------------|---------------|-------|\n| DuckDuckGo | Free (scraped) | Unlimited | None |\n| Yahoo | Free (scraped) | Unlimited | None — fragile, auto-skipped if broken |\n| SearXNG | Free (self-hosted, off by default) | Unlimited — 70+ engines¹ | Docker |\n| GitHub | Free (API) | Unlimited | None (token for higher rate limit) |\n| WolframAlpha | Free (API key) | 2,000 queries/month | [free key](https://developer.wolframalpha.com/) |\n| Brave Search | Monthly recurring | 2,000 queries/month | [dashboard](https://brave.com/search/api/) |\n| Tavily | Monthly recurring | 1,000 queries/month | [signup](https://app.tavily.com/sign-up) |\n| Exa | Monthly recurring | 1,000 queries/month | [signup](https://dashboard.exa.ai/signup) |\n| Linkup | Monthly recurring | 1,000 queries/month | [signup](https://linkup.so) |\n| Parallel AI | Monthly recurring | $5 credit with card on file, up to 5,000 searches/month | [signup](https://parallel.ai) |\n| Serper | One-time signup | 2,500 credits | [signup](https://serper.dev/signup) |\n| You.com | One-time signup | $20 credit | [platform](https://you.com/platform) |\n| Valyu | One-time signup | $10 credit | [platform](https://platform.valyu.ai) |\n\n¹ SearXNG aggregates Google, Bing, Yahoo, Startpage, Ecosia, Qwant, Wikipedia, and 60+ more — all behind a single self-hosted endpoint. Run `docker compose up -d` on any machine with 512MB of free RAM.\n\n² WolframAlpha returns **computed answers** (math, unit conversions, factual lookups), not web search results. It only activates in `grounding` and `research` modes. Queries it can't compute (general web searches) return empty — no error, no health penalty.\n\n**7,000+ free queries/month** from recurring free-tier providers with API keys (WolframAlpha 2k + Brave 2k + Tavily 1k + Exa 1k + Linkup 1k), or **up to 12,000+** when Parallel's monthly credit is available to an eligible account with a card on file. DuckDuckGo, Yahoo, and GitHub have no monthly cap. SearXNG is disabled by default (enable in `.env`). Routing priority: **Tier 0** (free: SearXNG*, DuckDuckGo, Yahoo, GitHub, WolframAlpha) → **Tier 1** (monthly recurring: Brave, Tavily, Exa, Linkup, Parallel) → **Tier 3** (one-time: Serper, You.com, Valyu, SearchAPI). Budget-exhausted providers are skipped automatically.\n\nThese are package-level provider tiers and advertised quotas, not proof of\ncurrent production availability. The production authority fails closed for a\ncredentialed provider until its registration fingerprint, account scope,\nbudget, and approved test evidence are present. A protected key value alone\ndoes not prove that the current key works; see [docs/STATUS.md](docs/STATUS.md)\nfor the live classification.\n\n## HTTP API\n\nAll endpoints prefixed with `/api`. OpenAPI docs at `http://localhost:8000/docs`.\n\nLocal loopback calls can use the API without auth. Remote HTTP callers must send `ARGUS_API_KEY` as either `Authorization: Bearer ...` or `X-API-Key: ...`. Privileged routes under `/api/admin/*` require `ARGUS_ADMIN_API_KEY` (or fall back to `ARGUS_API_KEY` if no separate admin key is configured).\n\n```bash\n# Search\ncurl -X POST http://localhost:8000/api/search \\\n -H \"Content-Type: application/json\" \\\n -d '{\"query\": \"python web frameworks\", \"mode\": \"discovery\", \"max_results\": 5}'\n\n# Search with score attribution\ncurl -X POST http://localhost:8000/api/search \\\n -H \"Content-Type: application/json\" \\\n -d '{\"query\": \"python web frameworks\", \"include_attribution\": true}'\n\n# Multi-turn search (conversational refinement)\ncurl -X POST http://localhost:8000/api/search \\\n -H \"Content-Type: application/json\" \\\n -d '{\"query\": \"what about async?\", \"session_id\": \"my-session\"}'\n\n# Extract content from a working URL\ncurl -X POST http://localhost:8000/api/extract \\\n -H \"Content-Type: application/json\" \\\n -d '{\"url\": \"https://example.com/article\"}'\n\n# Recover a dead or moved URL\ncurl -X POST http://localhost:8000/api/recover-url \\\n -H \"Content-Type: application/json\" \\\n -d '{\"url\": \"https://example.com/old-page\", \"title\": \"Example Article\"}'\n\n# Network-free process liveness (container health target)\ncurl http://localhost:8000/api/live\n\n# Public minimal startup and cached readiness\ncurl http://localhost:8000/api/startup\ncurl http://localhost:8000/api/ready\n\n# Authenticated operator status, health compatibility, and budgets\ncurl -H \"Authorization: Bearer $ARGUS_ADMIN_API_KEY\" \\\n http://localhost:8000/api/admin/status\ncurl -H \"Authorization: Bearer $ARGUS_ADMIN_API_KEY\" \\\n http://localhost:8000/api/admin/budgets\ncurl -H \"Authorization: Bearer $ARGUS_ADMIN_API_KEY\" \\\n http://localhost:8000/api/admin/maya-outbox/status\ncurl -H \"Authorization: Bearer $ARGUS_ADMIN_API_KEY\" \\\n http://localhost:8000/api/admin/maya-outbox/dead-letters\n# After correcting the cause of a permanent rejection:\ncurl -X POST -H \"Authorization: Bearer $ARGUS_ADMIN_API_KEY\" \\\n http://localhost:8000/api/admin/maya-outbox/DELIVERY_ID/recover\n```\n\n`/api/health` remains a 200 liveness compatibility route. It intentionally\ndoes not check PostgreSQL, providers, Maya, or the browser, so a dependency\noutage cannot cause container restart storms. See\n[production operations](docs/operations.md) for the canonical topology and\noperator procedures, and [operational status](docs/operations-status.md) for endpoint semantics,\nreadiness classification, observation expiry, and safe telemetry.\n\n#### Search modes\n\n| Mode | Use for | Example |\n|------|---------|---------|\n| `discovery` | Related pages, canonical sources | \"Find the official docs for X\" |\n| `research` | Broad exploratory retrieval | \"Latest approaches to Y?\" |\n| `recovery` | Finding moved/dead content | \"This URL is 404\" |\n| `grounding` | Fact-checking with live sources | \"Verify this claim about Z\" |\n\nTier-based routing always applies first. Within each tier, the mode selects provider order.\n\n#### Response format\n\n```json\n{\n \"query\": \"python web frameworks\",\n \"mode\": \"discovery\",\n \"results\": [\n {\n \"url\": \"https://fastapi.tiangolo.com\",\n \"title\": \"FastAPI\",\n \"snippet\": \"Modern Python web framework\",\n \"provider\": \"duckduckgo\",\n \"score\": 0.0164,\n \"score_attribution\": {\"duckduckgo\": 0.0164},\n \"egress\": \"unknown\",\n \"machine\": null\n }\n ],\n \"total_results\": 1,\n \"cached\": false,\n \"traces\": [\n {\"provider\": \"duckduckgo\", \"status\": \"success\", \"results_count\": 5, \"latency_ms\": 312}\n ]\n}\n```\n\nEach result includes `url`, `title`, `snippet`, `domain`, `provider`, and `score`. The `traces` array shows which providers were called and their outcomes.\n\nWhen `include_attribution` is true, each result also includes\n`score_attribution`: a provider-to-score map that decomposes the result's\nReciprocal Rank Fusion score. RRF is additive, so each provider's attribution is\nexactly its own rank contribution, and the values sum to `score`. Attribution is\noff by default and cached separately from non-attributed searches.\n\n#### Budgets\n\n```json\n{\n \"budgets\": {\n \"brave\": {\"remaining\": 1847, \"monthly_usage\": 153, \"usage_count\": 153, \"exhausted\": false},\n \"duckduckgo\": {\"remaining\": 0, \"monthly_usage\": 0, \"usage_count\": 42, \"exhausted\": false}\n },\n \"token_balances\": {\"jina\": 9833638}\n}\n```\n\nEach provider tracks usage. Tier 1 (monthly) uses a 30-day rolling window; tier 3 (one-time) uses a lifetime counter that never resets. When a provider hits its budget, Argus skips it and moves to the next. Free providers (DuckDuckGo, GitHub) have no limit. SearXNG is free but disabled by default. Set `ARGUS_*_MONTHLY_BUDGET_USD` to enforce custom limits per provider.\n\n## Dashboard\n\nRun the HTTP server and open `/dashboard`:\n\n```bash\nargus serve\n# http://127.0.0.1:8000/dashboard\n```\n\nThe dashboard shows provider budget burn, over-pace and exhausted providers,\nquery volume for the last 30 days, usage by machine, and recent provider\nactivity. Budget cards refresh automatically.\n\nSet `ARGUS_ADMIN_API_KEY` to require dashboard login. If no admin key is set,\nthe dashboard is open to anyone who can reach the server, which is suitable only\nfor trusted local use.\n\nFor subpath deployment behind a reverse proxy, set `ARGUS_ROOT_PATH` to the\nexternal path prefix:\n\n```bash\nARGUS_ROOT_PATH=/argus argus serve\n```\n\nThat makes dashboard redirects, links, and HTMX fragment URLs work when the\nproxy serves Argus at a path such as `https://khamel.com/argus/`.\n\nFor direct public HTTPS, the repo includes a Caddy profile:\n\n```bash\nARGUS_DOMAIN=argus.example.com ACME_EMAIL=you@example.com \\\n docker compose --profile proxy up -d\n```\n\nFor an existing Authentik/nginx deployment, keep authentication at the proxy\nlayer and set `ARGUS_ROOT_PATH` to the public prefix.\n\n## Integration\n\n### CLI\n\n```bash\nargus search -q \"python web framework\" # zero-config, uses DuckDuckGo\nargus search -q \"python web framework\" --mode research -n 20\nargus search -q \"python web framework\" --free # free providers only (no paid API calls)\nargus search -q \"python web framework\" --attribution # show per-provider score attribution\nargus search -q \"fastapi\" --session my-session # multi-turn context\nargus extract -u \"https://example.com/article\" # extract clean text\nargus extract -u \"https://example.com/article\" -d nytimes.com # auth extraction\nargus recover-url -u \"https://dead.link\" -t \"Title\"\nargus doctor # full setup diagnostics\nargus health # provider status\nargus budgets # budget + token balances\nargus mcp check # validate MCP setup\nargus set-balance -s jina -b 9833638 # track token balance\nargus test-provider -p brave # smoke-test a provider\nargus serve # start API server\nargus mcp serve # start MCP server\nargus mcp init # add MCP config to project\n```\n\nAll commands support `--json` for structured output.\n\n\nHow sessions work
\n\nPass `session_id` to any search call. Argus stores each query and extracted URL through the same SQLAlchemy repository used by the retrieval ledger (`ARGUS_DB_URL`, PostgreSQL in production and SQLite for direct local use). Reusing the same `session_id` gives the broker context from prior queries — follow-up searches are automatically refined using earlier conversation context. Sessions persist across restarts. Omit `session_id` for stateless, one-shot searches.\n\nLegacy sessions from the former budget SQLite database can be reconciled\nwithout mutating the target first:\n\n```bash\nargus ledger reconcile-sessions \\\n --source sqlite:///argus_budgets.db \\\n --target \"$ARGUS_DB_URL\"\n# Review source/imported/skipped/conflicting, then repeat with --apply.\n```\n\nThe import is idempotent: an identical existing session is skipped and a\ndifferent session with the same ID is reported as conflicting.\n\n \n\n### MCP\n\nMCP is a stateless execution adapter over the authenticated HTTP API. It does\nnot construct providers or a broker and does not own browser, database,\nbudget, session, health, or outbox state. Configure the adapter process with:\n\n```bash\nexport ARGUS_AUTHORITY_URL=http://argus-api:8000\nexport ARGUS_AUTHORITY_TOKEN=replace-with-a-scoped-caller-token\n```\n\nThe deployed production endpoint supports both the verified MCP `2025-11-25`\ncompatibility contract and the MCP `2026-07-28` stateless transport revision.\nThe newer path is one-shot and does not require an initialize handshake or\n`Mcp-Session-Id`; durable policy, budgets, sessions, and evidence remain owned\nby the HTTP authority. See\n[`docs/research/2026-08-11-mcp-stateless-production-authority.md`](docs/research/2026-08-11-mcp-stateless-production-authority.md)\nfor the source-backed boundary and required no-spend probes.\n\n**Option A — Local adapter (stdio)**\n\nInstall the adapter on the same machine as your MCP client:\n\n```json\n{\n \"mcpServers\": {\n \"argus\": {\n \"command\": \"argus\",\n \"args\": [\"mcp\", \"serve\"]\n }\n }\n}\n```\n\nUse the full path if `argus` isn't on PATH: `\"/home/you/.local/bin/argus\"`.\nThe adapter inherits `ARGUS_AUTHORITY_URL` and `ARGUS_AUTHORITY_TOKEN` from\nthe client process. To run a local broker instead, development environments\nmust explicitly set `ARGUS_MCP_STANDALONE=true`; production rejects it.\n\nWorks with **Claude Code**, **Codex CLI**, **OpenCode**, **Cursor**, and any stdio-based MCP client. Use `argus mcp init --global --client all` to write native client configs for the current machine.\n\nDetailed client setup and verification commands live in [docs/mcp-clients.md](docs/mcp-clients.md).\n\n**Option B — Remote MCP adapter (clients over Tailscale)**\n\nRun Argus on one machine, connect every client over the network. No local install on clients.\n\nOn the adapter host:\n```bash\nexport ARGUS_API_KEY=replace-with-a-long-random-secret\nexport ARGUS_AUTHORITY_URL=http://argus-api:8000\nexport ARGUS_AUTHORITY_TOKEN=\"$ARGUS_API_KEY\"\nargus mcp serve --transport streamable-http --host YOUR_TAILSCALE_IP --port 8001\n```\n\nRemote MCP credentials must also be valid scoped credentials at the HTTP\nauthority because the adapter forwards each authenticated bearer token\nunchanged. For stdio, `ARGUS_AUTHORITY_TOKEN` is the caller credential.\n\nTo keep the HTTP API and remote MCP service running after reboot on a systemd host:\n```bash\ncat >mcp.env <<'EOF'\nARGUS_AUTHORITY_URL=http://argus-api:8000\nARGUS_AUTHORITY_TOKEN=replace-with-scoped-caller-token\nARGUS_API_KEY=replace-with-the-same-scoped-caller-token\nEOF\nchmod 600 mcp.env\nARGUS_MCP_ENV_FILE=\"$PWD/mcp.env\" scripts/install-systemd.sh\nsystemctl status argus argus-mcp --no-pager\n```\n\nThe installer validates the minimal adapter environment, installs it as\nroot-only `/etc/argus/mcp.env`, then installs and starts both units. The MCP\nunit never loads the authority's `.env`, provider vaults, database settings,\nbrowser paths, or writable data volumes.\n\nOn each client:\n\n| Client | Config |\n|--------|--------|\n| **Claude Code** | `{\"mcpServers\":{\"argus\":{\"type\":\"http\",\"url\":\"http://:/mcp\",\"headers\":{\"Authorization\":\"Bearer \"}}}}` in `~/.claude.json` (global) or `.mcp.json` (project) |\n| **OpenCode** | `{\"mcp\":{\"argus\":{\"type\":\"remote\",\"url\":\"http://:/mcp\",\"enabled\":true,\"headers\":{\"Authorization\":\"Bearer \"}}}}` in `~/.config/opencode/config.json` (global) or `.opencode/opencode.json` (project) |\n| **Cursor** | Same as Claude Code — reads `.mcp.json` |\n| **Codex CLI** | `[mcp_servers.argus]` section in `~/.codex/config.toml` with `url` and `bearer_token_env_var = \"ARGUS_API_KEY\"` — export that variable in the shell that launches Codex; `argus mcp init` never writes the token to disk |\n| **Gemini CLI** | `gemini mcp add argus http://:/mcp -t http -H \"Authorization: Bearer \"` |\n| **Antigravity** | `{\"mcpServers\":{\"argus\":{\"serverUrl\":\"http://:/mcp\",\"headers\":{\"Authorization\":\"Bearer \"}}}}` |\n\nWith [Tailscale](https://tailscale.com), `` is your machine's Tailscale IP (e.g. `100.x.x.x`). One server, every machine on your mesh gets search.\n\n**One-command provisioning:**\n\n```bash\n# Load secrets, then push config to any machine:\neval $(secrets decrypt argus | grep -E 'ARGUS_REMOTE_URL|ARGUS_API_KEY' | sed 's/^/export /')\n\ncurl -s https://raw.githubusercontent.com/Khamel83/argus/main/scripts/provision-mcp-client.sh | bash -s local # this machine; uses local stdio if argus is installed\ncurl -s https://raw.githubusercontent.com/Khamel83/argus/main/scripts/provision-mcp-client.sh | bash -s user@100.x.x.x # remote machine\n```\n\nThe script writes Claude/Cursor, Codex, and OpenCode configs on the target.\nLocal stdio does not require an MCP listener key, but the adapter still\nrequires its scoped `ARGUS_AUTHORITY_TOKEN`. Remote MCP mode requires\n`ARGUS_REMOTE_URL` and `ARGUS_API_KEY`. Requires Python 3.\n\n`argus mcp init` also generates configs automatically:\n```bash\nargus mcp init --global # local stdio adapter for Claude Code + OpenCode + Cursor\nargus mcp init --client codex # local stdio for Codex (writes ~/.codex/config.toml)\nargus mcp init --client opencode # local stdio for OpenCode\nARGUS_REMOTE_URL=http://argus.local:8271 ARGUS_API_KEY=... argus mcp init --global --client all\nargus mcp init --client gemini # prints gemini mcp add command\nargus mcp init --global --client all # everything above\n```\n\n### Release Status\n\nPushing `main` does not publish PyPI. Package and MCP Registry publication happens through the GitHub publish workflow on release creation or manual dispatch. See [docs/releasing.md](docs/releasing.md) for version sync, preflight checks, and publish verification.\n\n**Transports**: `stdio` (default local adapter), `sse` (legacy remote), and\n`streamable-http` (modern remote, `\"type\":\"http\"` in config). Remote MCP\ntransports require `ARGUS_API_KEY`; every transport delegates execution to\n`ARGUS_AUTHORITY_URL`.\n\nAvailable tools:\n- HTTP-backed stdio and remote MCP: `search_web`, `extract_content`,\n `recover_url`, `expand_links`, `search_health`, `search_budgets`,\n `recover_dead_article`, `capture_site`, and `build_research_pack`\n- Explicit standalone development additionally exposes local-only\n `test_provider`, `cookie_health`, `valyu_answer`, path/file tools, and\n resources. These are intentionally absent from production adapters.\n\n`search_web` accepts `free_only=true` to restrict results to free (tier-0) providers only, and `include_attribution=true` to include per-provider score attribution in the Markdown response.\n\n\n### Using Argus from MCP vs HTTP\n\nTwo transports, one rule: **agents use MCP, everything else uses HTTP.**\n\n- **MCP** (`argus mcp serve`) — a stateless authenticated adapter for AI\n harnesses that speak MCP natively. Core tools delegate to the HTTP authority:\n `search_web`, `extract_content`, `recover_url`, `expand_links`, and workflow\n starts. MCP restarts cannot fork accounting, sessions, health, or outbox state.\n- **HTTP** (`POST /api/search`, `POST /api/extract`,\n `POST /api/workflows/...`) — for scripts, cron jobs, and service\n integrations (including Maya). Send a scoped caller credential in\n `Authorization`; body `\"caller\"` values are diagnostic labels and cannot\n override the authenticated identity.\n\nThe cross-service transport and role contract for the wider fleet\n(Maya / Hermes / Argus) is canonical in\n[Maya's architecture documentation](https://github.com/Khamel83/maya/blob/main/docs/ARCHITECTURE.md).\n### Python\n\nDirect broker and extraction imports are a standalone development convenience.\nProduction Python callers use the authenticated HTTP API so all execution and\ndurable accounting remain in one authority.\n\n```python\nfrom argus.broker.router import create_broker\nfrom argus.models import SearchQuery, SearchMode\nfrom argus.extraction import extract_url\n\nbroker = create_broker()\n\nresponse = await broker.search(\n SearchQuery(query=\"python web frameworks\", mode=SearchMode.DISCOVERY, max_results=10),\n compute_attribution=True,\n)\nfor r in response.results:\n print(f\"{r.title}: {r.url} (score: {r.score:.3f})\")\n print(r.score_attribution)\n\ncontent = await extract_url(response.results[0].url)\nprint(content.title)\nprint(content.text)\n```\n\n## Content Extraction\n\nArgus tries up to twelve methods to extract content from any URL: auth extraction for paywalls, then local extractors (trafilatura, Crawl4AI, Obscura, Playwright, residential IP), then external APIs (Jina, Valyu Contents, Firecrawl, You.com, Wayback, archive.is). Each attempt is quality-checked for completeness and garbage output. See [docs/providers.md](docs/providers.md) for the full extractor comparison.\n\n**Completeness assessment** runs automatically after every successful extraction. Argus scores five signals — trailing ellipsis, feed truncation markers (\"Read more\", WordPress RSS footers), mid-sentence endings, abrupt final paragraphs, and suspicious round word counts — and returns `is_complete`, `completeness_confidence`, and `truncation_type` alongside the text. When confidence is ≥ 85%, Argus continues trying the next extractor rather than returning a partial result; this means a trafilatura fetch that ends with \"...\" will automatically fall through to Playwright, Jina, Wayback, etc. Callers that already have text (e.g. RSS feed items) can use `POST /api/assess-content` to check completeness without triggering extraction.\n\n**Obscura** (optional) is a lightweight Rust headless browser (~70MB binary, 30MB RAM) with built-in stealth mode — it sets `navigator.webdriver=undefined`, randomizes canvas/GPU/audio fingerprints per session, and blocks 3,520 tracker domains. This directly addresses bot detection on JS-heavy and anti-scraping sites that block standard Playwright/Chrome. No API key, no rate limit — fully local.\n\nTwo ways to use it:\n\n| Mode | Setup | What you get |\n|------|-------|-------------|\n| **CLI extraction step** | Install binary on `$PATH` | Argus auto-detects it; stealth browser as fallback step before Playwright |\n| **CDP backend for Playwright** | Run `obscura serve --stealth --port 9222`, set `ARGUS_OBSCURA_CDP_URL=ws://127.0.0.1:9222` | Playwright uses Obscura as its browser engine — stealth + 30MB vs 200MB + DOM-to-Markdown output |\n\nInstall the binary: [github.com/h4ckf0r0day/obscura/releases](https://github.com/h4ckf0r0day/obscura/releases)\n\n**Extract** gets the full text of a working URL and tells you whether that text is complete. **Recover-URL** finds alternatives when a URL is dead, paywalled, or radically changed.\n\n## Architecture\n\n```\nCaller (CLI/HTTP/MCP/Python) → SearchBroker → tier-sorted providers → RRF ranking → response\n ↕ SessionStore (optional)\n Extractor (on demand) → 12-step fallback chain with quality gates\n```\n\n| Module | Responsibility |\n|--------|---------------|\n| `argus/broker/` | Tier-based routing, ranking, dedup, caching, health, budgets |\n| `argus/providers/` | Provider adapters (one per search API) |\n| `argus/extraction/` | 12-step URL extraction fallback chain with quality gates |\n| `argus/sessions/` | Multi-turn session store and query refinement |\n| `argus/api/` | Authenticated production execution authority |\n| `argus/cli/` | HTTP caller in production; direct execution in development |\n| `argus/mcp/` | Stateless MCP-to-HTTP adapter |\n| `argus/persistence/` | Shared PostgreSQL authority state; SQLite standalone development |\n| `argus/operations/` | Cached readiness, typed dependency observations, process identity, bounded metrics |\n\nAdd new providers or extractors with a single adapter file. See [CONTRIBUTING.md](CONTRIBUTING.md) for the interface.\n\n### How a Query Works\n\n```\nquery arrives → cache? → build provider queue → execute sequentially → RRF fuse → dedup → respond\n```\n\n1. **Cache check.** `SearchCache` hashes the normalized query, mode, and whether attribution was requested (SHA256). Hit returns immediately with a TTL of 168 hours (7 days).\n\n2. **Provider queue.** `resolve_routing()` takes the mode-specific preference list and stable-sorts by tier: tier 0 (free) first, tier 1 (monthly) next, tier 3 (one-time) last. Example for discovery mode:\n ```\n searxng → duckduckgo → yahoo → github → brave → exa → tavily → linkup → parallel → serper → you → valyu\n ```\n\n3. **Sequential execution with gates.** Each provider is checked in order. Four gates must pass before an API call:\n - **Config** — is the provider enabled and configured (API key present)?\n - **Health** — has it failed 5+ consecutive times (triggers 60-minute cooldown)?\n - **Budget** — for tier 1+: is the budget exhausted? For tier 1 (monthly), pacing checks if the 7-day usage rate would drain the remaining budget in under a week — empty days bank headroom. For tier 3 (one-time), a lifetime counter gates access — exhaustion is the sole check.\n - **Execute** — the actual HTTP call. Successes reset failure counters; failures increment them.\n\n4. **RRF fusion.** Results from all queried providers are merged using Reciprocal Rank Fusion (`k=60`). Each result's score is the sum of `1/(k + rank)` across every provider that returned it. Results appearing in multiple providers rank higher.\n\n5. **Dedup and truncate.** URLs are normalized (stripped `www.`, tracking params like `utm_*`, trailing slashes) and deduplicated. The merged list is truncated to `max_results` (default 10).\n\n6. **Cache and persist.** The authority writes the final response to its\n in-memory cache and configured SQL repository (PostgreSQL in production,\n SQLite for standalone development). Search results and extractions include\n provenance metadata (`egress`, `machine`, `source_type`) for downstream\n audit. Existing databases are upgraded additively at startup.\n\n## Configuration\n\nAll config via environment variables. See `.env.example` for the full list. Limited API-key providers are opt-in: set both the API key and `ARGUS__ENABLED=true`. Missing keys degrade gracefully — providers are skipped, not errors.\n\nWhen running from the repo, Argus now auto-loads `.env` and `.env.local` (without overriding already-exported environment variables). Disable this behavior with `ARGUS_AUTOLOAD_DOTENV=false`.\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `ARGUS_NODE_ROLE` | `primary` | Production authority is `primary`; adapters are `caller`; direct/worker execution is development-only |\n| `ARGUS_AUTHORITY_URL` | — | HTTP API base URL required by production CLI and MCP adapters |\n| `ARGUS_AUTHORITY_TOKEN` | — | Scoped caller token for production CLI and MCP adapters |\n| `ARGUS_MCP_STANDALONE` | `false` | Explicit development-only local MCP execution |\n| `ARGUS_EGRESS_TYPE` | `unknown` | `residential`, `datacenter`, or `unknown` |\n| `ARGUS_RESIDENTIAL_POLICY` | `fallback` | `off`, `fallback`, `prefer_on_datacenter`, `prefer_for_domains`, or `always` |\n| `ARGUS_SEARXNG_ENABLED` | `false` | Set `true` when you have a SearXNG Docker container |\n| `ARGUS_SEARXNG_BASE_URL` | `http://127.0.0.1:8080` | SearXNG endpoint |\n| `ARGUS_SEARXNG_RESIDENTIAL_BASE_URL` | — | Remote residential SearXNG endpoint (e.g. over Tailscale) |\n| `ARGUS__ENABLED` | `false` for limited API-key providers | Opt in to providers that consume limited credits or quotas |\n| `ARGUS_BRAVE_API_KEY` | — | Brave Search API key |\n| `ARGUS_SERPER_API_KEY` | — | Serper API key |\n| `ARGUS_TAVILY_API_KEY` | — | Tavily API key |\n| `ARGUS_EXA_API_KEY` | — | Exa API key |\n| `ARGUS_LINKUP_API_KEY` | — | Linkup API key |\n| `ARGUS_PARALLEL_API_KEY` | — | Parallel AI API key |\n| `ARGUS_YOU_API_KEY` | — | You.com API key |\n| `ARGUS_VALYU_API_KEY` | — | Valyu API key (search, contents, answer) |\n| `ARGUS_FIRECRAWL_API_KEY` | — | Firecrawl API key (content extraction) |\n| `ARGUS_GITHUB_API_KEY` | — | GitHub token (higher rate limit) |\n| `ARGUS_DATA_ROOT` | platformdirs user data dir | Override the Argus runtime corpus root |\n| `ARGUS_*_MONTHLY_BUDGET_USD` | provider-specific | Query-count budget for most providers; USD budget for Valyu |\n| `ARGUS_CRAWL4AI_ENABLED` | false | Enable Crawl4AI extraction step |\n| `ARGUS_YOU_CONTENTS_ENABLED` | false | Enable You.com Contents API extraction |\n| `ARGUS_OBSCURA_CDP_URL` | — | Obscura CDP endpoint (e.g. `ws://127.0.0.1:9222`) — makes Playwright use Obscura as its browser engine |\n| `ARGUS_OBSCURA_TIMEOUT_SECONDS` | 20 | Timeout for Obscura CLI subprocess calls |\n| `ARGUS_CACHE_TTL_HOURS` | 168 | Result cache TTL |\n| `ARGUS_BIND_HOST` | `127.0.0.1` | Host used by `argus serve` unless `--host` is passed |\n| `ARGUS_PORT` | `8000` | Port used by `argus serve` unless `--port` is passed |\n| `ARGUS_AUTOLOAD_DOTENV` | `true` | Auto-load `.env` / `.env.local` from cwd and repo root for CLI/API/MCP processes |\n| `ARGUS_API_KEY` | — | Required for non-local HTTP API and remote MCP callers |\n| `ARGUS_ADMIN_API_KEY` | — | Enables dashboard login and admin API authentication |\n| `ARGUS_ACCEPTED_OPERATION_AUTHORITY` | `legacy` | Atomic authority selection. `evidence` activates the registered planner, readiness, evidence repository, extraction finalizer, and HTTP presenters as one unit |\n| `ARGUS_ALLOWED_HOSTS` | — | Exact comma-separated HTTP Host allowlist; required for a remote production listener |\n| `ARGUS_ALLOWED_ORIGINS` | — | Exact comma-separated browser Origin allowlist. Set explicitly, including an empty value, for remote production |\n| `ARGUS_RETRIEVAL_SESSION_SECRET` | — | Stable random secret of at least 32 characters used to bind v2 retrieval sessions to authenticated principals |\n| `ARGUS_ORGANIZATION_POLICY_VERSION` | `1` | Stable organization-policy identity included in accepted execution cohorts |\n| `ARGUS_ROOT_PATH` | — | Public subpath prefix for dashboard links and redirects, e.g. `/argus` |\n| `ARGUS_MAYA_CAPTURE_URL` | — | Maya's dedicated Argus retrieval-capture endpoint; delivery stays disabled when unset |\n| `ARGUS_MAYA_CAPTURE_TOKEN` | — | Dedicated shared secret for Maya capture delivery; never reuse the generic Maya ingest token |\n| `ARGUS_MAYA_OUTBOX_BATCH_SIZE` | `20` | Maximum durable captures claimed by one delivery pass (bounded to 100) |\n| `ARGUS_MAYA_ACKNOWLEDGED_RETENTION_DAYS` | `7` | Days to retain acknowledged capture bodies before preserving audit metadata only |\n\n`/api/v2/*` is additive and returns a canonical version-2 envelope. It remains\nfail-closed with `unready` while the evidence authority is disabled. Unsafe\nHost, Origin, credential, media-type, and body-size combinations are rejected\nbefore provider, extractor, session, or persistence work. Version-1 routes\nretain their established response shapes.\n\n## When Not To Use Argus\n\nArgus is best when you need search, capture, provenance, and local artifacts together.\n\nAvoid it when:\n\n- you only need one search API and do not need fallback or budget controls\n- you only need a one-off page scrape with no persistent corpus or report output\n- you need an end-user search UI rather than backend retrieval infrastructure\n- you need fully deterministic summarization with no heuristic or LLM-assisted steps\n\n## FAQ\n\n**How is this different from calling Tavily/Serper directly?**\nArgus calls them for you — plus 13 other providers. You get one ranked, deduplicated result set instead of managing multiple API keys and stitching results together. Free providers are tried first, so you only burn credits when needed.\n\n**Can I run only one provider?**\nYes. Set only the API key for the provider you want. All others are silently skipped. For zero-config, just install and go — DuckDuckGo + Yahoo handle search with no keys.\n\n**Do I need Docker?**\nNo. `pip install argus-search` works immediately on any machine with Python 3.11+. Docker is only needed for SearXNG (set `ARGUS_SEARXNG_ENABLED=true` in `.env`) or Crawl4AI (local JS rendering).\n\n**Which Python version should contributors use?**\nUse Python 3.12 for repo development and verification: `uv sync --python 3.12 --extra dev --extra mcp` then `uv run pytest tests/ -v --tb=short`. The published package still supports Python 3.11+.\n\n**What is the safest way to deploy Argus on a network?**\nUse Tailscale or another private network, bind explicitly to the trusted interface, set `ARGUS_API_KEY`, and reserve `/api/admin/*` for `ARGUS_ADMIN_API_KEY`. Treat direct internet exposure as an advanced mode behind a reverse proxy.\n\n## License\n\nMIT — see [CHANGELOG.md](CHANGELOG.md) for release history.\n",
"bytes": 41050,
"sha": "6e16d0e9ca730700b8e9d75a0df14492acf556c5ea979fe5ba51d58d875849f4",
"repo_slug": "khamel83/argus",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_khamel83_argus_6f8526d0/readme"
}