{
  "markdown": "# searxng-deepdive\n\n[![Tests](https://github.com/burakaydinofficial/searxng-deepdive/actions/workflows/test.yml/badge.svg)](https://github.com/burakaydinofficial/searxng-deepdive/actions/workflows/test.yml)\n[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)\n\nAn [MCP](https://modelcontextprotocol.io/) server for [SearXNG](https://docs.searxng.org/)\ndesigned for LLM agents doing real research. Four tools with agent-friendly\nschemas, multi-page result fanout, lightweight URL→Markdown reading, and\ntool descriptions generated dynamically from the live engine pool of *your*\nSearXNG instance.\n\n## Why another mcp-searxng?\n\nExisting packages are minimal — most expose a single `search(query)` tool\nwith no way for the model to ask for more results, target specific engines,\nor constrain by category. The richer ones bake static descriptions, so the\nLLM never learns what's actually enabled on *this* instance. None of them\ntreat agent-tool-selection ergonomics as a design priority.\n\n`searxng-deepdive` opens those knobs up:\n\n| Feature | This | npm `mcp-searxng` (ihor-sokoliuk) | PyPI `mcp-searxng` (SecretiveShell) |\n|---|---|---|---|\n| Engine targeting | ✅ via `search_on_engines` | ❌ | ❌ |\n| Category targeting | ✅ via `search_by_category` | ❌ | ❌ |\n| Multi-page fanout in one call | ✅ via `pages: N` | ❌ (one page per call) | ❌ |\n| Pagination | ✅ via `pageno` | ✅ | ❌ |\n| Compact response trim | ✅ via `format: \"compact\"` | ❌ | ❌ |\n| Dynamic descriptions per instance | ✅ live engine list injected | ❌ static | ❌ static |\n| Validation with cross-tool hints | ✅ engine-vs-category, case-insensitive | ❌ | ❌ |\n| Zero-result hints | ✅ time_range / unresponsive engines / single-engine | ❌ | ❌ |\n| URL reader (HTML→Markdown) | ✅ with TOC scan + section extraction | ✅ basic | ❌ |\n| Test suite | ✅ 102 unit + integration | minimal | ❌ |\n\n## Quickstart\n\nInstall via `npx -y` from any MCP client:\n\n```json\n{\n  \"mcpServers\": {\n    \"searxng\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"searxng-deepdive\"],\n      \"env\": { \"SEARXNG_URL\": \"http://127.0.0.1:7979/\" }\n    }\n  }\n}\n```\n\n`SEARXNG_URL` should point at your running SearXNG instance. Need one?\nThe companion repo [SearXNG-Compose](https://github.com/burakaydinofficial/SearXNG-Compose)\nships a plug-and-play Docker stack tuned for LLM consumption.\n\n**Requirements:** Node.js 22 or newer.\n\n## Tools\n\nThe server registers four tools. The LLM picks among them based on the\ndescriptions below, augmented at startup with the live engine and category\nlist from your instance.\n\n### `search(query, [...])`\n\nBroad web search across the full enabled engine pool. Use when you don't\nhave a specific source preference. Returns merged, deduplicated results\nacross however many engines respond.\n\n### `search_on_engines(query, engines, [...])`\n\nSearch using only the specified engines (e.g. `[\"arxiv\", \"pubmed\", \"semantic scholar\"]`).\nThe tool description registered with the MCP client includes the actual\nlist of engines enabled on your instance — agents don't have to guess\nnames. Validation rejects invalid names with a \"did you mean\" hint when\nthey look like categories instead of engines.\n\n### `search_by_category(query, categories, [...])`\n\nSearch within specific categories — runs every engine tagged with each.\nDescription includes the live category list and which engines belong to\neach. Same validation: invalid category names produce a clear error\nthat points at `search_on_engines` when the offending value is actually\nan engine name.\n\n### `web_url_read(url, [readHeadings, section, paragraphRange, startChar, maxLength])`\n\nFetch a URL and convert its HTML to clean Markdown. Lightweight HTTP +\nHTML→Markdown (no headless browser) — handles ~80% of the static-HTML\nweb (Wikipedia, docs sites, blogs, news, GitHub READMEs).\n\nToken-efficient extraction modes (priority order, first set wins):\n\n- `readHeadings: true` — return only the heading list as a hierarchical TOC\n- `section: \"Installation\"` — return content under matching heading\n- `paragraphRange: \"3-7\"` — 1-indexed paragraph slice\n- `startChar` + `maxLength` — character window pagination\n\nRecommended workflow for long pages: TOC scan first (`readHeadings`), then\ntargeted read (`section`). Far more token-efficient than fetching the full\npage up front.\n\nIf `readHeadings` comes back with no entries (Reddit threads, comment\nsections, blog posts that use bold paragraphs instead of `<h*>` tags),\nthe page is structurally flat — fall through to `paragraphRange` for\nsequential sampling, or just fetch without an extraction mode.\n\n`web_url_read` also accepts JSON, YAML, and TOML content-types directly\n(spec files, package manifests, registry API responses, CI workflow\nYAML), so research agents can read these without the HTML-only stub.\n\nFor JS-rendered SPAs and bot-protected sites this tool returns minimal/empty\ncontent — fall back to a Chromium-backed reader (e.g. Crawl4AI) for those.\n\n### Common parameters across all search tools\n\n- `pageno` — 1-indexed starting page (default 1)\n- `pages` — multi-page fanout in one call (1–5, default 1)\n- `time_range` — `day` / `week` / `month` / `year` (warning: not all engines support this; some return empty when set)\n- `language` — BCP-47 code or `all`\n- `safe_search` — 0 / 1 / 2\n- `format` — `compact` (default) or `full`\n\n## Configuration\n\n| Env var | Default | Meaning |\n|---|---|---|\n| `SEARXNG_URL` | `http://127.0.0.1:8080` | Base URL of the SearXNG instance |\n\n## Development\n\n```bash\ngit clone <this repo>\ncd searxng-deepdive          # or wherever you cloned to\nnpm install\nnpm run build                # tsc\nnpm test                     # vitest\nSEARXNG_URL=http://127.0.0.1:7979 npm run probe    # exercise the SearXNG client\nSEARXNG_URL=http://127.0.0.1:7979 npm run dev      # start the MCP stdio server\n```\n\n### Pointing an MCP client at the source during development\n\nUse `tsx` to run from `src/` directly so you don't need to rebuild on every edit:\n\n```json\n{\n  \"mcpServers\": {\n    \"searxng\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"tsx\", \"/absolute/path/to/searxng-deepdive/src/index.ts\"],\n      \"env\": { \"SEARXNG_URL\": \"http://127.0.0.1:7979/\" }\n    }\n  }\n}\n```\n\n> **MCP clients cache the subprocess.** When you edit code, the running\n> server keeps the old behavior until the subprocess is killed and\n> respawned. Quit the host (LM Studio, Claude Desktop, etc.) fully and\n> reopen — closing the chat window alone usually isn't enough. Symptom of\n> not doing this: a fix you just shipped doesn't appear to take effect.\n\n## Testing\n\n```\nnpm test\n```\n\nTest coverage spans seven files:\n\n- **normalize-name** — case-insensitive name handling\n- **validators** — engine/category validation with cross-reference hints\n- **zero-result-hint** — every hint trigger and its inverse\n- **trim-to-compact** — response trimming + hint inclusion\n- **descriptions** — anti-pattern regex checks for the description copy\n  that misled real models in earlier versions (\"ignored by engines\",\n  \"Default 'auto'\", etc.) — failing build if they reappear\n- **searxng-client** — HTTP client with `MockAgent`: malformed JSON,\n  HTML 502 pages, 429 rate-limit handling, multi-page fanout dedup,\n  all-pages-fail throws\n- **url-reader** — extraction modes + HTTP integration\n\n## Design notes\n\n- **Why four tools instead of one with optional engine/category params?**\n  Cleaner agent decision-making. With distinct tools the LLM sees explicit\n  purposes; with one fat tool it has to remember when to set which optional\n  flags. Trade-off: more entries in the MCP tool list, mostly identical\n  handler code. Net: better agent ergonomics, especially for smaller models.\n\n- **Why `format: \"compact\"` as default?** SearXNG's full result objects are\n  several times heavier than just url+title+content+engine. For the typical\n  agent workflow (rank candidates, pick a few to fetch in detail), the\n  compact form is what the LLM actually uses. `format: \"full\"` is one\n  parameter away when you need scores, dates, authors, or DOI.\n\n- **Why dynamic descriptions?** Static descriptions either list every\n  upstream engine (most aren't enabled on a given instance — wastes context)\n  or list none (LLM has no idea what to put in `engines`). Live introspection\n  of `/config` at server startup gives the LLM exactly the right hint for\n  *this* instance.\n\n- **Why convert silent-wrong into informatively-wrong?** Real LM Studio\n  testing showed agents repeatedly stuck in retry loops because failed\n  searches looked successful (zero results, looked like \"no matches\"; or\n  60 garbage results, looked like the search ran). The validation +\n  zero-result-hint pattern surfaces the actual cause every time. The\n  description-anti-pattern test suite locks in copy that was empirically\n  shown to mislead models.\n\n## Security notes\n\nThis package is designed to run **locally**, inside the user's trust\nboundary, alongside an MCP-speaking LLM client (Claude Desktop, LM Studio,\nCursor, etc.). The trust model assumes:\n\n- the LLM is acting on the user's behalf\n- the user controls what model is connected to the server\n- the MCP transport is stdio, not exposed to remote callers\n\nWithin that boundary, two surfaces are worth knowing about:\n\n- **`web_url_read` will fetch any HTTP(S) URL the model hands it**, with\n  up to five redirects. On a host that can route to private networks,\n  the model can therefore reach intranet services, link-local addresses,\n  or cloud-instance metadata endpoints (`169.254.169.254`, etc.). This\n  is by design for a local research tool but means you should not run\n  this MCP server in topologies where an untrusted party can pick the\n  URLs (e.g. a hosted MCP gateway facing the public internet). If you do\n  want to lock it down, set `SEARXNG_DEEPDIVE_BLOCK_PRIVATE=1` to refuse\n  private / loopback / link-local / metadata destinations — enforced on\n  every redirect hop and against the actual dialed IP, so DNS rebinding\n  can't slip past — and allow specific internal hosts back in with\n  `SEARXNG_DEEPDIVE_ALLOWED_HOSTS`. Non-HTTP(S) schemes — including\n  `file://` — are always rejected, so the tool never reads local files.\n  Body size is capped at 10 MB, a single request has a 45 s total deadline,\n  and the returned Markdown is length-capped (paginate with\n  `startChar`/`maxLength`), so a malformed or oversized upstream can't\n  trivially exhaust memory or the caller's context window.\n\n- **The `search` tools forward the model's query verbatim to SearXNG.**\n  SearXNG is the trust boundary for upstream engine traffic; this\n  package does not add additional rate-limiting or query rewriting.\n\n- **Tool output is adversarial input — prompt injection is possible.**\n  Search-result snippets and the Markdown returned by `web_url_read`\n  both contain text the model will read as part of its working\n  context. A page or snippet you don't control can carry instructions\n  (\"ignore previous instructions and …\"). This isn't a defect in\n  this MCP server — it's inherent to any tool that returns external\n  text — but agent loops that auto-act on tool output without human\n  review are the threat model. Treat tool output as untrusted input,\n  especially for `web_url_read` against URLs the model picked rather\n  than the user.\n\nThis package is provided **as-is under MIT** with no warranty or\nliability for damages — see [LICENSE](LICENSE). Report suspected\nvulnerabilities privately via [GitHub Security Advisories](https://github.com/burakaydinofficial/searxng-deepdive/security/advisories/new)\nrather than opening a public issue. See [SECURITY.md](SECURITY.md).\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n",
  "bytes": 11620,
  "sha": "f202e4495bbee6df6bf72bad7856421e80f026030106705066b3d57f1577cf04",
  "repo_slug": "burakaydinofficial/searxng-deepdive",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_burakaydinofficial_searxng_dee_29c382d7/readme"
}