{ "markdown": "# TinySearch\n\n\n\n

\n \n \"TinySearch\"\n \n

\n\n

\n Spend tokens on answers, not webpages.\n

\n\n

\n TinySearch searches, crawls, and reranks the web locally, then gives your\n agent only the evidence worth putting in its context.\n

\n\n

\n Documentation\n ·\n Quick start\n ·\n Python\n ·\n Discord\n

\n\n[![Website](https://img.shields.io/badge/tinysuite.dev-home-000000?logo=googlechrome&logoColor=white)](https://tinysuite.dev)\n[![PyPI version](https://img.shields.io/pypi/v/tinysuite-search?label=pypi)](https://pypi.org/project/tinysuite-search/)\n[![PyPI Downloads](https://static.pepy.tech/personalized-badge/tinysuite-search?period=total&units=INTERNATIONAL_SYSTEM&left_color=BLACK&right_color=GREEN&left_text=downloads)](https://pepy.tech/projects/tinysuite-search)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Release](https://img.shields.io/github/v/release/TinySuiteHQ/TinySearch?label=release)](https://github.com/TinySuiteHQ/TinySearch/releases)\n[![Last Commit](https://img.shields.io/github/last-commit/TinySuiteHQ/TinySearch)](https://github.com/TinySuiteHQ/TinySearch/commits/main)\n[![Docker Pulls](https://img.shields.io/docker/pulls/marcellm01/tinysearch?label=docker%20pulls)](https://hub.docker.com/r/marcellm01/tinysearch)\n[![Discord](https://img.shields.io/badge/Discord-Join%20community-5865F2?logo=discord&logoColor=white)](https://discord.gg/mFFKF9bf5e)\n![MCP Server](https://img.shields.io/badge/MCP-server-blue)\n![FastAPI](https://img.shields.io/badge/FastAPI-supported-009688)\n\nTinySearch is a self-hosted web-research tool for AI agents. It searches the\nweb, reads the best pages, removes low-value content, and returns compact\nevidence with source URLs.\n\nYour model receives the useful passages instead of paying to process entire\nwebpages.\n\nTinySearch is part of [TinySuite](https://tinysuite.dev), a suite of focused\ntools designed to make agentic operations cheaper by minimizing token usage\nthrough smart retrieval, selection, and context-management techniques.\n\n

\n \"TinySearch\n

\n\n## Choose a tier\n\n| Tier | Use it when | Entry point | Search backend |\n| --- | --- | --- | --- |\n| 1. Python library | You are building with TinySuite or Python | `pip install tinysuite-search` | DDGS |\n| 2. One-command MCP | An MCP client should launch TinySearch for you | `uvx --from \"tinysuite-search[server]\" tinysearch` | DDGS |\n| 3. Docker + SearXNG | You want the full self-hosted stack and HTTP MCP | `docker compose ... up -d` | Bundled SearXNG |\n\nTiers 1 and 2 need no search service. Tier 3 adds a dedicated SearXNG service,\npersistent model storage, and a network MCP endpoint. See the\n[installation guide](https://tinysuite.dev/docs/tinysearch/) for the Docker\nsetup.\n\n## The expensive part of agent research is context\n\nA search result is not yet useful evidence. Agents often have to open several\npages, ingest navigation and boilerplate, and spend paid input tokens deciding\nwhich passages matter.\n\nTinySearch moves that work in front of the model:\n\n```mermaid\nflowchart LR\n A[Question] --> B[Search and crawl]\n B --> C[Local hybrid reranking]\n C --> D[Compact evidence
with source URLs]\n D --> E[Your agent]\n```\n\nThat lowers cost in three ways:\n\n- **Smaller model context.** Only the best-ranked evidence chunks are returned,\n within a controlled evidence budget.\n- **No metered search API required by default.** TinySearch can search through\n DDGS without a paid search provider.\n- **Local retrieval by default.** ONNX embeddings and hybrid reranking run on\n your machine instead of creating embedding API charges.\n\nSearch broadly. Read locally. Pay the model only for the evidence that matters.\n\nThis is retrieval, not summarization: TinySearch selects the passages worth\nkeeping with local BM25 and embedding rerank, it doesn't run a model over the\npage to rewrite or condense it. Every returned chunk is the original page\ntext, unedited, so what you cite is what the page actually said. That keeps\nthe pipeline fast and free to run locally, at the cost of not compacting as\naggressively as a dedicated reduction model could. A learned reduction step\nis a direction we may explore later; it isn't part of TinySearch today.\n\nActual savings depend on the pages, evidence limits, client model, and provider\npricing. TinySearch reduces the web content sent to the model; it does not\ncontrol what the client does with that evidence afterward.\n\n

\n \"Benchmark:\n

\n\nThe cost panel uses an illustrative $3.00 per million input-token rate and\nexcludes search, crawling, model output, and downstream agent use.\n\nThe naive baseline isn't a strawman product, it's the same pages TinySearch\ncrawled for each query, fed to the model unfiltered, the way a generic\n\"search, then fetch the page\" tool (a plain web-search-plus-fetch loop, the\nkind built into most coding agents) would. Measured against the current\nrecommended flow (`search` then `scrape_urls`) and counted on the actual MCP\ntool-result text, TinySearch's primary interface. Reproduce or rerun it\nyourself:\n\n```bash\npython scripts/benchmark_token_savings.py --json-out report.json\n```\n\n## Quick start\n\nWith [`uv`](https://docs.astral.sh/uv/) installed, add TinySearch to any MCP\nclient:\n\n```json\n{\n \"mcpServers\": {\n \"tinysearch\": {\n \"command\": \"uvx\",\n \"args\": [\n \"--python\",\n \"3.12\",\n \"--from\",\n \"tinysuite-search[server]\",\n \"tinysearch\"\n ]\n }\n }\n}\n```\n\nThe client launches TinySearch over stdio when it needs it. No repository\nclone, hosted account, or paid search key is required.\n\nFast `search` starts without Chromium or an embedding model. The first scrape\ninitializes Chromium; focused scraping also initializes the configured\nembedding model. Pre-warm both ahead of time if you will use those workflows:\n\n```bash\nuvx --from \"tinysuite-search[server]\" tinysearch setup\n```\n\n

\n \"TinySearch\n

\n\nPrefer Docker, a remote MCP endpoint, or a source checkout? Follow the\n[installation guide](https://tinysuite.dev/docs/tinysearch/).\n\n## Three MCP tools\n\n| Tool | Use it when |\n| --- | --- |\n| `search(items)` | You need fast, backend-ordered discovery without crawling or reranking; batch independent subquestions when useful |\n| `scrape_urls(items)` | You know one to five pages; each item may use `*` for its configured clean page-order token budget |\n| `get_current_datetime()` | A question depends on the current date or time |\n\nTinySearch deliberately stays focused. It is a retrieval layer, not another\nagent, chat interface, hosted search product, or permanent web index.\n\nSee the complete [MCP tool reference](https://tinysuite.dev/docs/tinysearch/mcp-tools/)\nfor parameters and response contracts.\n\n## What your agent gets\n\nTinySearch does not spend another model call writing the final answer. The\nrecommended flow is `search` for lightweight discovery, then `scrape_urls` for\nthe pages worth reading.\n\nSearch returns structured JSON. Use one item for a simple lookup; add multiple\nitems only for independent subquestions or source strategies. `domains` is a\nhard positive source restriction and accepts a domain plus its subdomains:\n\n```json\n{\"items\":[{\"query\":\"Form 8-K Tesla\",\"domains\":[\"sec.gov\"]}]}\n```\n\nEach search item reports its own results and compact backend attempts. A zero\nresult response is distinct from a blocked, unavailable, or invalid backend.\n`scrape_urls` returns selected Markdown evidence and separate related-link\nnavigation candidates, each with independent configured token ceilings.\n\nMCP still uses its standard JSON-RPC transport envelope, including\nprotocol-level errors and optional `structuredContent`. Python and FastAPI keep\ntheir structured JSON contracts for applications that need to store, inspect,\nor transform the evidence.\n\n## How it works\n\n1. `search` returns backend-ordered titles, URLs, previews, upstream dates, and\n backend outcomes without starting Chromium or an embedding model.\n2. `scrape_urls` reads one to five known pages concurrently. Omit an item's\n query or use `\"*\"` to keep clean Markdown in page order within the\n configured token budget.\n3. Supply a focused item query when TinySearch should chunk and hybrid-rank\n that page before returning evidence.\n\n## Python library\n\nTinySearch also works as a regular Python package:\n\n```bash\npip install tinysuite-search\n```\n\n## Optional OpenTelemetry export\n\nTinySearch emits vendor-neutral traces and metrics only when OpenTelemetry is\nexplicitly configured. Normal library, MCP, and FastAPI behavior is unchanged\nwhen telemetry is not installed or not configured.\n\nInstall the optional exporter support for a standalone MCP server:\n\n```bash\nuvx --from \"tinysuite-search[server,telemetry]\" tinysearch\n```\n\nThe official Docker image includes the same optional support. Set standard OTel\nvariables on either deployment; the common endpoint enables traces and metrics:\n\n```bash\nOTEL_SERVICE_NAME=tinysearch \\\nOTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 \\\nOTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \\\ntinysearch serve\n```\n\n`http/protobuf` is the default and `grpc` is also supported. Use\n`OTEL_TRACES_EXPORTER=none`, `OTEL_METRICS_EXPORTER=none`, or\n`OTEL_SDK_DISABLED=true` to disable telemetry. Standard resource, header,\ntimeout, sampler, and signal-specific endpoint settings are passed through to\nthe OpenTelemetry SDK; treat `OTEL_EXPORTER_OTLP_HEADERS` as a secret.\n\nTinySearch exports operation/stage timing, outcomes, counts, backend state,\nbrowser use, token counts, and embedding model metadata. It never exports\nqueries, URLs, domains, prompts, documents, snippets, request headers,\ncredentials, configuration paths, raw errors, exception stacks, MCP arguments,\nor MCP results. Direct Python-library users configure their own OTel provider;\nTinySearch only auto-configures its standalone MCP and FastAPI server entry\npoints.\n\n```python\nimport asyncio\nfrom tinysearch import scrape_urls, search\n\n\nasync def main():\n results = await search([{\"query\": \"Python async tasks\"}])\n print(results[\"items\"][0][\"results\"])\n\n page_url = results[\"items\"][0][\"results\"][0][\"url\"]\n evidence = await scrape_urls([{\n \"url\": page_url,\n \"query\": \"How does asyncio cancellation work?\",\n }])\n print(evidence[\"results\"])\n\n\nasyncio.run(main())\n```\n\nThe Python API returns stable, JSON-serializable results. `search` accepts one\nto five items and uses the configured per-item result limit. `scrape_urls` accepts a per-call `max_tokens`\nbudget (4,000 by default); omit an item's scrape query or use `\"*\"` for\npage-order mode. Rendering structured evidence into an LLM prompt is explicit,\nso applications can store, inspect, transform, or budget the result first.\n\nThe optional FastAPI app mirrors these surfaces. `POST /search` accepts the\nsame batch JSON contract.\n`POST /scrape` accepts one to five `{ \"url\", \"query\" }` items and always\nreturns structured per-item outcomes.\nThe app also exposes `/health`, `/current_datetime`, and read-only `/config`;\nconfiguration writes require explicit environment opt-in.\n\n## Search backends\n\nTinySearch selects a web-search backend from config, so you can start with no\nsearch service and add one later without changing code.\n\n- `\"ddgs\"` (native default): queries the [`ddgs`](https://pypi.org/project/ddgs/)\n package's automatic backend selection in-process. No SearXNG deployment\n required.\n- `\"searxng\"` (Docker default): queries a self-hosted SearXNG instance. Falls\n back to `ddgs` on backend failure unless `search_backend_fallback` is set to\n `false`.\n- `\"duckduckgo\"`: skips SearXNG and queries `ddgs` in DuckDuckGo-only mode.\n- `\"auto\"`: tries SearXNG, then falls back to `ddgs` on any backend failure.\n\nSet the `BRAVE_SEARCH_API_KEY` environment variable to add Brave's official\nWeb Search API as a keyed fallback for the `ddgs` and `duckduckgo` backends.\nBrave is only consulted when the primary call errors or returns no results.\n\nFull key reference, SearXNG JSON-output setup, and Compose details live in the\n[configuration reference](https://tinysuite.dev/docs/tinysearch/configuration/).\n\n## External browser over CDP\n\nTinySearch uses its bundled Playwright Chromium by default. To use a browser\nthat you operate separately, set its Chrome DevTools Protocol endpoint in the\nconfig file:\n\n```json\n{\n \"browser_cdp_url\": \"http://browser:9222\"\n}\n```\n\nServer processes also accept `TINYSEARCH_BROWSER_CDP_URL`. When either setting\nis present, TinySearch connects through Crawl4AI instead of installing or\nlaunching the bundled Chromium. The external browser owns its executable,\nprofile, proxy, and fingerprint configuration; TinySearch does not select or\ninstall a particular browser backend.\n\nTreat a CDP endpoint as privileged remote control of the browser. Keep it on a\nprivate network or loopback interface, require authentication when it crosses\na host boundary, and do not expose port 9222 directly to the public internet.\nWhen TinySearch itself runs in Docker, `localhost` refers to the TinySearch\ncontainer, so use an endpoint reachable from that container.\n\nThe CDP endpoint is operator-managed and cannot be changed through the HTTP\n`PUT /config` endpoint, even when configuration writes are enabled. Set it in\nthe startup environment or the file selected by `TINYSEARCH_CONFIG_PATH`, then\nrestart TinySearch. HTTP clients can continue updating other settings by\nomitting `browser_cdp_url` from their partial update.\n\n## Why TinySearch\n\n- **No vendor in the loop.** No TinySearch account, no required API key, no\n per-request billing, no analytics service or hosted scraped-data cache. The\n infrastructure you'd otherwise pay a search API for runs on your machine.\n- **Source-grounded by construction.** Every evidence chunk is the original\n page text, still attached to its originating URL, so a claim in your\n agent's answer traces back to one specific passage instead of stopping at\n \"the vendor's model said this.\"\n- **Built around token efficiency.** Page selection and passage selection\n happen locally, before content enters model context.\n- **Useful without paid infrastructure.** DDGS search and local ONNX embeddings\n are the defaults.\n- **Bring your own stack when needed.** SearXNG and OpenAI-compatible embedding\n providers remain optional.\n- **Works where agents already work.** Use MCP over stdio, Streamable HTTP,\n Python, FastAPI, or Docker.\n\n## Part of TinySuite\n\n[TinySuite](https://tinysuite.dev) is a product suite built around one idea:\nagents should spend tokens on useful work, not operational overhead.\n\nEach tool focuses on a different part of the agent workflow and uses targeted\ntechniques to reduce unnecessary context before it reaches the model.\nTinySearch handles the web-research layer by turning pages into a small,\nranked, source-grounded evidence packet.\n\n## Documentation\n\nThe README is the product overview. Detailed setup and operational material\nlives in the TinySuite documentation:\n\n- [TinySearch overview and installation](https://tinysuite.dev/docs/tinysearch/)\n- [Configuration reference](https://tinysuite.dev/docs/tinysearch/configuration/)\n- [MCP tools](https://tinysuite.dev/docs/tinysearch/mcp-tools/)\n- [Troubleshooting](https://tinysuite.dev/docs/tinysearch/troubleshooting/)\n\nThe repository also contains an annotated example configuration at\n[`configs/tinysearch_config.json`](configs/tinysearch_config.json).\n\n## When not to use TinySearch\n\nTinySearch is intentionally lightweight. Use a commercial search API,\npersistent crawler, or full search index when you need:\n\n- guaranteed search coverage or an SLA\n- large-scale or scheduled indexing\n- long-term page storage and change history\n- enterprise observability and access controls\n\n## Development\n\n```bash\ngit clone https://github.com/TinySuiteHQ/TinySearch\ncd TinySearch\npython -m venv .venv\nsource .venv/bin/activate\npip install -e \".[server]\"\npython -m unittest discover tests\n```\n\nTinySearch supports Python 3.12 and newer. CI tests Python 3.12, 3.13, and 3.14\nacross Linux, macOS, and Windows.\n\n## Entrypoints\n\n- `tinysearch.search` and `tinysearch.scrape_urls`: structured Python API\n- `tinysearch.get_current_datetime`: structured UTC date and time\n- `tinysearch.to_prompt`: pure structured-evidence prompt renderer\n- `tinysearch mcp`: stdio MCP server (also the no-argument default)\n- `tinysearch serve`: Streamable HTTP MCP server\n- `tinysearch.servers.fastapi_server:app`: optional FastAPI application\n\n## Community\n\nQuestions, ideas, and bug reports are welcome:\n\n- [Join the TinySearch Discord](https://discord.gg/mFFKF9bf5e)\n- [Open a GitHub issue](https://github.com/TinySuiteHQ/TinySearch/issues)\n- [Email the maintainer](mailto:hello.marcbuilds@gmail.com)\n\n## Privacy and license\n\nTinySearch reads public pages and returns selected excerpts to the calling\nclient. Search, crawling, local embeddings, and reranking can run without\nsending page content to an embedding provider. If you choose an\nOpenAI-compatible embedding backend, that provider receives the text sent for\nvectorization.\n\nTinySearch is available under the [MIT License](LICENSE). Downloaded model\nweights remain subject to their respective model-card licenses. See\n[NOTICE](NOTICE) for third-party distribution details.", "bytes": 18144, "sha": "4c21e30ce4914596885a0b165d5573dd57e6f26756902a72b44e8d2245881940", "repo_slug": "marcellm01/tinysearch", "fonte": "repo", "truncated": false, "api": "https://agentalog.com/api/listings/mcp_io_github_marcellm01_tinysearch_f096c709/readme" }