{
  "markdown": "<!-- mcp-name: io.github.Brightdotdev/darwin-rag -->\n\n# Darwin RAG\n\nA local-first RAG engine that ingests documents, indexes them with BM25 + dense embeddings, and exposes search via an MCP server for AI agent integration.\n\n- **Ingestion** — PDF, Markdown, HTML, images (OCR), CSV, Excel, ODS, URLs\n- **Indexing** — BM25 keyword + dense embedding hybrid index with configurable chunking strategies\n- **Search** — Hybrid, semantic, or keyword retrieval with reranking, diversity rerank, and structural penalties\n- **Generation** — LLM-backed answer synthesis via LiteLLM (OpenAI, Anthropic, Gemini, etc.)\n- **Observability** — Structured logging with per-session history, queryable via MCP\n- **Isolation** — Multiple independent stores for tenant/project separation\n- **Deployment** — stdio (AI agent subprocess), SSE, or Streamable HTTP; Docker-ready\n- **Local-first** — Everything runs locally, fully offline-capable after setup\n\nBuilt by [BrightDotDev](https://github.com/brightdotdev).  \n[![License: MIT with Attribution](https://img.shields.io/badge/License-MIT%20with%20Attribution-blue.svg)](LICENSE)\n\n---\n\n## Quick Start\n\n### 1. Install\n\n```bash\npip install darwin-rag\n```\n\n### 2. Set up models\n\n```bash\n# Interactive — detects hardware, pick your models\ndarwin-admin setup interactive\n\n# Or one-shot (embedding-only, no prompts)\ndarwin-admin setup --preset required\n```\n\n### 3. Start the MCP server & connect\n\n```bash\n# stdio mode — for AI agent subprocess (Claude Desktop, Cursor, etc.)\ndarwin mcp\n\n# Or HTTP mode — for remote clients\ndarwin mcp --http --port 8765\n```\n\nConfigure your MCP client:\n\n```json\n{\n  \"mcpServers\": {\n    \"darwin\": {\n      \"command\": \"darwin\",\n      \"args\": [\"mcp\"],\n      \"env\": {\n        \"OPENAI_API_KEY\": \"sk-...\"  // At least one LLM provider key\n      }\n    }\n  }\n}\n```\n\nOr generate config automatically:\n\n```bash\ndarwin config claude          # Claude Desktop config\ndarwin config cursor          # Cursor config\ndarwin config all --copy      # All clients + copy to clipboard\n```\n\n---\n\n## MCP Tools\n\n| Tool | Description |\n|------|-------------|\n| `search_darwin` | Query the knowledge base with hybrid/semantic/keyword search |\n| `search_lists` | Search structured data (CSVs, JSON arrays) by field values |\n| `get_search_results` | List saved search results |\n| `get_search_result_by_id` | Load a saved search result by filename |\n| `get_schema` | Inspect schemas for structured files (keys, types, record counts) |\n| `run_pipeline` | Ingest + index documents from a path or URL |\n| `purge_artifacts` | Delete pipeline artifacts for specific files |\n| `create_store` | Create a new isolated data store |\n| `list_files` | List all tracked files with pipeline status |\n| `file_status` | Detailed status for a single file across all stages |\n| `get_logs` | Query session logs (oldest first, INFO excluded) |\n\n> Full documentation: [docs/mcp.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/mcp.md)\n\n---\n\n## Remote / HTTP Mode\n\nStart the server on a network-accessible endpoint:\n\n```bash\n# SSE transport (legacy)\ndarwin mcp --sse --host 0.0.0.0 --port 8765\n\n# Streamable HTTP transport (recommended for production)\ndarwin mcp --http --host 0.0.0.0 --port 8765\n```\n\nConfigure your MCP client with the URL:\n\n```json\n{\n  \"mcpServers\": {\n    \"darwin\": {\n      \"url\": \"http://your-host:8765/mcp\"  // or /sse for SSE mode\n    }\n  }\n}\n```\n\n---\n\n## Environment Variables\n\n| Variable | Required | Description |\n|----------|----------|-------------|\n| `OPENAI_API_KEY` | No* | OpenAI provider key |\n| `ANTHROPIC_API_KEY` | No* | Anthropic provider key |\n| `GEMINI_API_KEY` | No* | Google Gemini provider key |\n| `MISTRAL_API_KEY` | No* | Mistral AI provider key |\n| `GROQ_API_KEY` | No* | Groq provider key |\n| `COHERE_API_KEY` | No* | Cohere provider key |\n| `TOGETHER_API_KEY` | No* | Together AI provider key |\n| `OPENROUTER_API_KEY` | No* | OpenRouter provider key |\n| `DEEPSEEK_API_KEY` | No* | DeepSeek provider key |\n| `DARWIN_BASE_DIR` | No | Override the base data directory |\n| `NO_COLOR` | No | Set to any value to disable ANSI color output |\n\n*\\* At least one LLM provider key is required for answer generation. Search/indexing works without any.*\n\n---\n\n## Setup Details\n\n| Command | What it does |\n|---------|-------------|\n| `darwin-admin setup interactive` | Guided setup — detect hardware, choose models |\n| `darwin-admin setup --preset required` | Download embedding model only (fastest) |\n| `darwin-admin setup --preset recommended` | Embedding + reranker + OCR models |\n| `darwin-admin setup logging` | Reconfigure logging only |\n| `darwin-admin setup validate` | Validate current setup |\n\nSee [docs/setup.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/setup.md) for the full walkthrough including Docker, from-source install, and API key configuration.\n\n---\n\n## CLI Reference\n\n**`darwin`** — User CLI\n\n| Command | Description |\n|---------|-------------|\n| `darwin mcp` | Start MCP server (stdio, `--sse` or `--http` for network) |\n| `darwin config [client]` | Generate MCP client config |\n\n**`darwin-admin`** — Power-user CLI\n\n| Command | Description |\n|---------|-------------|\n| `darwin-admin setup` | Setup models, logging, and configuration |\n| `darwin-admin status` | System status overview |\n| `darwin-admin models` | Model registry: list, install, switch, keys |\n| `darwin-admin store` | Data store: status, files, audit, health, repair |\n| `darwin-admin pipeline` | Ingestion pipeline: run, ingest, index, purge |\n| `darwin-admin search` | Interactive search |\n| `darwin-admin logs` | Structured log viewer and management |\n| `darwin-admin system` | System information |\n| `darwin-admin uninstall` | Remove Darwin data and configuration |\n\nSee [docs/admin.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/admin.md) for the full command reference.\n\n---\n\n## Python API\n\nFor embedding darwin-rag as a library in your own app:\n\n```python\nfrom core import Darwin\n\nd = Darwin()\nd.ingest(\"./papers\", recursive=True)\nresults = d.search(\"what is this paper about\")\nrecords = d.search_records(filters={\"status\": \"active\"})\n```\n\nFull reference: [docs/api.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/api.md)\n\n---\n\n## Documentation\n\n| Doc | What |\n|-----|------|\n| [setup.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/setup.md) | Full setup walkthrough |\n| [mcp.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/mcp.md) | MCP server, tools, resources, transports |\n| [admin.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/admin.md) | Admin CLI reference |\n| [architecture.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/architecture.md) | For developers and contributors |\n| [pipeline.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/pipeline.md) | Ingestion & indexing |\n| [retrieval.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/retrieval.md) | Search engine |\n| [storage.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/storage.md) | DarwinStore |\n| [models.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/models.md) | Model registry & inference |\n| [logger.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/logger.md) | Structured logging |\n| [orchestrators.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/orchestrators.md) | High-level business logic |\n| [api.md](https://github.com/BrightDotDev/DARWIN/blob/main/docs/api.md) | Python API (Darwin class) |\n\n---\n\n## Contributing\n\nFound a bug? Want to add something? You're welcome here.\n\n- **Issues** — open one at [github.com/BrightDotDev/DARWIN/issues](https://github.com/BrightDotDev/DARWIN/issues)\n- **Code** — fork, branch, PR. Keep it minimal.\n- **AI-generated code is fine** — but you own what you ship. Test it before submitting.\n\nRead [CONTRIBUTING.md](https://github.com/BrightDotDev/DARWIN/blob/main/CONTRIBUTING.md) for the full guidelines.\n\n---\n\n## License\n\nMIT with Attribution — see [LICENSE](LICENSE).\n\nCore architecture and implementation by [BrightDotDev](https://github.com/brightdotdev).\n",
  "bytes": 8047,
  "sha": "a12bc3d3d83b34c0c08b2ce6afe04f28a3149d6da6bd6f40a9f4dc72f9c97e61",
  "repo_slug": "brightdotdev/darwin",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_brightdotdev_darwin_rag_085db82d/readme"
}