{
  "markdown": "<!-- mcp-name: io.github.thomasxm/crowdsentinel-mcp-server -->\n\n<div align=\"center\">\n  <img src=\"https://raw.githubusercontent.com/thomasxm/crowdsentinel-assets/main/logo.png\" alt=\"CrowdSentinel Logo\" width=\"400\"/>\n\n# CrowdSentinel MCP Server\n\n### AI-Powered Threat Hunting & Incident Response Framework\n\n[![PyPI](https://img.shields.io/pypi/v/crowdsentinel-mcp-server.svg)](https://pypi.org/project/crowdsentinel-mcp-server/)\n[![PyPI Downloads](https://img.shields.io/pypi/dm/crowdsentinel-mcp-server.svg)](https://pypi.org/project/crowdsentinel-mcp-server/)\n[![Python](https://img.shields.io/pypi/pyversions/crowdsentinel-mcp-server?cacheSeconds=3600)](https://pypi.org/project/crowdsentinel-mcp-server/)\n[![License](https://img.shields.io/badge/License-GPL%20v3-blue.svg)](LICENSE)\n[![MCP](https://img.shields.io/badge/MCP-Compatible-purple.svg)](https://modelcontextprotocol.io/)\n[![Tools](https://img.shields.io/badge/MCP%20Tools-139-brightgreen.svg)](https://github.com/thomasxm/CrowdSentinels-AI-MCP)\n[![Rules](https://img.shields.io/badge/Detection%20Rules-6%2C060-blue.svg)](https://github.com/thomasxm/CrowdSentinels-AI-MCP)\n\n[![Install from PyPI](https://img.shields.io/badge/Install-pip%20install%20crowdsentinel--mcp--server-3775A9?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/crowdsentinel-mcp-server/)\n\n[MCP Official Registry](https://registry.modelcontextprotocol.io/v0/servers?search=io.github.thomasxm/crowdsentinel-mcp-server) · [PyPI Package](https://pypi.org/project/crowdsentinel-mcp-server/)\n\n**Open-source threat hunting orchestrator connecting LLMs to enterprise security data via Model Context Protocol (MCP)**\n\n[Quick Start](#quick-start) · [Installation](#installation) · [CLI Usage](#cli-usage) · [Features](#key-features) · [Architecture](#architecture) · [Documentation](#documentation) · [Examples](#usage-examples)\n\n</div>\n\n> **Warning**\n> This project is in active development and intended for **security testing, research, and educational purposes only**. It is not production-ready. Do not deploy in production environments. APIs, tool interfaces, and data formats may change without notice. Use at your own risk.\n\n---\n\n## Demo\n\n\nhttps://github.com/user-attachments/assets/0d0381f0-5b68-43b2-8630-19ec130885b2\n\n\n\n---\n\n## What is CrowdSentinel?\n\n**CrowdSentinel** transforms traditional SIEM querying into intelligent, framework-driven investigations using natural language. It serves as a unified security intelligence layer that connects large language models to enterprise security data sources, enabling:\n\n- **Natural Language Threat Hunting** — Query Elasticsearch using plain English\n- **AI-Guided Investigation Workflows** — Built-in prompts guide agents through proper IR methodology\n- **Persistent Investigation State** — Memory-managed IoC tracking, forensic timelines, and cross-query correlation that survives across sessions (8GB FIFO storage)\n- **Cross-Tool IoC Correlation** — IoCs discovered in one tool are automatically available to all others\n- **Multi-Source Analysis** — Elasticsearch, EVTX logs (Chainsaw), PCAP files (Wireshark), live endpoint forensics (Velociraptor), local host forensics (osquery)\n- **Velociraptor Endpoint Forensics** — 25 MCP tools for live artefact collection (processes, network, persistence, execution evidence, NTFS MFT, SRUM, remote EVTX) with automatic IoC extraction\n- **Encoded-Command Deobfuscation** — Automatically peels base64/UTF-16LE/gzip/hex layers off encoded commands (e.g. PowerShell `-EncodedCommand`) so C2 addresses hidden inside them surface as IoCs\n- **Adaptive Schema Intelligence** — Data-driven schema registry (ECS, Sysmon, Windows Security, auditd, Zeek, Packetbeat, AWS CloudTrail) adapts hunts to each index's field conventions\n- **DFIR Knowledge Resources** — 9 MCP resources exposing investigation playbooks, Pyramid of Pain reference, and cross-correlation guidance directly to connected AI agents\n- **Standalone CLI** — Full threat hunting from the terminal without an MCP client\n\n---\n\n<a name=\"installation\"></a>\n## Installation\n\n### Install from PyPI (recommended)\n\n```bash\n# Install with pip\npip install crowdsentinel-mcp-server\n\n# Or install with uv\nuv pip install crowdsentinel-mcp-server\n\n# Download detection rules, Chainsaw, and Sigma rules (one-time)\ncrowdsentinel setup\n\n# Validate connector configuration and readiness (offline, no external calls)\ncrowdsentinel doctor\n```\n\n**Optional extras:**\n\n```bash\n# Velociraptor live endpoint forensics (adds 25 MCP tools)\npip install 'crowdsentinel-mcp-server[velociraptor]'\n\n# Long-tail deobfuscation coverage via chepy (XOR, exotic encodings).\n# The stdlib decoder core works without this — chepy only extends it.\npip install 'crowdsentinel-mcp-server[deobf]'\n```\n\nDetection rules (6,060 Lucene + EQL + ES|QL) are **bundled with the package** — no download needed. The `setup` command downloads additional tools:\n- Chainsaw binary for EVTX analysis\n- 3,000+ Sigma rules for Chainsaw\n\nDownloaded tools are stored in `~/.crowdsentinel/` and persist across package upgrades.\n\n**System dependency for PCAP analysis:**\n\n```bash\n# Required for network traffic analysis and cross-tool IoC correlation\nsudo apt install tshark    # Debian/Ubuntu/Kali\nsudo dnf install wireshark-cli  # Fedora/RHEL\nbrew install wireshark     # macOS\n```\n\n### Run directly with uvx (no install needed)\n\n```bash\n# Elasticsearch 8.x (default)\nuvx crowdsentinel-mcp-server\n\n# Other backends\nuvx crowdsentinel-mcp-server-es7   # Elasticsearch 7.x\nuvx crowdsentinel-mcp-server-es9   # Elasticsearch 9.x\nuvx opensearch-mcp-server          # OpenSearch 1.x/2.x/3.x\n```\n\n### Install from source\n\n```bash\ngit clone https://github.com/thomasxm/CrowdSentinels-AI-MCP.git\ncd CrowdSentinels-AI-MCP\nchmod +x setup.sh && ./setup.sh\n\n# Or install everything, including all optional extras (deobf/chepy, velociraptor)\n./setup.sh --full\n```\n\nThe setup script will:\n- Install dependencies (pipx, uv, Claude Code CLI if needed)\n- Bundle 6,060 detection rules and download Chainsaw binary\n- Prompt for Elasticsearch credentials (never hardcoded)\n- Configure the MCP server with Claude Code\n- Validate your connection\n\nWith `--full`, every optional extra is installed too, so all 139 MCP tools have their dependencies available — the recommended mode for a production-ready server.\n\n### Installed Size\n\nCrowdSentinel bundles 6,060 detection rules and integrates with external analysis tools. Below is the full disk space breakdown so you can plan accordingly.\n\n**Core package (via `pip` or `uvx`):**\n\n| Component | Size | Notes |\n|---|---|---|\n| **CrowdSentinel package** | **49 MB** | The server itself |\n| — Bundled Sigma rules (`src/rules/`) | 30 MB | 6,060 pre-converted detection rules |\n| — Elastic TOML rules (`src/detection-rules/`) | 17 MB | Original TOML format rules + hunting queries |\n| — Python code (clients, tools, etc.) | 2 MB | Actual application code |\n| **Dependencies** | **64 MB** | All transitive deps |\n| — `cryptography` | 14 MB | Largest dependency (TLS) |\n| — `elasticsearch` | 8.3 MB | ES Python client |\n| — `pygments` | 5.2 MB | Syntax highlighting |\n| — `pydantic_core` | 5 MB | Validation engine |\n| — `opensearchpy` | 3.6 MB | OpenSearch client |\n| — Others (27 packages) | ~28 MB | mcp, fastmcp, httpx, anthropic, etc. |\n| **Core total** | **113 MB** | `pip install crowdsentinel-mcp-server` |\n\n**Additional tools (via `crowdsentinel setup`):**\n\n| Component | Download | Installed | Notes |\n|---|---|---|---|\n| Chainsaw binary (v2.13.1) | ~3 MB | ~15 MB | EVTX log analysis engine |\n| Sigma rules (SigmaHQ) | ~3 MB | ~30 MB | 3,000+ Sigma rules for Chainsaw |\n| Chainsaw mappings | — | <1 MB | Event log source mappings |\n| **Setup total** | **~6 MB** | **~46 MB** | Stored in `~/.crowdsentinel/` |\n\n**System dependency (via package manager):**\n\n| Component | Installed | Install Command | Notes |\n|---|---|---|---|\n| tshark + Wireshark libs | ~132 MB | `sudo apt install tshark` | PCAP network analysis — required for cross-tool IoC correlation |\n\n**Full installation summary:**\n\n| Scenario | Total Disk Space |\n|---|---|\n| Core only (`pip install`) | ~113 MB |\n| Core + setup (`crowdsentinel setup`) | ~159 MB |\n| Full platform (+ tshark) | **~291 MB** |\n\n> **Note:** PyPI download size is only **8.9 MB** (wheel) thanks to compression of the bundled detection rules.\n\n---\n\n<a name=\"quick-start\"></a>\n## Quick Start\n\n### 1. Set environment variables\n\n```bash\nexport ELASTICSEARCH_HOSTS=\"https://localhost:9200\"\nexport ELASTICSEARCH_API_KEY=\"your_api_key\"\n# Or use username/password:\n# export ELASTICSEARCH_USERNAME=\"elastic\"\n# export ELASTICSEARCH_PASSWORD=\"your_password\"\nexport VERIFY_CERTS=\"false\"\n```\n\n### 2. Connect to an MCP Client\n\nCrowdSentinel works with any MCP-compatible AI agent. Choose your client below:\n\n<details>\n<summary><b>Claude Code (CLI)</b></summary>\n\n```bash\nclaude mcp add crowdsentinel \\\n  -e ELASTICSEARCH_HOSTS=\"https://localhost:9200\" \\\n  -e ELASTICSEARCH_API_KEY=\"your_api_key\" \\\n  -e VERIFY_CERTS=\"false\" \\\n  -- uvx crowdsentinel-mcp-server\n```\n</details>\n\n<details>\n<summary><b>Claude Desktop</b></summary>\n\nEdit `~/.config/Claude/claude_desktop_config.json` (Linux) or `~/Library/Application Support/Claude/claude_desktop_config.json` (macOS):\n\n```json\n{\n  \"mcpServers\": {\n    \"crowdsentinel\": {\n      \"command\": \"uvx\",\n      \"args\": [\"crowdsentinel-mcp-server\"],\n      \"env\": {\n        \"ELASTICSEARCH_HOSTS\": \"https://localhost:9200\",\n        \"ELASTICSEARCH_API_KEY\": \"your_api_key\",\n        \"VERIFY_CERTS\": \"false\"\n      }\n    }\n  }\n}\n```\n</details>\n\n<details>\n<summary><b>VS Code Copilot</b></summary>\n\nCreate `.vscode/mcp.json` in your workspace:\n\n```json\n{\n  \"servers\": {\n    \"crowdsentinel\": {\n      \"command\": \"uvx\",\n      \"args\": [\"crowdsentinel-mcp-server\"],\n      \"env\": {\n        \"ELASTICSEARCH_HOSTS\": \"https://localhost:9200\",\n        \"ELASTICSEARCH_API_KEY\": \"your_api_key\",\n        \"VERIFY_CERTS\": \"false\"\n      }\n    }\n  }\n}\n```\n\nThen enable MCP in VS Code settings: `\"chat.mcp.enabled\": true`\n</details>\n\n<details>\n<summary><b>Cursor</b></summary>\n\nCreate or edit `~/.cursor/mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"crowdsentinel\": {\n      \"command\": \"uvx\",\n      \"args\": [\"crowdsentinel-mcp-server\"],\n      \"env\": {\n        \"ELASTICSEARCH_HOSTS\": \"https://localhost:9200\",\n        \"ELASTICSEARCH_API_KEY\": \"your_api_key\",\n        \"VERIFY_CERTS\": \"false\"\n      }\n    }\n  }\n}\n```\n</details>\n\n<details>\n<summary><b>Roo Code (VS Code Extension)</b></summary>\n\nCreate `.roo/mcp.json` in your workspace:\n\n```json\n{\n  \"mcpServers\": {\n    \"crowdsentinel\": {\n      \"command\": \"uvx\",\n      \"args\": [\"crowdsentinel-mcp-server\"],\n      \"env\": {\n        \"ELASTICSEARCH_HOSTS\": \"https://localhost:9200\",\n        \"ELASTICSEARCH_API_KEY\": \"your_api_key\",\n        \"VERIFY_CERTS\": \"false\"\n      }\n    }\n  }\n}\n```\n\nOr configure via Roo Code settings panel: Settings > MCP Servers > Add Server.\n</details>\n\n<details>\n<summary><b>5ire</b></summary>\n\nIn 5ire settings (v0.15.0+), add an MCP server with:\n\n- **Command**: `uvx`\n- **Arguments**: `crowdsentinel-mcp-server`\n- **Environment Variables**:\n  - `ELASTICSEARCH_HOSTS` = `https://localhost:9200`\n  - `ELASTICSEARCH_API_KEY` = `your_api_key`\n  - `VERIFY_CERTS` = `false`\n\n> **Note:** 5ire v0.14.0 has known MCP compatibility issues. Use v0.15.0+ for reliable operation.\n</details>\n\n<details>\n<summary><b>Any MCP Client (Generic)</b></summary>\n\n**stdio transport** (default — works with most clients):\n```json\n{\n  \"mcpServers\": {\n    \"crowdsentinel\": {\n      \"command\": \"uvx\",\n      \"args\": [\"crowdsentinel-mcp-server\"],\n      \"env\": {\n        \"ELASTICSEARCH_HOSTS\": \"https://localhost:9200\",\n        \"ELASTICSEARCH_API_KEY\": \"your_api_key\",\n        \"VERIFY_CERTS\": \"false\"\n      }\n    }\n  }\n}\n```\n\n**SSE transport** (for web-based clients):\n```bash\ncrowdsentinel-mcp-server --transport sse --port 8001\n# Connect to: http://localhost:8001/sse/\n```\n\n**HTTP transport** (for REST API clients):\n```bash\ncrowdsentinel-mcp-server --transport streamable-http --port 8001\n# Connect to: http://localhost:8001/mcp/\n```\n</details>\n\n### 3. Or use the CLI directly\n\n```bash\n# Download rules and tools (one-time)\ncrowdsentinel setup\n\n# Check cluster health\ncrowdsentinel health\n\n# Hunt for threats\ncrowdsentinel hunt \"powershell encoded\" -i winlogbeat-*\n\n# Run detection rules\ncrowdsentinel rules -p windows --tactic credential_access\ncrowdsentinel detect windows_builtin_win_alert_mimikatz_keywords_lucene -i winlogbeat-*\n\n# Analyse PCAP files\ncrowdsentinel pcap overview capture.pcap\ncrowdsentinel pcap beaconing capture.pcap\n\n# Hunt EVTX logs with Chainsaw\ncrowdsentinel chainsaw hunt /path/to/evtx/ --sigma-rules /path/to/sigma/\n```\n\n---\n\n<a name=\"cli-usage\"></a>\n## CLI Usage\n\nCrowdSentinel provides a full CLI for threat hunting from the terminal:\n\n```bash\npip install crowdsentinel-mcp-server\ncrowdsentinel setup    # Download rules, Chainsaw, Sigma (one-time)\ncrowdsentinel --help\n```\n\n### Available Commands\n\n| Command | Description | Example |\n|:--------|:------------|:--------|\n| `setup` | Download detection rules, Chainsaw, and Sigma rules | `crowdsentinel setup` |\n| `health` | Show cluster health | `crowdsentinel health` |\n| `indices` | List all indices | `crowdsentinel indices` |\n| `hunt` | IR-focused threat hunt with IoC extraction | `crowdsentinel hunt \"powershell\" -i winlogbeat-*` |\n| `eql` | Execute an EQL query | `crowdsentinel eql \"process where process.name == 'cmd.exe'\" -i winlogbeat-*` |\n| `esql` | Execute an ES\\|QL query | `crowdsentinel esql \"FROM logs-* \\| LIMIT 10\"` |\n| `detect` | Execute a detection rule by ID | `crowdsentinel detect win_susp_logon -i winlogbeat-*` |\n| `rules` | List available detection rules | `crowdsentinel rules -p windows --tactic credential_access --type eql` |\n| `schema` | Detect schema for an index pattern | `crowdsentinel schema -i winlogbeat-*` |\n| `ioc` | Hunt for a specific Indicator of Compromise | `crowdsentinel ioc 203.0.113.42 --type ip -i winlogbeat-*` |\n| `analyse` | Analyse search results from stdin (JSON) | `cat results.json \\| crowdsentinel analyse -c \"context\"` |\n| `analyse --mcp` | AI agent analysis using all 139 MCP tools | `crowdsentinel hunt \"query\" \\| crowdsentinel analyse --mcp -c \"context\"` |\n| `auth` | Manage LLM authentication for agent mode | `crowdsentinel auth login` |\n| `pcap` | Analyse PCAP files (overview, beaconing, lateral movement) | `crowdsentinel pcap beaconing capture.pcap` |\n| `chainsaw` | Hunt EVTX logs with Chainsaw and Sigma rules | `crowdsentinel chainsaw hunt /path/to/evtx/` |\n| `doctor` | Validate connector configuration and readiness (offline — no external calls) | `crowdsentinel doctor` |\n\n### Output Formats\n\nAll commands support `--output/-o` with three formats:\n\n```bash\ncrowdsentinel hunt \"failed login\" -i winlogbeat-* -o json     # Structured JSON (default)\ncrowdsentinel hunt \"failed login\" -i winlogbeat-* -o table    # Human-readable table\ncrowdsentinel hunt \"failed login\" -i winlogbeat-* -o summary  # Condensed summary\n```\n\n### Agent Mode (`--mcp`)\n\nThe `analyse --mcp` flag replaces deterministic analysis with an AI agent that autonomously uses all 139 MCP tools to investigate. The agent follows the 4-phase IR methodology: hunt, analyse, correlate, report.\n\n**Authentication:**\n\n```bash\n# Option 1: Browser sign-in (ChatGPT subscription — no API billing)\ncrowdsentinel auth login\n\n# Option 2: Anthropic (setup-token or API key)\ncrowdsentinel auth login --provider anthropic\n\n# Option 3: Environment variable\nexport ANTHROPIC_API_KEY=\"sk-ant-...\"   # or OPENAI_API_KEY\n\n# Option 4: Local models (Ollama, vLLM — free)\ncrowdsentinel analyse --mcp --model-url http://localhost:11434/v1 --model llama3.1\n\n# Check auth status\ncrowdsentinel auth status\n```\n\n**Agent flags:**\n\n| Flag | Default | Description |\n|:-----|:--------|:------------|\n| `--mcp` | off | Enable AI agent with MCP tools |\n| `--mcp-server NAME:CMD` | none | Add external MCP server (e.g., VirusTotal) |\n| `--model` | auto-detect | LLM model to use |\n| `--model-url` | none | OpenAI-compatible API endpoint |\n| `--max-steps` | 30 | Maximum tool calls |\n| `--timeout` | 300 | Maximum seconds |\n\n### Pipeline Examples\n\n**Deterministic analysis (no API key needed):**\n\n```bash\n# Hunt then analyse\ncrowdsentinel hunt \"powershell encoded\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse -c \"Encoded PowerShell commands\" -o summary\n\n# Investigate failed authentication attempts\ncrowdsentinel hunt \"event.code:4625\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse -c \"Failed login brute force investigation\" -o summary\n\n# Triage process execution and privilege escalation\ncrowdsentinel hunt \"event.code:4688 OR event.code:4672 OR event.code:1\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse -c \"Process execution and privilege escalation\" -o summary\n```\n\n**AI agent investigation (requires auth):**\n\n```bash\n# Credential dumping investigation — agent hunts, analyses kill chain, checks adjacent stages\ncrowdsentinel hunt \"mimikatz OR lsass OR procdump\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse --mcp -c \"Credential dumping tools investigation\" --max-steps 15 -o summary\n\n# Encoded PowerShell — full IR workflow with kill chain and adjacent stage hunting\ncrowdsentinel hunt \"powershell -enc OR FromBase64String\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse --mcp -c \"Full IR workflow: encoded PowerShell\" --max-steps 30 -o table\n\n# Process execution with detection rules\ncrowdsentinel hunt \"event.code:4688\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse --mcp -c \"Execute detection rules against process creation\" --max-steps 20 -o summary\n\n# PCAP beaconing — agent generates IoCs and maps to kill chain\ncrowdsentinel pcap beaconing capture.pcap -o json | \\\n  crowdsentinel analyse --mcp -c \"Investigate beaconing for C2 infrastructure\" --max-steps 10 -o summary\n\n# Anti-forensics investigation\ncrowdsentinel hunt \"event.code:1102\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse --mcp -c \"Security log cleared - anti-forensics\" --max-steps 10 -o summary\n\n# With external MCP server (e.g., VirusTotal)\ncrowdsentinel hunt \"powershell\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse --mcp --mcp-server \"vt:uvx virustotal-mcp-server\" \\\n  -c \"Check IoCs against VirusTotal\" -o summary\n```\n\n---\n\n<a name=\"key-features\"></a>\n## Key Features\n\n<table>\n<tr>\n<td width=\"50%\">\n\n### 139 MCP Tools\nThreat hunting, detection rules, forensics, endpoint collection, network analysis, cross-correlation, command deobfuscation, and IoC enrichment — all accessible via natural language\n\n### 6,060 Detection Rules\nPre-built Lucene, EQL & ES|QL rules with automatic MITRE ATT&CK mapping\n\n### Investigation State\nPersistent IoC tracking across tools and sessions with cross-source correlation and FIFO storage\n\n</td>\n<td width=\"50%\">\n\n### 4 Security Frameworks\n- Cyber Kill Chain (7 stages)\n- Pyramid of Pain (6 levels)\n- Diamond Model (4 vertices)\n- MITRE ATT&CK (automatic mapping)\n\n### 5 Data Sources + Threat Intel\n- Elasticsearch / OpenSearch (SIEM)\n- Velociraptor (live endpoint forensics)\n- osquery (local host forensics)\n- EVTX logs (Chainsaw + Sigma)\n- PCAP files (Wireshark/TShark)\n- IoC enrichment (Shodan, VirusTotal, AbuseIPDB, ThreatFox)\n- MISP (export events, search attributes)\n\n</td>\n</tr>\n</table>\n\n---\n\n<a name=\"architecture\"></a>\n## Architecture\n\n```\n┌─────────────────────────────────────────────────────────────────┐\n│                 LLM Client / Claude Code CLI                    │\n└─────────────────────────────┬───────────────────────────────────┘\n                              │ MCP Protocol (stdio/SSE/HTTP)\n                              │ HTTP: Origin validation (anti DNS-rebinding)\n                              │       + transport-level audit logging\n                              ▼\n┌─────────────────────────────────────────────────────────────────┐\n│                    CrowdSentinel MCP Server                     │\n│  ┌───────────────┐ ┌───────────────┐ ┌───────────────────────┐  │\n│  │  139 Tools    │ │ 6,060 Rules   │ │ Security Frameworks   │  │\n│  │ - Hunting     │ │ - Lucene      │ │ - Cyber Kill Chain    │  │\n│  │ - Detection   │ │ - EQL         │ │ - Pyramid of Pain     │  │\n│  │ - Forensics   │ │ - ES|QL       │ │ - Diamond Model       │  │\n│  │ - Endpoint    │ │               │ │ - MITRE ATT&CK        │  │\n│  │ - Network     │ ├───────────────┤ │                       │  │\n│  │ - Enrichment  │ │ Schema        │ │ 9 MCP Resources       │  │\n│  │ - Correlation │ │ Registry      │ │ (DFIR knowledge base) │  │\n│  │ - Deobfuscate │ │ (data-driven) │ │                       │  │\n│  └───────────────┘ └───────────────┘ └───────────────────────┘  │\n│  ┌─────────────────────────────────────────────────────────────┐│\n│  │              Investigation State (Persistent)               ││\n│  │    Cross-source IoC sharing, auto-capture, STIX 2.1 export  ││\n│  └─────────────────────────────────────────────────────────────┘│\n└─────┬──────────┬──────────┬──────────────┬─────────────┬────────┘\n      │          │          │              │             │\n      ▼          ▼          ▼              ▼             ▼\n┌───────────┐ ┌──────────┐ ┌────────┐ ┌───────────┐ ┌───────────┐\n│Elastic-   │ │Velocirap-│ │osquery │ │ Chainsaw  │ │ Wireshark │\n│search/    │ │tor       │ │(local  │ │(EVTX/     │ │  (PCAP)   │\n│OpenSearch │ │(EDR/DFIR)│ │ host)  │ │ Sigma)    │ │ (Network) │\n│  (SIEM)   │ │(Endpoint)│ │        │ │(Offline)  │ │           │\n└───────────┘ └──────────┘ └────────┘ └───────────┘ └───────────┘\n                          │\n  ┌───────────────────────┼───────────────────────┐\n  ▼                       ▼                       ▼\n┌───────────────┐ ┌───────────────┐ ┌───────────────────────────┐\n│ Shodan        │ │  VirusTotal   │ │  AbuseIPDB + ThreatFox    │\n│ InternetDB    │ │     (v3)      │ │  + MISP (export/search)   │\n│  (free/no key)│ │  (free tier)  │ │  (free tier)              │\n└───────────────┘ └───────────────┘ └───────────────────────────┘\n                          │\n                          ▼ (Roadmap)\n┌───────────────┐ ┌───────────────┐ ┌───────────────┐\n│    Splunk     │ │     Wazuh     │ │     Zeek      │\n│               │ │  (EDR/XDR)    │ │   (NSM/IDS)   │\n└───────────────┘ └───────────────┘ └───────────────┘\n```\n\n---\n\n## What's Included\n\n### Tool Categories (139 Tools)\n\n| Category | Tools | Description |\n|:---------|:-----:|:------------|\n| **Elasticsearch Core** | 19 | Index, document, cluster, alias, data stream, raw API operations |\n| **Threat Hunting** | 8 | Attack pattern detection, kill-chain stage hunting, timeframe hunts, anomaly detection |\n| **EQL & ES\\|QL Hunting** | 11 | Event-sequence queries, ES\\|QL pipes, saved hunt history, capability detection |\n| **Asset Discovery & Schema Intelligence** | 12 | Index discovery, data-driven schema detection (ECS, Sysmon, auditd, Zeek, CloudTrail…), smart search |\n| **IoC Analysis & Deobfuscation** | 6 | IoC extraction, encoded-command decoding, kill-chain mapping, investigation reports |\n| **Detection Rules** | 9 | 6,060 rule library — list, execute, validate, suggest, batch execution |\n| **Investigation Prompts** | 5 | Fast triage spine — 10 critical IR questions |\n| **Chainsaw (EVTX)** | 5 | Sigma rule hunting, iterative IoC discovery |\n| **Wireshark (PCAP)** | 11 | Network forensics, beaconing, lateral movement detection, traffic decoding |\n| **Threat Intelligence** | 5 | IoC enrichment (Shodan, VirusTotal, AbuseIPDB, ThreatFox) + MISP export/search |\n| **Investigation State** | 10 | Persistent IoCs, cross-tool sharing, STIX 2.1 export, reporting |\n| **Velociraptor (Endpoint)** | 25 | Live forensic artefact collection — processes, network, persistence, execution evidence, MFT, SRUM |\n| **osquery (Local Host)** | 4 | SQL-based local host forensics — processes, sockets, users, ad-hoc queries |\n| **Cross-Source Correlation** | 5 | SIEM ↔ endpoint pivots, IoC-driven endpoint hunts, unified timelines, time correlation |\n| **Workflow Guidance & Status** | 4 | Next-step guidance, progressive disclosure, system status, storage cleanup |\n\n> Velociraptor tools register when `VELOCIRAPTOR_API_CONFIG` is set; osquery tools register when a local `osqueryi` binary is present; the 10 high-risk write tools are hidden when `DISABLE_HIGH_RISK_OPERATIONS=true`.\n\n### Security Frameworks\n\n| Framework | Purpose |\n|:----------|:--------|\n| **Cyber Kill Chain** | Hunt by attack stage (7 stages), predict adversary's next move |\n| **Pyramid of Pain** | Prioritise IoCs by difficulty for attackers to change (6 levels) |\n| **Diamond Model** | Map adversary, capability, infrastructure, victim relationships |\n| **MITRE ATT&CK** | Automatic technique mapping for all detections |\n\n### Detection Rules (6,060 Rules)\n\n| Type | Count | Source | Description |\n|:-----|------:|:-------|:------------|\n| **Lucene** | 1,966 | Sigma-converted | Fast pattern matching queries |\n| **EQL** | 3,963 | Sigma-converted + Elastic | Event sequences and correlations |\n| **ES\\|QL** | 131 | Elastic TOML rules | Pipe-based query language (ES 8.11+) |\n\n**Platforms:** Windows, Linux, macOS, Cloud (AWS/Azure/GCP), Network, Identity\n\n**Log Sources:** PowerShell, Sysmon, Security Events, Process Creation, Audit logs\n\n### Encoded-Command Deobfuscation\n\nReal intrusions hide C2 addresses and URLs inside encodings the SIEM stores verbatim — most commonly PowerShell `-EncodedCommand` (base64 of UTF-16LE), but also `FromBase64String`, base64→gzip, bare base64, and hex blobs. CrowdSentinel decodes these layers automatically during IoC extraction and via the dedicated `decode_command` tool, so the inner indicators surface instead of staying buried:\n\n- **Zero extra dependencies** — the decoder core uses only the Python standard library (`base64`/`zlib`/`codecs`/`binascii`)\n- **Optional long tail** — install the `deobf` extra (`pip install 'crowdsentinel-mcp-server[deobf]'`) to add [chepy](https://github.com/securisec/chepy) for XOR and exotic encodings; its absence degrades gracefully\n- **Provenance** — every decode records its recipe (e.g. `base64->utf-16le`) for the IR report\n- **Safe by design** — decode-only (nothing is executed), with depth and output-size bounds so a decompression bomb cannot hang the server\n\n---\n\n## Configuration\n\n### Environment Variables\n\n```bash\n# Connection (required — choose one)\nELASTICSEARCH_HOSTS=\"https://localhost:9200\"       # Self-hosted\n# OR\nELASTICSEARCH_CLOUD_ID=\"deployment:base64...\"      # Elastic Cloud\n\n# Authentication — choose one (in priority order):\nELASTICSEARCH_BEARER_TOKEN=\"service_token_here\"    # Service/bearer token\nELASTICSEARCH_API_KEY=\"your_api_key\"               # API key (recommended)\nELASTICSEARCH_USERNAME=\"elastic\"                   # Basic auth\nELASTICSEARCH_PASSWORD=\"your_password\"\n\n# TLS / Certificate verification\nVERIFY_CERTS=\"true\"                                # Verify against system CA bundle\n# VERIFY_CERTS=\"/path/to/ca.crt\"                   # Verify against custom CA certificate\n# ELASTICSEARCH_CA_CERT=\"/path/to/ca.crt\"          # Explicit CA certificate path\n# ELASTICSEARCH_CLIENT_CERT=\"/path/to/client.crt\"  # Client certificate (mTLS)\n# ELASTICSEARCH_CLIENT_KEY=\"/path/to/client.key\"   # Client private key (mTLS)\n\n# Options\nREQUEST_TIMEOUT=\"30\"                               # Request timeout in seconds\nDISABLE_HIGH_RISK_OPERATIONS=\"true\"                # Block all write operations\n\n# HTTP/SSE transport security (stdio unaffected)\n# MCP_ALLOWED_ORIGINS=\"https://app.example.com\"    # Extra allowed Origins (comma-separated); loopback always allowed\n\n# Velociraptor endpoint forensics (optional — enables 25 extra tools)\n# VELOCIRAPTOR_API_CONFIG=\"/path/to/api_client.yaml\"\n\n# Threat Intelligence (optional — Shodan InternetDB works without any key)\nVIRUSTOTAL_API_KEY=\"your_vt_key\"                   # Free: 500 lookups/day\nABUSEIPDB_API_KEY=\"your_abuse_key\"                 # Free: 1,000 lookups/day\nTHREATFOX_API_KEY=\"your_tf_key\"                    # Free: unlimited\n\n# MISP Integration (optional — offline JSON export works without a server)\nMISP_URL=\"https://misp.example.org\"                # MISP instance URL\nMISP_API_KEY=\"your_misp_key\"                       # MISP API key (40-char hex)\n# MISP_SSL_VERIFY=\"true\"                           # Set \"false\" for self-signed (dev only)\n```\n\n> **Security Warning:** Never use `VERIFY_CERTS=\"false\"` or plain-text passwords in production. Use API keys or service tokens with TLS certificate verification enabled. For self-signed certificates, set `ELASTICSEARCH_CA_CERT` to your CA certificate path.\n\n### Production Configuration Examples\n\n<details>\n<summary><b>Elastic Cloud</b></summary>\n\n```bash\nELASTICSEARCH_CLOUD_ID=\"my-deployment:dXMtY2VudHJhbC0x...\"\nELASTICSEARCH_API_KEY=\"your_cloud_api_key\"\nVERIFY_CERTS=\"true\"\n```\n</details>\n\n<details>\n<summary><b>Self-Hosted with Custom CA</b></summary>\n\n```bash\nELASTICSEARCH_HOSTS=\"https://es-cluster.internal:9200\"\nELASTICSEARCH_API_KEY=\"your_api_key\"\nELASTICSEARCH_CA_CERT=\"/etc/elasticsearch/certs/ca.crt\"\nVERIFY_CERTS=\"true\"\n```\n</details>\n\n<details>\n<summary><b>Mutual TLS (mTLS)</b></summary>\n\n```bash\nELASTICSEARCH_HOSTS=\"https://es-cluster.internal:9200\"\nELASTICSEARCH_CA_CERT=\"/etc/elasticsearch/certs/ca.crt\"\nELASTICSEARCH_CLIENT_CERT=\"/etc/elasticsearch/certs/client.crt\"\nELASTICSEARCH_CLIENT_KEY=\"/etc/elasticsearch/certs/client.key\"\nVERIFY_CERTS=\"true\"\n```\n</details>\n\n<details>\n<summary><b>Development / Testing (insecure)</b></summary>\n\n```bash\nELASTICSEARCH_HOSTS=\"http://localhost:9200\"\nELASTICSEARCH_USERNAME=\"elastic\"\nELASTICSEARCH_PASSWORD=\"test123\"\nVERIFY_CERTS=\"false\"\n```\n\n> This configuration is **not suitable for production**. Use API keys with TLS in production environments.\n</details>\n\n### Transport Modes\n\n| Mode | Use Case | Command |\n|:-----|:---------|:--------|\n| **stdio** (default) | MCP clients (Claude Code, etc.) | Auto-configured by setup.sh |\n| **SSE** | Web/streaming clients | `--transport sse --port 8000` |\n| **HTTP** | REST API access | `--transport streamable-http` |\n\nThe HTTP transports are hardened at the transport layer:\n\n- **Origin validation (anti DNS-rebinding)** — browser requests are rejected unless their `Origin` is loopback or listed in `MCP_ALLOWED_ORIGINS` (comma-separated; `*` alone disables the check). Requests without an `Origin` header — every non-browser MCP client — are unaffected.\n- **Audit logging** — every HTTP exchange is logged at the transport boundary (JSON-RPC method, tool name, body size, path), including requests the tool layer never sees, such as unknown tools or protocol errors. Logs are metadata-only, memory-bounded, and sanitised against log forgery.\n\n### Disabling Write Operations\n\nFor read-only access (recommended for production):\n\n```bash\nDISABLE_HIGH_RISK_OPERATIONS=\"true\"\n```\n\nThis hides destructive tools: `create_index`, `delete_index`, `index_document`, `delete_document`, `delete_by_query`, `create_data_stream`, `delete_data_stream`, `put_alias`, `delete_alias`, `general_api_request`\n\n---\n\n<a name=\"usage-examples\"></a>\n## Usage Examples\n\n### Example 1: Hunt for Brute Force Attacks\n\n```\nYou: Hunt for brute force attacks in the last hour\n\nCrowdSentinel: Searching for failed authentication attempts...\n[Calls hunt_by_timeframe with attack_types=[\"brute_force\"]]\n\nFound 47 failed login attempts from 3 source IPs targeting the \"admin\" account.\nTop sources:\n- 192.168.1.105: 28 attempts\n- 10.0.0.42: 15 attempts\n- 172.16.0.8: 4 attempts\n```\n\n### Example 2: Investigate a Suspicious Host\n\n```\nYou: Show me all activity on host DESKTOP-ABC123 in the last 24 hours\n\nCrowdSentinel: Generating a forensic timeline for that host...\n[Calls get_host_activity_timeline]\n\nTimeline shows:\n- 09:15: User \"jsmith\" logged in (Event 4624)\n- 09:17: PowerShell executed encoded command\n- 09:18: New scheduled task created \"WindowsUpdate\"\n- 09:22: Outbound connection to 203.0.113.42:443\n- 09:25: LSASS memory access detected\n```\n\n### Example 3: Kill Chain Analysis\n\n```\nYou: We found C2 beaconing. What should we hunt for next?\n\nCrowdSentinel: Hunting adjacent kill chain stages...\n[Calls hunt_adjacent_stages with current_stage=\"COMMAND_AND_CONTROL\"]\n\nLooking backwards (Installation stage):\n- Found scheduled task persistence on 3 hosts\n- Registry Run key modifications detected\n\nLooking forwards (Actions on Objectives):\n- Lateral movement to 2 additional hosts via SMB\n- Credential dumping attempts detected\n```\n\n### Example 4: Network Traffic Analysis\n\n```\nYou: Analyse this PCAP for C2 beaconing\n\nCrowdSentinel: Checking for regular connection intervals...\n[Calls detect_beaconing on the PCAP file]\n\nDetected beaconing pattern:\n- Host: 192.168.1.50 → 203.0.113.100:443\n- Interval: 60 seconds (±2s jitter)\n- Duration: 4 hours\n- Confidence: HIGH\n\nRecommend blocking 203.0.113.100 and investigating 192.168.1.50.\n```\n\n### Example 5: CLI Threat Hunt\n\n```bash\n# Hunt for encoded PowerShell\ncrowdsentinel hunt \"powershell -enc\" -i winlogbeat-* --timeframe 1440 -o json\n\n# Pipe results to analysis\ncrowdsentinel hunt \"event.code:4625\" -i winlogbeat-* -o json | \\\n  crowdsentinel analyse -c \"Failed authentication investigation\"\n\n# Search detection rules for lateral movement\ncrowdsentinel rules --tactic lateral_movement -p windows\n```\n\n---\n\n## Compatibility\n\n| Package | Backend | Install |\n|:--------|:--------|:--------|\n| `crowdsentinel-mcp-server` | Elasticsearch 8.x (default) | `pip install crowdsentinel-mcp-server` |\n| `crowdsentinel-mcp-server-es7` | Elasticsearch 7.x | `pip install crowdsentinel-mcp-server-es7` |\n| `crowdsentinel-mcp-server-es9` | Elasticsearch 9.x | `pip install crowdsentinel-mcp-server-es9` |\n| `opensearch-mcp-server` | OpenSearch 1.x, 2.x, 3.x | `pip install opensearch-mcp-server` |\n\n---\n\n## For Developers\n\n<details>\n<summary><b>Project Structure</b></summary>\n\n```\ncrowdsentinel-mcp-server/\n├── src/\n│   ├── server.py                 # MCP server entry point\n│   ├── version.py                # Version constant\n│   ├── risk_config.py            # Write operation controls\n│   ├── transport_security.py     # Origin validation for HTTP transports (anti DNS-rebinding)\n│   ├── transport_audit.py        # Transport-level audit logging (every HTTP exchange)\n│   │\n│   ├── cli/                      # Standalone CLI\n│   │   └── main.py               # CLI entry point (argparse) — 15 commands incl. doctor\n│   │\n│   ├── agent/                    # `analyse --mcp` AI agent mode (multi-provider LLM auth)\n│   ├── config/                   # Connector configuration\n│   │\n│   ├── clients/                  # Backend logic layer\n│   │   ├── base.py               # Base client, authentication\n│   │   ├── exceptions.py         # Exception handling decorators\n│   │   └── common/\n│   │       ├── client.py         # Unified SearchClient (multiple inheritance)\n│   │       ├── threat_hunting.py # Threat hunting queries\n│   │       ├── ioc_analysis.py   # IoC extraction & analysis\n│   │       ├── deobfuscate.py    # Encoded-command decoding (stdlib core + optional chepy)\n│   │       ├── cyber_kill_chain.py # Kill chain logic\n│   │       ├── rule_loader.py    # Detection rule loading\n│   │       ├── chainsaw_client.py # EVTX/Sigma integration\n│   │       ├── osquery_client.py # Local osquery execution (sqlglot-validated SQL)\n│   │       ├── misp_client.py    # MISP export/search\n│   │       ├── threat_intel.py   # Shodan/VirusTotal/AbuseIPDB/ThreatFox enrichment\n│   │       └── schemas/          # Data-driven schema registry\n│   │           ├── ecs.py, sysmon.py, windows_security.py, auditd.py\n│   │           └── defs/         # JSON schema defs (zeek, packetbeat, aws_cloudtrail, …)\n│   │\n│   ├── tools/                    # MCP tool interfaces (25 modules, 139 tools)\n│   │   ├── register.py           # Dynamic tool registration\n│   │   ├── threat_hunting.py     # Hunting tool definitions\n│   │   ├── rule_management.py    # Rule management tools\n│   │   ├── velociraptor_tools.py # Live endpoint forensics (25 tools)\n│   │   ├── osquery_tools.py      # Local host forensics (4 tools)\n│   │   ├── cross_correlation.py  # SIEM ↔ endpoint correlation (5 tools)\n│   │   ├── chainsaw_hunting.py   # Chainsaw tools\n│   │   ├── wireshark_tools.py    # Network analysis tools\n│   │   ├── threat_intel.py       # Enrichment + MISP tools\n│   │   └── investigation_state_tools.py # State management tools\n│   │\n│   ├── storage/                  # Persistent investigation state\n│   │   ├── investigation_state.py # Core state management\n│   │   ├── storage_manager.py    # File system storage (8GB FIFO)\n│   │   └── models.py             # Pydantic models (IoC, Investigation)\n│   │\n│   ├── utils/                    # Shared utilities\n│   │\n│   └── wireshark/                # Network traffic analysis\n│       ├── core/                 # TShark execution, PCAP parsing\n│       ├── hunting/              # Beaconing, lateral movement, IoC hunting\n│       ├── baseline/             # Traffic baseline creation\n│       ├── extraction/           # File carving from traffic\n│       └── reporting/            # NCSC-style reports, timelines\n│\n├── rules/                        # 6,060 detection rules (EQL + Lucene)\n├── chainsaw/                     # Chainsaw binary + 3,000+ Sigma rules\n├── skills/                       # Claude Code agent skills\n├── Makefile                      # Test pyramid + release automation\n└── tests/                        # Tiered test suite (unit / integration / api / e2e / system)\n```\n\n</details>\n\n<details>\n<summary><b>Design Patterns</b></summary>\n\n| Pattern | Usage |\n|:--------|:------|\n| **Multiple Inheritance** | `SearchClient` composes all specialised clients |\n| **Decorator** | Exception handling via `@handle_exceptions` |\n| **Factory** | `create_search_client()` creates appropriate client |\n| **Plugin Architecture** | Tools registered dynamically via `ToolsRegister` |\n| **Auto-Capture** | Tool results automatically analysed for IoCs |\n\n</details>\n\n<details>\n<summary><b>Adding a New Tool</b></summary>\n\n1. **Create client method** in `src/clients/common/your_module.py`:\n```python\nclass YourClient(SearchClientBase):\n    def your_method(self, param: str) -> dict:\n        # Implementation\n        return results\n```\n\n2. **Add to SearchClient** in `src/clients/common/client.py`:\n```python\nclass SearchClient(YourClient, OtherClients, ...):\n    pass\n```\n\n3. **Create tool wrapper** in `src/tools/your_tools.py`:\n```python\nclass YourTools:\n    def __init__(self, client, mcp):\n        self.client = client\n        self.mcp = mcp\n\n    def register_tools(self):\n        @self.mcp.tool()\n        def your_tool(param: str) -> str:\n            \"\"\"Tool description for LLM.\"\"\"\n            result = self.client.your_method(param)\n            return json.dumps(result)\n```\n\n4. **Register in server** in `src/server.py`:\n```python\nfrom src.tools.your_tools import YourTools\n\ndef _register_tools(self):\n    # ... existing tools ...\n    YourTools(self.client, self.mcp).register_tools()\n```\n\n</details>\n\n<details>\n<summary><b>Running Tests</b></summary>\n\nThe test suite is a bottom-up pyramid (see `tests/README.md`): hermetic unit tests, then live tiers that spin up a real Elasticsearch (via [testcontainers](https://testcontainers-python.readthedocs.io/)) and the real MCP server.\n\n```bash\nmake test-unit          # fast hermetic loop (default developer command)\nmake coverage           # unit tier + coverage policy gate\nmake check-tests        # policy: every new function vs origin/main has a test\nmake test-integration   # real ES via testcontainers (Docker required)\nmake test-api           # + real MCP server subprocess over streamable-http\nmake test-e2e           # + real MCP client journey over the wire\nmake test-suite         # the whole pyramid, bottom-up, with a summary table\n```\n\n| Tier | What it proves | Backing services |\n|:-----|:---------------|:-----------------|\n| Unit | Function-level behaviour, fail-fast validation | None (hermetic) |\n| Integration | App ↔ Elasticsearch trust boundary (auth, TLS, seeded data) | Real ES via testcontainers |\n| API | MCP-over-HTTP contract (schemas, errors, Origin security, audit logs) | Real ES + real server subprocess |\n| E2E | Full analyst journey via a real MCP client | Real ES + real server + `fastmcp.Client` |\n| System | Full pipeline against your own range cluster | BYO Elasticsearch |\n\nLive tiers are opt-in (`--run-integration` / `--run-api` / `--run-e2e` / `--run-system`), so plain `pytest tests/` stays fast and hermetic.\n\n</details>\n\n<details>\n<summary><b>Local Testing Environment</b></summary>\n\n```bash\n# Start Elasticsearch\ndocker-compose -f docker-compose-elasticsearch.yml up -d\n\n# Start OpenSearch\ndocker-compose -f docker-compose-opensearch.yml up -d\n```\n\n**Default credentials (testing only):**\n- Elasticsearch: `elastic` / `test123`\n- OpenSearch: `admin` / `admin`\n\n</details>\n\n---\n\n## Roadmap\n\n| Feature | Status | Description |\n|:--------|:------:|:------------|\n| **Velociraptor Integration** | **Done** | 25 live endpoint forensics tools + 5 cross-correlation tools via the Velociraptor API |\n| **osquery Integration** | **Done** | SQL-based local host forensics (processes, sockets, users, ad-hoc queries) |\n| **Encoded-Command Deobfuscation** | **Done** | Recover IoCs hidden in base64/UTF-16LE/gzip/hex-encoded commands (optional chepy for the long tail) |\n| **Threat Intel Enrichment** | **Done** | IoC enrichment via Shodan InternetDB, VirusTotal, AbuseIPDB, ThreatFox + STIX 2.1 export |\n| **MISP Integration** | **Done** | Export investigation IoCs as MISP events, search MISP attributes (offline JSON export without a server) |\n| **Zeek Integration** | Partial | Zeek logs indexed into ES are hunted via the data-driven schema registry; native NSM integration planned |\n| **Wazuh Integration** | Planned | Open-source EDR/XDR — its indexer is OpenSearch-based, so CrowdSentinel's existing OpenSearch client can hunt Wazuh alerts and agent telemetry |\n| **Splunk Integration** | Planned | Add Splunk as a data source alongside Elasticsearch |\n| **Sigma Rule Converter** | Planned | Convert Sigma rules to native ES/Splunk queries |\n| **Case Management** | Planned | Export investigations to TheHive, JIRA |\n| **Custom Rule Builder** | Planned | Create detection rules via natural language |\n\nSee [CHANGELOG.md](CHANGELOG.md) for detailed version history.\n\n---\n\n## Velociraptor Integration (Live Endpoint Forensics)\n\nCrowdSentinel integrates with [Velociraptor](https://docs.velociraptor.app/) for live endpoint forensic artefact collection. When configured, 25 additional MCP tools and 5 cross-correlation tools become available.\n\n### Setup\n\n```bash\n# Install optional dependencies\npip install crowdsentinel-mcp-server[velociraptor]\n\n# Set the Velociraptor API config path\nexport VELOCIRAPTOR_API_CONFIG=\"/path/to/api_client.yaml\"\n```\n\nThe API client config is generated from the Velociraptor server:\n```bash\nvelociraptor config api_client --name crowdsentinel --role administrator,api \\\n  --config server.config.yaml /path/to/api_client.yaml\n```\n\n### Available Tools\n\n| Category | Tools | Use Case |\n|:---------|:------|:---------|\n| **Discovery** | `velociraptor_client_info`, `velociraptor_list_clients`, `velociraptor_list_artifacts` | Resolve hostnames, enumerate enrolled endpoints, discover available artefacts |\n| **Generic Collection** | `velociraptor_collect_artifact`, `velociraptor_get_collection_results` | Collect any Velociraptor artefact and fetch its results |\n| **Live State** | `velociraptor_pslist`, `velociraptor_netstat`, `velociraptor_users`, `velociraptor_groups`, `velociraptor_mounts` | Running processes, active network connections, accounts, groups, mounted volumes |\n| **Execution Evidence** | `velociraptor_prefetch`, `velociraptor_amcache`, `velociraptor_shimcache`, `velociraptor_userassist`, `velociraptor_bam`, `velociraptor_srum` | Forensic proof of programme execution and resource usage |\n| **Persistence** | `velociraptor_services`, `velociraptor_scheduled_tasks`, `velociraptor_autoruns` | Service, scheduled task, and autorun persistence mechanisms |\n| **User Activity** | `velociraptor_shellbags`, `velociraptor_recentdocs`, `velociraptor_lnk`, `velociraptor_evidence_of_download` | User browsing, document access, link files, file downloads |\n| **Filesystem & Logs** | `velociraptor_ntfs_mft`, `velociraptor_evtx` | MFT search by name/path/timestamp, remote EVTX queries |\n| **Cross-Correlation** | `correlate_siem_with_endpoint`, `endpoint_to_siem_pivot`, `endpoint_hunt_from_ioc`, `correlate_by_time`, `build_unified_timeline` | Validate SIEM findings on endpoints, pivot from endpoint IoCs to fleet-wide SIEM searches, unified timelines |\n\n### Cross-Source IoC Sharing\n\nIoCs extracted from Velociraptor artefacts are automatically captured into the active investigation alongside SIEM, Chainsaw, and Wireshark findings. Use `get_shared_iocs` to retrieve the combined indicator set for cross-source correlation.\n\n```\nSIEM (Elasticsearch)          Endpoint (Velociraptor)\n       │                              │\n  hunt_by_timeframe()         velociraptor_pslist()\n       │                              │\n  auto_capture ──────► Investigation State ◄────── auto_capture\n                              │\n                     get_shared_iocs()\n```\n\n### DFIR Knowledge Resources\n\n9 MCP resources provide structured investigation reference data directly to connected AI agents:\n\n| Resource | Content |\n|:---------|:--------|\n| `crowdsentinel://investigation-workflow` | Mandatory 4-phase IR workflow |\n| `crowdsentinel://ioc-reference` | IoC types ranked by Pyramid of Pain |\n| `crowdsentinel://ioc-reference/data` | Machine-readable IoC reference data |\n| `crowdsentinel://cross-correlation-playbooks` | 5 investigation playbooks (suspicious process, brute force, lateral movement, persistence, exfiltration) |\n| `crowdsentinel://velociraptor-guide` | Artefact reference tables and \"found X in SIEM → check Y on endpoint\" decision tree |\n| `crowdsentinel://data-sources` | All data source capabilities and investigation decision matrix |\n| `crowdsentinel://tool-recommendations` | Which tool to reach for at each investigation step |\n| `crowdsentinel://schemas` | Supported log-source schemas (ECS, Sysmon, Windows Security, auditd, Zeek, …) |\n| `crowdsentinel://schemas/list` | Machine-readable schema listing |\n\n---\n\n<a name=\"documentation\"></a>\n## Documentation\n\n### User Guides\n\n| Document | Description |\n|:---------|:------------|\n| [FIRST_TIME_SETUP.md](FIRST_TIME_SETUP.md) | Detailed first-time setup instructions |\n| [HOW_TO_USE.md](HOW_TO_USE.md) | Comprehensive usage guide |\n| [QUICK_START.md](QUICK_START.md) | 5-minute quick start |\n| [TRANSPORT_MODES.md](TRANSPORT_MODES.md) | stdio, SSE, HTTP configuration |\n\n### Feature Guides\n\n| Document | Description |\n|:---------|:------------|\n| [THREAT_HUNTING_GUIDE.md](THREAT_HUNTING_GUIDE.md) | Threat hunting workflows |\n| [DETECTION_RULES_GUIDE.md](DETECTION_RULES_GUIDE.md) | Using 6,060 detection rules |\n| [CYBER_KILL_CHAIN_GUIDE.md](CYBER_KILL_CHAIN_GUIDE.md) | Kill chain analysis |\n| [CHAINSAW_GUIDE.md](CHAINSAW_GUIDE.md) | EVTX log analysis with Sigma |\n| [INVESTIGATION_PROMPTS_GUIDE.md](INVESTIGATION_PROMPTS_GUIDE.md) | Fast triage spine |\n| [AI_AGENT_INTEGRATION.md](AI_AGENT_INTEGRATION.md) | Workflow guidance for AI agents |\n\n### Developer Guides\n\n| Document | Description |\n|:---------|:------------|\n| [ARCHITECTURE.md](ARCHITECTURE.md) | Detailed architecture documentation |\n| [CONTRIBUTING.md](CONTRIBUTING.md) | Contribution guidelines |\n\n---\n\n## Contributing\n\nContributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n\n---\n\n## Licence\n\nGNU General Public Licence v3.0 — See [LICENSE](LICENSE) for details.\n\n---\n\n## Cyber Range\n\nCrowdSentinel ships with a companion **[crowdsentinel-range](https://github.com/thomasxm/crowdsentinel-range)** — a fully ephemeral AWS lab that generates ground-truth-labeled attack telemetry so you can investigate real detections rather than synthetic data.\n\n### Current range (v1) — 2-host AWS lab\n\n```\n┌──────────────────────────── AWS VPC (private subnet) ────────────────────────────────┐\n│                                                                                        │\n│  ┌──────────────────────────────┐      ┌──────────────────────────────────────────┐  │\n│  │   Linux SIEM (Ubuntu 22.04)  │      │   Windows Range Box (Server 2022)        │  │\n│  │   Elasticsearch 8.x (TLS)   │◄─────│   Sysmon (olafhartong) + Winlogbeat 9    │  │\n│  │   Auditbeat + Filebeat       │      │   Atomic Red Team + PurpleSharp          │  │\n│  └──────────────────────────────┘      └──────────────────────────────────────────┘  │\n│                         SSM VPC endpoints — no inbound ports open                     │\n└───────────────────────────────────────┬────────────────────────────────────────────--┘\n                                         │ SSM port-forward :19200\n                              ┌──────────▼──────────────────────┐\n                              │  ./range-mcp.sh → CrowdSentinel │\n                              │  MCP Server → Claude Code       │\n                              └─────────────────────────────────┘\n```\n\n**3 commands to go from zero to hunting:**\n```bash\ncd crowdsentinel-range && make up && make attack-windows\ncd .. && ./range-mcp.sh\n# then in Claude Code: /mcp → \"investigate potential intrusions on the windows machine\"\n```\n\n> Demo: [73,328 events triaged in ~40 seconds](https://github.com/thomasxm/crowdsentinel-range/blob/main/docs/demo-windows-triage-investigation.md) — 2 shellcode payloads decoded, C2 IP extracted from raw bytes, full IR report with 19 MITRE techniques.\n\n---\n\n### Future range (v2) — enterprise AD environment\n\nThe next range generation targets **full Active Directory kill-chain coverage**: initial access → lateral movement → domain compromise, with a network security monitor layer, runtime security instrumentation, centralised vulnerability management, and deception infrastructure. Inspired by [DetectionLab](https://github.com/clong/DetectionLab), [Ludus](https://docs.ludus.cloud/), and [SimuLand](https://github.com/Azure/SimuLand).\n\n```\n┌───────────────────────────────────── AWS / Multi-VPC ───────────────────────────────────────────────────┐\n│                                                                                                           │\n│  ┌──── Attacker Subnet ────┐   ┌───────────── Corporate Subnet ──────────────────────────────────────┐  │\n│  │                          │   │                                                                       │  │\n│  │  Kali Linux (C2 server)  │   │  ┌─────────────────┐  ┌──────────────────┐  ┌──────────────────┐  │  │\n│  │  Caldera + Metasploit    │   │  │  Domain          │  │  Windows 11      │  │  Windows 11      │  │  │\n│  │  Covenant C2             │──▶│  │  Controller      │  │  Workstation 1   │  │  Workstation 2   │  │  │\n│  │                          │   │  │  (AD DS, DNS)    │  │  (domain-joined) │  │  (domain-joined) │  │  │\n│  └──────────────────────────┘   │  └─────────────────┘  └──────────────────┘  └──────────────────┘  │  │\n│                                  │  ┌──────────────────────────────────────────────────────────────┐  │  │\n│  ┌──── DMZ Subnet ─────────┐    │  │   Linux File/Web Server (IIS / Apache, SMB share)            │  │  │\n│  │                          │    │  └──────────────────────────────────────────────────────────────┘  │  │\n│  │  Nginx reverse proxy     │    └───────────────────────────────────────────────────────────────────┘  │\n│  │  Simulated internet SVC  │                                                                            │\n│  └──────────────────────────┘   ┌───────────── Management / SIEM Subnet ──────────────────────────┐    │\n│                                  │                                                                   │    │\n│                                  │  ┌─────────────────────┐   ┌───────────────────────────────┐   │    │\n│                                  │  │  Elasticsearch +     │   │  Zeek / Suricata              │   │    │\n│                                  │  │  Kibana (SIEM)       │   │  (NSM — full packet + alerts) │   │    │\n│                                  │  └─────────────────────┘   └───────────────────────────────┘   │    │\n│                                  │  ┌─────────────────────┐   ┌───────────────────────────────┐   │    │\n│                                  │  │  Velociraptor        │   │  Falco (runtime security)     │   │    │\n│                                  │  │  (live EDR/forensics)│   │  eBPF probe (CO-RE, default)  │   │    │\n│                                  │  └─────────────────────┘   │  + kernel module fallback     │   │    │\n│                                  │  ┌─────────────────────┐   └───────────────────────────────┘   │    │\n│                                  │  │  DefectDojo          │   ┌───────────────────────────────┐   │    │\n│                                  │  │  (vuln management)   │   │  Honeypots + Canary tokens     │   │    │\n│                                  │  │  Falco·Zeek·scanners │   │  (deception layer)             │   │    │\n│                                  │  └─────────────────────┘   └───────────────────────────────┘   │    │\n│                                  │  ┌─────────────────────────────────────────────────────────┐   │    │\n│                                  │  │  Grafana — centralized dashboards + alerting             │   │    │\n│                                  │  │  ES · Falco · Suricata · Velociraptor · DefectDojo       │   │    │\n│                                  │  └─────────────────────────────────────────────────────────┘   │    │\n│                                  └───────────────────────────────────────────────────────────────┘    │\n│                                         VyOS router — inter-subnet ACLs + egress filter               │\n└───────────────────────────────────────────────────────────────────────────────────────────────────────┘\n                                                  │ SSM / VPN\n                                    CrowdSentinel MCP + Claude Code\n```\n\n**v2 target capabilities:**\n\n| Capability | v1 (current) | v2 (planned) |\n|------------|-------------|--------------|\n| Hosts | 2 (Linux SIEM + Windows) | 7–10 (DC + 2× WS + Linux + NSM + attacker + SIEM) |\n| Network segments | 1 (private subnet) | 4 (attacker / corp / DMZ / management) |\n| Active Directory | — | Full AD DS domain + GPO + Kerberos |\n| Lateral movement coverage | Limited | Pass-the-Hash, Kerberoasting, DCSync |\n| Network detection | — | Zeek (NSM) + Suricata IDS |\n| Runtime security | — | Falco; modern eBPF probe (CO-RE, bundled, default) + kernel module fallback; syscall-level host + container detection |\n| Endpoint forensics | — | Velociraptor live triage |\n| Vuln management | — | DefectDojo; aggregates Falco alerts, Zeek/Suricata findings, scanner results (OpenVAS, Trivy) into unified tracking + remediation workflow |\n| Dashboards | — | Grafana; centralised metrics + alert visualisation across ES, Falco, Suricata, Velociraptor, and DefectDojo |\n| Deception layer | — | Honeypot + canary tokens |\n| C2 infrastructure | ART atomics only | Caldera + Metasploit + Covenant |\n| Attack scenarios | Windows atomics + EVTX replay | Full kill-chain: initial access → domain compromise |\n| CrowdSentinel coverage | ES hunt + IR | ES + Zeek/Suricata + Falco + Velociraptor cross-source correlation; DefectDojo/Grafana as upstream context |\n\n**Design references:** [DetectionLab](https://github.com/clong/DetectionLab) (Sysmon+WEF+Splunk convergence), [Ludus](https://docs.ludus.cloud/) (modular Proxmox/Ansible, VLAN segmentation), [SimuLand](https://github.com/Azure/SimuLand) (multi-domain AD forest, cloud-native), [SANS FOR508](https://www.sans.org/cyber-security-courses/advanced-incident-response-threat-hunting-training/) (artifact-first DFIR workflow), [Falco](https://falco.org/docs/concepts/falco-kernel-testing/) (CO-RE eBPF probe architecture, kernel module fallback).\n\n## Acknowledgements\n\nCrowdSentinel stands on the shoulders of outstanding open-source projects. Thank you to all of their maintainers and contributors.\n\n**Protocol & server framework**\n\n- [Model Context Protocol](https://modelcontextprotocol.io/) — the protocol connecting LLMs to tools, by Anthropic\n- [FastMCP](https://github.com/jlowin/fastmcp) — the Python MCP server framework this project is built on\n\n**Analysis engines & forensic tools**\n\n- [Chainsaw](https://github.com/WithSecureLabs/chainsaw) — EVTX log analyser by WithSecure Labs\n- [Velociraptor](https://docs.velociraptor.app/) — live endpoint forensics and DFIR platform by Velocidex/Rapid7\n- [osquery](https://osquery.io/) — SQL-powered host instrumentation by the osquery Foundation\n- [Wireshark / TShark](https://www.wireshark.org/) — network protocol analysis by the Wireshark Foundation\n- [Chepy](https://github.com/securisec/chepy) — CyberChef-like decoding library powering optional long-tail deobfuscation\n- [sqlglot](https://github.com/tobymao/sqlglot) — SQL parser used to safely validate osquery queries\n\n**Detection content**\n\n- [Sigma / SigmaHQ](https://github.com/SigmaHQ/sigma) — the community detection rule standard and rule corpus\n- [Elastic detection-rules](https://github.com/elastic/detection-rules) — Elastic's prebuilt EQL/ES|QL detection rules\n- Community-contributed Sigma and custom rules\n\n**Data platforms & threat intelligence**\n\n- [Elasticsearch](https://github.com/elastic/elasticsearch-py) and [OpenSearch](https://github.com/opensearch-project/opensearch-py) Python clients\n- [MISP / PyMISP](https://github.com/MISP/PyMISP) — open-source threat intelligence platform and client\n- [stix2](https://github.com/oasis-open/cti-python-stix2) — OASIS STIX 2.1 library for structured IoC export\n- [Shodan InternetDB](https://internetdb.shodan.io/), [VirusTotal](https://www.virustotal.com/), [AbuseIPDB](https://www.abuseipdb.com/), and [abuse.ch ThreatFox](https://threatfox.abuse.ch/) — IoC enrichment services with free tiers\n\n**Development & testing**\n\n- [testcontainers-python](https://github.com/testcontainers/testcontainers-python) — real Elasticsearch containers for the live test tiers\n- [pytest](https://pytest.org/), [Ruff](https://github.com/astral-sh/ruff), [Bandit](https://github.com/PyCQA/bandit) — test runner, linting, and SAST\n\n**Methodology frameworks**\n\n- Cyber Kill Chain (Lockheed Martin), Pyramid of Pain (David J. Bianco), Diamond Model (Caltagirone, Pendergast & Betz), [MITRE ATT&CK](https://attack.mitre.org/)\n\n---\n\n<div align=\"center\">\n\n**Made for the security community by [medjedtxm](https://github.com/thomasxm)**\n\n[![GitHub](https://img.shields.io/badge/GitHub-CrowdSentinel-181717?style=for-the-badge&logo=github)](https://github.com/thomasxm/CrowdSentinels-AI-MCP)\n[![PyPI](https://img.shields.io/badge/PyPI-crowdsentinel--mcp--server-3775A9?style=for-the-badge&logo=pypi&logoColor=white)](https://pypi.org/project/crowdsentinel-mcp-server/)\n\n</div>\n",
  "bytes": 59068,
  "sha": "fabaecc301a4f5f567263480e7ab470a7b2bdf7ef54b041e5e9eea3603eb6d28",
  "repo_slug": "thomasxm/crowdsentinels-ai-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_thomasxm_crowdsentinel_mcp_ser_7143c1ae/readme"
}