{
  "markdown": "# sensegrep\n\n**semantic grep for AI coding agents**\n\n[![npm version](https://img.shields.io/npm/v/@sensegrep/core)](https://www.npmjs.com/package/@sensegrep/core)\n[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)\n[![CI](https://github.com/Stahldavid/sensegrep/actions/workflows/ci.yml/badge.svg)](https://github.com/Stahldavid/sensegrep/actions/workflows/ci.yml)\n\nsensegrep understands your code semantically. Instead of matching text patterns, it uses AI embeddings and tree-sitter AST parsing to find code by *meaning* - so you can search for \"authentication logic\" and actually find your auth functions, even if they never contain the word \"authentication\".\n\nAI agents should not read more code, they should read the right code. sensegrep combines semantic search, exact matching, and AST-aware structural retrieval to deliver smaller, more relevant context.\n\nFor agent routing, `search` is the default for discovery: questions about behavior, concepts, structure, or where/how something is implemented. Use `literal` only for a known exact string or regex, an exhaustive occurrence audit, or verification/refinement after semantic discovery. If the intent is ambiguous, start with `search`.\n\n![Sensegrep time-to-value demo](assets/time-to-value.gif)\n\nMP4 fallback: [assets/time-to-value.mp4](assets/time-to-value.mp4)\n\nWatch full product demo (25s): [assets/time-to-value-full.mp4](assets/time-to-value-full.mp4)\n\n## Why sensegrep?\n\nTraditional search tools (grep, ripgrep, ast-grep) match **text patterns**. sensegrep matches **concepts**:\n\n| Feature | grep/ripgrep | ast-grep | sensegrep |\n|---------|-------------|----------|-----------|\n| Exact text match | Yes | Yes | Yes (via `--pattern`) |\n| AST-aware | No | Yes | Yes (tree-sitter) |\n| Semantic search | No | No | **Yes (AI embeddings)** |\n| Symbol metadata filters | No | Partial | **Yes (30+ filters)** |\n| Duplicate detection | No | No | **Yes (logical duplicates)** |\n| Tree-shaking output | No | No | **Yes (collapse irrelevant code)** |\n| MCP server for AI agents | No | No | **Yes** |\n\n## Quickstart\n\n### Claude Code Plugin (recommended)\n\nThe fastest way to get sensegrep into Claude Code — zero configuration:\n\n```bash\nclaude plugin marketplace add Stahldavid/sensegrep\nclaude plugin install sensegrep\n```\n\nThis automatically sets up the MCP server and teaches Claude when and how to use sensegrep instead of grep. No manual JSON editing required.\n\n> **Marketplace setup** (required on first install):\n> ```bash\n> claude plugin marketplace add Stahldavid/sensegrep\n> claude plugin install sensegrep\n> ```\n>\n> After the marketplace has been added once, the explicit marketplace form also works:\n> ```bash\n> claude plugin install sensegrep@sensegrep\n> ```\n>\n> Running `claude plugin install sensegrep@sensegrep` on a fresh machine before `claude plugin marketplace add Stahldavid/sensegrep` will fail because Claude Code does not know the `sensegrep` marketplace yet.\n\n### CLI\n\n```bash\nnpm i -g @sensegrep/cli\n\n# Smoke-test local CLI/core/config without calling embeddings\nsensegrep selftest --root .\n\n# Default local embeddings use Ollama when no API key/provider is configured\nollama pull qwen3-embedding:0.6b\nsensegrep index --root .\n\n# Search by meaning\nsensegrep search \"error handling and retry logic\" --type function --exported --exclude \"*.md\"\n\n# Known exact text, deterministic and exhaustive: no embedding call\nsensegrep literal \"X-Goog-Message-Number\" --include \"src/**\"\n\n# Compact evidence cards, followed by deterministic expansion\nsensegrep search \"authentication flow\" --purpose understand --json\nsensegrep show <result-id> --before 10 --after 20\n\n# Exhaustive within the ripgrep-visible filesystem, independent of the index\nsensegrep literal \"TODO:\" --filesystem --max-output-bytes 50000 --json\n\n# Complete changed-file review coverage in bounded batches\nsensegrep audit \"security regressions\" --base origin/main --require-coverage --continue-uncovered --batch-tokens 4000 --max-total-tokens 8000 --max-output-bytes 32000 --max-batches 8\n\n# Build a reading map for a broad theme\nsensegrep survey \"authentication login token\" --language typescript --limit 4\n\n# Break a broad topic into coherent subthemes\nsensegrep cluster \"checkout payment order cart\" --limit 4\n\n# Find duplicates\nsensegrep detect-duplicates --threshold 0.85\nsensegrep detect-duplicates --threshold 0.85 --timeout 30s --resume-cursor 0 --json\n```\n\nAgent-facing JSON is minified and `minimal` by default. Opt into `--json-detail content`,\n`--diagnostic`, `--json-detail full`, or `--pretty` only when that additional payload is needed.\nSchema v2 uses one card vocabulary across transports: `id`, `file`, `lines`, `symbol`,\n`kind`, `rank`, and `relevance`. Minimal output keeps retrieval sufficiency, compact index\nstate, and structured warnings; budgets appear only when constrained. Physical output\nbudgets apply to the final serialized JSON and retain a partial evidence card when possible.\nDuplicate JSON excludes source code unless `--show-code` is supplied. Survey and cluster\nJSON default to actionable summary mode with representative IDs.\n\n### Cursor Plugin\n\nInstall from the Cursor marketplace or via CLI:\n\n```bash\ncursor plugin install sensegrep\n```\n\nIncludes the MCP server, an always-on rule to prefer sensegrep over grep, and a skill with full filter reference.\nCursor plugin status: pending marketplace approval.\n\nOne-click MCP install link for Cursor:\n\n[![Add to Cursor](https://img.shields.io/badge/Add%20to-Cursor-111111?logoColor=white)](https://stahldavid.github.io/sensegrep/cursor-install/)\n\nFallback deeplink (copy/paste if needed):\n\n```text\ncursor://anysphere.cursor-deeplink/mcp/install?name=sensegrep&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBzZW5zZWdyZXAvbWNwQGxhdGVzdCJdfQ%3D%3D\n```\n\n### Codex Plugin\n\nInstall from the public marketplace — no manual config:\n\n```bash\ncodex plugin marketplace add Stahldavid/sensegrep\ncodex plugin install sensegrep\n```\n\nSee the [Codex recipe](docs/recipes/codex.md) for the manual `~/.codex/config.toml` setup.\n\n### MCP Server (for Codex or manual setup)\n\n```bash\nnpx -y @sensegrep/mcp\n```\n\nAdd to your MCP configuration:\n\n```json\n{\n  \"servers\": {\n    \"sensegrep\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@sensegrep/mcp\"]\n    }\n  }\n}\n```\n\nOr with `npm` global install first:\n\n```bash\nnpm install -g @sensegrep/mcp\n```\n\n```json\n{\n  \"servers\": {\n    \"sensegrep\": {\n      \"command\": \"sensegrep-mcp\"\n    }\n  }\n}\n```\n\nThe MCP server provides canonical `sensegrep_search`, `sensegrep_show`, `sensegrep_literal`, `sensegrep_context`, `sensegrep_survey`, `sensegrep_cluster`, `sensegrep_graph`, `sensegrep_index`, and `sensegrep_detect_duplicates` tools. Legacy dotted names such as `sensegrep.search` remain available as compatibility aliases where supported.\n\n### Agent Skill — CLI (no MCP server)\n\nFor terminal-first agents or CI, you don't need an MCP server. Install the CLI and the\n`sensegrep-cli` [Agent Skill](docs/agent-skills.md), which teaches the agent to run\n`sensegrep` commands directly:\n\n```bash\nnpm i -g @sensegrep/cli\nnpx skills add Stahldavid/sensegrep --skill sensegrep-cli -g\n```\n\nSee [docs/agent-skills.md](docs/agent-skills.md) for when to use the MCP tools vs the CLI skill.\n\n### VS Code Extension\n\nSearch for **\"Sensegrep\"** in the VS Code marketplace, or install from [the extension page](https://marketplace.visualstudio.com/items?itemName=sensegrep.sensegrep).\n\nFeatures: semantic search sidebar, duplicate detection, code lens, semantic folding, and multi-root auto-indexing/watch mode.\n\n## Recipes\n\nCopy-paste setup and practical workflows:\n\n- [Claude Code recipe](docs/recipes/claude-code.md)\n- [Cursor recipe](docs/recipes/cursor.md)\n- [Codex recipe](docs/recipes/codex.md)\n- [CI with GitHub Actions](docs/recipes/ci-github-actions.md)\n- [Generic CI recipe](docs/recipes/ci-generic.md)\n\nFull index: [docs/recipes/README.md](docs/recipes/README.md)\n\n## How It Works\n\n```\nSource Code\n    │\n    ▼\n┌─────────────┐    ┌──────────────┐    ┌──────────────┐\n│  Tree-Sitter │───▶│   Chunker    │───▶│  Embeddings  │\n│  AST Parser  │    │  (symbols +  │    │ (Ollama,     │\n│              │    │   metadata)  │    │ Gemini, etc.)│\n└─────────────┘    └──────────────┘    └──────────────┘\n                                              │\n                                              ▼\n                                       ┌──────────────┐\n                          Query ──────▶│   LanceDB    │\n                                       │ Vector Search│\n                                       └──────┬───────┘\n                                              │\n                                              ▼\n                                       ┌──────────────┐\n                                       │ Tree-Shaker  │──▶ Results\n                                       │ (collapse    │\n                                       │  irrelevant) │\n                                       └──────────────┘\n```\n\n1. **Parse**: Tree-sitter extracts AST nodes with full metadata (symbol type, exports, complexity, docs, decorators)\n2. **Chunk**: Code is split into semantic chunks aligned to symbol boundaries\n3. **Embed**: Each chunk is embedded using Ollama, Gemini, an OpenAI-compatible embeddings API, or Amazon Bedrock.\n4. **Store**: Embeddings + metadata are stored in LanceDB for fast vector search\n5. **Search**: Vector and lexical candidates are fused, structurally filtered, and optionally reranked\n6. **Tree-shake**: Results are collapsed to show only relevant code, hiding unrelated symbols\n\n## Supported Languages\n\n- **TypeScript** / **JavaScript** (TSX/JSX included)\n- **Python** (dataclasses, protocols, decorators, async generators, TypedDict, and more)\n- **Java** (classes, interfaces, records, annotations, methods, and tree-shaken results)\n- **Vue** (single-file components with `<script>` / `<script setup>` semantic support)\n- More coming: C#, HTML (see [feature branches](https://github.com/Stahldavid/sensegrep/branches))\n\n## Search Filters\n\nsensegrep supports 30+ structural filters that can be combined with semantic search:\n\n```bash\n# Find exported async functions with high complexity\nsensegrep search \"data processing\" --type function --exported --async --min-complexity 5\n\n# Find Python dataclasses\nsensegrep search \"user model\" --type class --variant dataclass --language python\n\n# Find undocumented complex code (refactoring candidates)\nsensegrep search \"business logic\" --min-complexity 10 --has-docs false\n\n# Filter by decorator\nsensegrep search \"route handler\" --type function --decorator route\n\n# Keep docs and markdown out of results\nsensegrep search \"authentication flow\" --include \"src/**/*.ts\" --exclude \"*.md\"\n\n# Build a reading map for onboarding a domain\nsensegrep survey \"authentication login token\" --language typescript --limit 4\n\n# Split a broad backend topic into subthemes\nsensegrep cluster \"price list commission ncm uf packaging\" --language java --include \"backend-api/**/*.java\"\n\n# Build an agent context pack with a hard estimated-token budget\nsensegrep context \"authentication request flow\" --max-tokens 8000\n\n# Review only code changed against a Git base\nsensegrep audit \"security and regression risks\" --base origin/main\n\n# Navigate the local symbol graph\nsensegrep references loadUser\nsensegrep impact loadUser --depth 3\nsensegrep trace handleRequest loadUser\n```\n\nGraph nodes use canonical file/range identities. Ambiguous same-name call targets are omitted instead of expanded into speculative impact edges.\n\n## Index Operations\n\n```bash\n# Inspect local work before any embedding request\nsensegrep index --dry-run --no-watch\n\n# Interrupted full builds resume their fingerprint-matched staging table\nsensegrep index --full --no-watch\n\n# Compare safe concurrency candidates without changing saved config\nsensegrep benchmark --concurrency 1,2,4 --samples 16\n\n# Keep independent indexes for different models/settings\nsensegrep index --profile fast --no-watch\nsensegrep profiles\n\n# Keep LanceDB, graph, tools, and watcher warm behind a local HTTP endpoint\nsensegrep daemon start\nsensegrep daemon endpoint\nsensegrep daemon call --tool search --arguments '{\"query\":\"request routing\",\"limit\":5}'\n\n# Agent-native planning and task-level quality evaluation\nsensegrep investigate \"where does payment gate room access?\" --dry-run\nsensegrep eval sensegrep-eval.yaml\n```\n\nDiagnostic commands are strictly read-only. An incompatible index reports `migrationRequired: true`; rebuild it with `sensegrep index migrate --no-watch`. Full builds stage and validate a new Lance table before switching metadata, while prior generations are retained for concurrent readers and rollback diagnostics.\n\nChanged files reuse vectors for content-identical chunks, even when neighboring chunks or metadata changed. Full indexes checkpoint staging tables and skip IDs already persisted after a restart. For large indexes, LanceDB ANN and scalar indexes are created automatically at 10,000 chunks; set `SENSEGREP_ANN_MIN_CHUNKS=0` to disable or choose another threshold.\n\n### Language plugins\n\nLanguage support can be extended without modifying core. A plugin exports a `LanguageSupport` object (default, `language`, or `languages[]`) with its extensions and optional `chunk(content, filePath)` implementation. Add project-relative ESM modules or package names to `languagePlugins` in `sensegrep.config.json`, or set `SENSEGREP_LANGUAGE_PLUGINS` to a comma-separated list.\n\n```json\n{\n  \"languagePlugins\": [\"./tools/sensegrep-ruby.mjs\"]\n}\n```\n\n## Embeddings Configuration\n\nsensegrep supports local Ollama by default plus Gemini, OpenAI-compatible APIs, and Amazon Bedrock. If no API key or provider is configured, it defaults to Ollama at `http://127.0.0.1:11434` with `qwen3-embedding:0.6b` (1024 dimensions, 32K context). Run `sensegrep selftest --root .` before indexing to confirm the selected provider/model/dimension and credential/endpoint guidance without making embedding calls.\n\n```bash\n# Default local Ollama embeddings (no API key)\nollama pull qwen3-embedding:0.6b\nsensegrep search \"auth flow\"\n\n# Recommended: Gemini embeddings (best quality)\nexport SENSEGREP_PROVIDER=gemini\nexport GEMINI_API_KEY=\"your_ai_studio_key\"\nsensegrep search \"auth flow\" --provider gemini --embed-model gemini-embedding-001\n\n# OpenAI-compatible provider\nexport SENSEGREP_PROVIDER=openai\nexport SENSEGREP_OPENAI_API_KEY=\"your_api_key\"\nexport SENSEGREP_OPENAI_BASE_URL=\"https://api.fireworks.ai/inference/v1\"\nsensegrep search \"auth flow\" --provider openai --embed-model fireworks/qwen3-embedding-8b\n\n# OpenRouter + Qwen3 Embedding 8B with smaller Matryoshka vectors\nexport SENSEGREP_PROVIDER=openai\nexport SENSEGREP_OPENAI_API_KEY=\"your_openrouter_key\"\nexport SENSEGREP_OPENAI_BASE_URL=\"https://openrouter.ai/api/v1\"\nexport SENSEGREP_EMBED_MODEL=\"qwen/qwen3-embedding-8b\"\nexport SENSEGREP_EMBED_DIM=1024\nexport SENSEGREP_OPENAI_BATCH_SIZE=96\n\n# Amazon Bedrock + Cohere Embed v4\nexport AWS_REGION=\"us-east-1\"\nexport SENSEGREP_BEDROCK_API_KEY=\"your_bedrock_api_key\"\nsensegrep search \"auth flow\" --provider bedrock --embed-model cohere.embed-v4:0 --embed-dim 1536\n```\n\nLocal OpenAI-compatible embedding servers also work if they implement `/v1/embeddings`; set `SENSEGREP_PROVIDER=openai`, `SENSEGREP_OPENAI_BASE_URL` to the server's `/v1` base URL, and `SENSEGREP_EMBED_DIM` to the exact returned vector dimension. For native Ollama, use `SENSEGREP_PROVIDER=ollama`, `SENSEGREP_OLLAMA_BASE_URL` if not using the default, and the exact `SENSEGREP_EMBED_DIM` for your Ollama model.\n\nGlobal defaults via `~/.config/sensegrep/config.json`:\n\n```json\n{\n  \"provider\": \"ollama\",\n  \"embedModel\": \"qwen3-embedding:0.6b\",\n  \"embedDim\": 1024\n}\n```\n\nCommon environment variables:\n\n- `SENSEGREP_PROVIDER` (`ollama`, `gemini`, `openai`, `bedrock`)\n- `SENSEGREP_EMBED_MODEL`\n- `SENSEGREP_EMBED_DIM`\n- `SENSEGREP_OLLAMA_BASE_URL` (Ollama, default `http://127.0.0.1:11434`)\n- `GEMINI_API_KEY` / `GOOGLE_API_KEY` (Gemini)\n- `SENSEGREP_OPENAI_API_KEY` / `FIREWORKS_API_KEY` / `OPENAI_API_KEY` (OpenAI-compatible)\n- `SENSEGREP_OPENAI_BASE_URL` (OpenAI-compatible, default `https://api.fireworks.ai/inference/v1`)\n- `SENSEGREP_OPENAI_BATCH_SIZE` (OpenAI-compatible request batch size)\n- `SENSEGREP_OPENAI_CONCURRENCY` / `SENSEGREP_EMBED_CONCURRENCY` (provider request concurrency)\n- `SENSEGREP_INDEX_EMBED_CONCURRENCY` (concurrent index embedding batches)\n- `SENSEGREP_QUERY_CACHE` (`true` by default; set `false` for controlled benchmarks or sensitive environments)\n- `SENSEGREP_QUERY_CACHE_TTL_MS` / `SENSEGREP_QUERY_CACHE_MAX_ENTRIES` (persistent query-vector cache bounds)\n- `SENSEGREP_ADAPTIVE_HYBRID_DELAY_MS` / `SENSEGREP_ADAPTIVE_HYBRID_MIN_SCORE` (adaptive hybrid tuning)\n- `SENSEGREP_ANN_MIN_CHUNKS` (automatic ANN threshold; `0` disables)\n- `SENSEGREP_PROFILE` (named side-by-side index profile)\n- `SENSEGREP_LANGUAGE_PLUGINS` (comma-separated ESM language plugins)\n- `SENSEGREP_OPENROUTER_REFERER` / `SENSEGREP_OPENROUTER_TITLE` (optional OpenRouter attribution headers)\n- `SENSEGREP_BEDROCK_API_KEY` (Amazon Bedrock bearer API key; omit when using the AWS SDK credential chain)\n- `SENSEGREP_BEDROCK_REGION` / `AWS_REGION` / `AWS_DEFAULT_REGION` (Amazon Bedrock)\n- `SENSEGREP_ROOT` (MCP root directory)\n- `SENSEGREP_WATCH` (MCP watcher toggle)\n\nFor the complete and official runtime variable list, see `docs/mcp-setup.md`.\n\nOpenRouter Qwen embeddings use one provider request at a time by default. Benchmarking on\nhigh-latency endpoints commonly shows that additional concurrent requests reduce throughput;\nuse `sensegrep benchmark --concurrency 1,2,4 --json` before overriding the default.\n\n### Index compatibility\n\nEach index records the embedding provider, model, dimension, distance metric, and a non-secret endpoint/configuration fingerprint. If you change provider, model, base URL, dimension, local server pooling behavior, or task-prefix strategy, rebuild the index with `sensegrep index --root . --full --no-watch`. Same dimension does **not** make embeddings interchangeable; two 768-dimensional models still produce different vector spaces.\n\nMore embedding providers and API integrations may be added in the future.\n\n## Packages\n\n| Package | Description | npm |\n|---------|-------------|-----|\n| [@sensegrep/core](packages/core) | Search engine library | [![npm](https://img.shields.io/npm/v/@sensegrep/core)](https://www.npmjs.com/package/@sensegrep/core) |\n| [@sensegrep/cli](packages/cli) | Command-line interface | [![npm](https://img.shields.io/npm/v/@sensegrep/cli)](https://www.npmjs.com/package/@sensegrep/cli) |\n| [@sensegrep/mcp](packages/mcp) | MCP server for AI agents | [![npm](https://img.shields.io/npm/v/@sensegrep/mcp)](https://www.npmjs.com/package/@sensegrep/mcp) |\n| [sensegrep](packages/vscode) | VS Code extension | [Marketplace](https://marketplace.visualstudio.com/items?itemName=sensegrep.sensegrep) |\n| [sensegrep-plugin](plugin/sensegrep-plugin) | Claude Code plugin | `claude plugin marketplace add Stahldavid/sensegrep && claude plugin install sensegrep` |\n| [sensegrep-cursor](plugin/sensegrep-cursor) | Cursor plugin | `cursor plugin install sensegrep` |\n| [sensegrep (Codex)](plugins/sensegrep) | Codex plugin | `codex plugin marketplace add Stahldavid/sensegrep && codex plugin install sensegrep` |\n\n## Case Studies\n\nReproducible qualitative examples from public repositories:\n\n- [Case studies](docs/case-studies.md)\n- [Use cases](docs/use-cases.md)\n- [Parallel-agent workflows](docs/parallel-agents.md)\n\n## Roadmap\n\n- [ROADMAP.md](ROADMAP.md)\n- Benchmark methodology vs `ripgrep` / `ast-grep` is scheduled for Month 2.\n\n## Contributing\n\nSee [the architecture guide](docs/architecture.md) for runtime and persistence design, and [CONTRIBUTING.md](CONTRIBUTING.md) for development setup and contribution guidelines. The VS Code extension stores provider API keys in VS Code `SecretStorage`; workspace settings contain only non-secret configuration.\n\n## Community\n\n- [Code of Conduct](CODE_OF_CONDUCT.md)\n- [Security Policy](SECURITY.md)\n- [Support](SUPPORT.md)\n\n## License\n\n[Apache-2.0](LICENSE)\n",
  "bytes": 20090,
  "sha": "19411713c6ba1d5255ec75071f16b241bc4ad5bce5404bef66fa386a3ed0289f",
  "repo_slug": "stahldavid/sensegrep",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_stahldavid_sensegrep_49bf304f/readme"
}