{
  "markdown": "# Rocket.Chat Code Analyzer\n\nThis project is a prototype for reducing LLM context cost when analyzing large TypeScript repositories.\n\nInstead of loading full source files up front, it builds a compact structural index of exports and reads implementation details only when needed.\n\n## Why This Exists\n\nLarge monorepos can consume a massive number of tokens before an assistant answers a single question. This project demonstrates a practical workflow to keep that cost predictable:\n\n1. Build a typed repository skeleton from exported symbols.\n2. Let the agent reason over the skeleton first.\n3. Read only the files and line ranges needed for deeper answers.\n\n## Current Architecture\n\nThe codebase currently has three primary pieces:\n\n- `src/repoIndex.ts`\n     Walks a target directory, parses TypeScript with `ts-morph`, and extracts exported signatures for functions, classes, interfaces, type aliases, and enums.\n\n- `src/LazyFileReader.ts`\n     Reads file content on demand with controls for maximum lines, optional line ranges, and symbols-only mode. It also enforces a base directory boundary to prevent path traversal.\n\n- `src/demo.ts`\n     End-to-end demonstration script. It builds the skeleton, simulates selective file reads, and logs benchmark output to `benchmark-results.json`.\n\nThe project now also includes a standard Gemini CLI extension manifest at the repository root, so the repo can be linked directly as an extension during development.\n\nIt now includes layered caching for both repository indexing and on-demand file reads:\n\n- In-memory + disk cache for `repo_index`\n- In-memory cache for `read_file` raw snapshots and symbols-only views\n- MCP tools for cache stats and explicit cache invalidation\n## Project Layout\n\n```text\nsrc/\n     repoIndex.ts\n     LazyFileReader.ts\n     demo.ts\ngemini-extension.json\nGEMINI.md\ngemini-extension/\n     mcp/mcp-server.example.json\n     tools/index.ts\ntests/\n     repoIndex.test.ts\n     mcpLazyServer.test.ts\nbenchmark-results.json\nbenchmark-results-mcp.json\n```\n\n## Setup\n\nRequirements:\n\n- Node.js 18+\n- npm\n\nInstall dependencies:\n\n```bash\nnpm install\n```\n\nBuild the project (required for extension runtime):\n\n```bash\nnpm run build\n```\n\nCreate local environment file:\n\n```bash\ncopy .env.example .env\n```\n\nThen set your key in `.env`:\n\n```env\nGEMINI_API_KEY=your-key-here\n```\n\nPowerShell alternative (session-only):\n\n```powershell\n$env:GEMINI_API_KEY = \"your-key-here\"\n```\n\n## Method 1: Local Sparse Index + Lazy Reader\n\nThis method runs everything locally from this repository and is the baseline implementation.\n\n## Usage\n\nRun the demo against a target directory:\n\n```bash\nnpx tsx src/demo.ts ./src \"What are the main exports in this codebase?\"\n```\n\nArguments:\n\n- Arg 1: target directory (default: `.`)\n- Arg 2: question string (default: a generic exports question)\n\nWhat the demo does:\n\n1. Builds an index of exported symbols.\n2. Estimates skeleton token cost vs naive full-read cost.\n3. Simulates reading only selected files.\n4. Appends a run record to `benchmark-results.json`.\n\n## Method 2: MCP + Gemini CLI Lazy Loading\n\nOption 2 moves repository reads to an MCP server so gemini-cli can call tools instead of loading large file sets directly into prompt context.\n\nThis is useful for questions like:\n\n- How are messages sent in Rocket.Chat?\n- How does user authentication work?\n- How are permissions checked?\n- What is the E2E encryption flow?\n\n### What was added\n\n- `src/mcpLazyServer.ts`\n     MCP stdio server exposing two tools:\n     - `repo_index` to return a typed skeleton for a target directory\n     - `read_file` to lazily fetch only needed file content\n     - `index_cache_stats` to inspect index/read cache status\n     - `index_cache_invalidate` to clear stale cache state\n\n- `gemini-extension/mcp/mcp-server.example.json`\n     Example MCP server registration file for gemini-cli style configurations.\n\n### Run the MCP server\n\n```bash\nnpm run mcp:server\n```\n\nIf you typed `npm runmcp:server`, that command will fail. Use `npm run mcp:server` with a space after `run`.\n\n### Integrate with gemini-cli (standard extension flow)\n\n1. Build the extension once:\n\n```bash\nnpm run build\n```\n\n2. Link this repository as a Gemini extension:\n\n```bash\ngemini extensions link .\n```\n\n3. Restart gemini-cli.\n4. Verify the extension is active:\n\n```bash\ngemini extensions list\n```\n\n5. Ask gemini-cli to use MCP tools with an instruction like:\n\n```text\nUse MCP tools for code analysis.\nCall repo_index first for targetDir=\"<ABSOLUTE_PATH_TO_TARGET_REPO_SUBDIR>\".\nThen call read_file only when implementation details are needed.\n```\n\n### How to call Gemini from terminal\n\n1. Start Gemini CLI:\n\n```bash\ngemini\n```\n\n2. In the interactive prompt, ask a scoped question and force tool usage:\n\n```text\nHow does message sending work in Rocket.Chat?\nUse MCP tools.\nCall repo_index first with targetDir=\"<ABSOLUTE_PATH_TO_TARGET_REPO_SUBDIR>\".\nThen call read_file only for relevant files.\n```\n\n3. Confirm the tool calls appear in output (`repo_index`, then `read_file`).\n\n6. For deep architecture questions such as message flow, auth flow, permissions, and E2E encryption, keep the same pattern:\n      - `repo_index` once at the beginning\n      - `read_file` only for specific files and sections\n\nThe `gemini-extension.json` manifest uses `${extensionPath}` so it runs cross-platform without hardcoded absolute paths.\n\n### Full walkthrough on Windows\n\n1. Open PowerShell in this repo:\n\n```powershell\ncd \"<ABSOLUTE_PATH_TO_CODE_ANALYZER>\"\n```\n\n2. Install dependencies once:\n\n```powershell\nnpm install\n```\n\n3. Build before starting MCP server:\n\n```powershell\nnpm run build\n```\n\n4. Start MCP server (correct command):\n\n```powershell\nnpm run mcp:server\n```\n\n5. If you typed `npm runmcp:server`, it fails because `run` and script name must be separate.\n6. For extension-based integration, run `gemini extensions link .` once and restart gemini-cli.\n7. Ask one of your target questions and explicitly request MCP tool usage:\n\n```text\nHow are messages sent in Rocket.Chat?\nUse MCP tools.\nCall repo_index first for targetDir=\"<ABSOLUTE_PATH_TO_TARGET_REPO_SUBDIR>\".\nThen call read_file only for required files.\n```\n\n8. Verify in gemini-cli output that tool calls appear for `repo_index` and `read_file`.\n9. Record MCP benchmark run separately:\n\n```powershell\nnpx tsx src/demo.ts --mode mcp \"<ABSOLUTE_PATH_TO_TARGET_REPO_SUBDIR>\" \"How are messages sent in Rocket.Chat?\"\n```\n\n10. MCP mode appends results to `benchmark-results-mcp.json` and keeps `benchmark-results.json` unchanged.\n\n### Expected index cache behavior\n\n- First `repo_index` call on a target directory: cache miss (index build).\n- Repeated `repo_index` call in the same process: memory cache hit.\n- Repeated `repo_index` call after restart with no relevant changes: disk cache hit.\n- Any indexable file change: cache invalidates and rebuilds automatically.\n- Use `forceRefresh=true` in `repo_index` to bypass cache manually.\n- Use `index_cache_invalidate` to clear index cache and optionally clear `read_file` cache.\n\nCache metadata is returned in the `repo_index` response as:\n\n- `cache.enabled`\n- `cache.hit`\n- `cache.layer`\n- `cache.cacheFile`\n- `cache.fingerprint`\n\n### Verify MCP server is reachable\n\n1. Start server in one terminal:\n\n```bash\nnpm run mcp:server\n```\n\n2. In gemini-cli, run a prompt that explicitly requests tool usage.\n3. You should see tool calls to `repo_index` and `read_file` instead of broad source dumps.\n\n### Capture final MCP benchmark results\n\n1. Run analysis with MCP enabled for your target question.\n2. Use `npx tsx src/demo.ts --mode mcp <targetDir> \"<question>\"` to append a run to `benchmark-results-mcp.json`.\n3. Keep `benchmark-results.json` as your local baseline and mock comparison.\n\nCurrent MCP benchmark snapshot is included in `benchmark-results-mcp.json`.\n\n## Current Results Summary\n\nMethod 1 (Local sparse index + lazy reader):\n\n- `benchmark-results.json` contains local baseline runs.\n- Example measured run: 309,357 naive tokens reduced to 14,252 total session tokens.\n\nMethod 2 (MCP + Gemini CLI lazy loading):\n\n- `benchmark-results-mcp.json` contains MCP-specific runs.\n- Current snapshot preserves the same measured token profile while moving retrieval to MCP tool calls.\n\n### Why this reduces token cost\n\n1. Skeleton first: the model gets compact exported signatures instead of full source files.\n2. Lazy fetches: implementation is retrieved only when necessary.\n3. Scoped reads: `symbolsOnly`, `lineRange`, and `maxLines` keep payloads bounded.\n\n## Working Example: Message Sending Analysis in Rocket.Chat\n\nThis project was validated against the Rocket.Chat codebase to trace how messages are sent through the system. The analysis demonstrates both the skeletal index approach and live MCP tool-calling.\n\n### Message Sending Flow (Traced via repo_index + read_file)\n\nThe MCP server successfully extracted and analyzed the complete message pipeline:\n\n1. **Entry Point (Meteor Method)**: The client calls the `sendMessage` Meteor method, which performs initial checks, enforces rate limits, and triggers `executeSendMessage`.\n\n2. **Validation & Preparation (executeSendMessage)**: This step validates the message size, ensures the room exists, checks timestamps, and confirms the sender's identity. It also verifies if the user has permission to send messages in the specific room.\n\n3. **Core Logic (sendMessage Function)**:\n   - **Apps-Engine Hooks**: Triggers `IPreMessageSentPrevent`, `IPreMessageSentExtend`, and `IPreMessageSentModify` events.\n   - **beforeSave Hooks**: Executes various filters (bad words, markdown, mentions, etc.) through the `Message.beforeSave` service call.\n   - **Persistence**: The message is inserted into the Messages collection.\n   - **Post-Persistence Apps-Engine**: Triggers `IPostMessageSent` or `IPostSystemMessageSent`.\n\n4. **Post-Save Actions (afterSaveMessage)**:\n   - **Callbacks**: Runs the `afterSaveMessage` callback, which includes `notifyUsersOnMessage`.\n   - **Notifications & Updates**: Updates room activity trackers, adjusts user subscription unread counts/alerts, and broadcasts changes to clients via DDP (e.g., `notifyOnRoomChangedById`).\n   - **Service-Level Post-Save**: `Message.afterSave` handles additional asynchronous tasks like OEmbed link parsing.\n\n### MCP Server Status\n\nThe MCP server is **running and successfully integrated with gemini-cli**:\n\n```\nConfigured MCP servers:\n- rocketChatLazyIndex - Ready (4 tools)\n  Tools:\n    - mcp_rocketChatLazyIndex_read_file\n    - mcp_rocketChatLazyIndex_repo_index\n          - mcp_rocketChatLazyIndex_index_cache_stats\n          - mcp_rocketChatLazyIndex_index_cache_invalidate\n```\n\n### Live Performance Metrics\n\n- Latest measured run (`Rocket.Chat/apps/meteor/server`): 307,582 naive tokens reduced to 12,002 total session tokens.\n- Files indexed: 148\n- Index cache: enabled (`indexCacheHit: false` on rebuild run)\n\n- **Session ID**: f1718aad-c001-4b0f-9bbd-27b662c82aa0\n- **Tool Calls**: 10 (9 successful, 1 duplicate)\n- **Success Rate**: 90.0%\n- Latest measured run (`Rocket.Chat/apps/meteor/server`): 307,582 naive tokens reduced to 11,595 total session tokens.\n- Files indexed: 148\n- Index cache: enabled (`indexCacheHit: true`)\n\n**Wall Time**: 2m 42s  \n**Agent Active**: 47.7s\n\n- **API Time**: 24.0s (50.2%)\n- **Tool Time**: 23.7s (49.8%)\n\n**Token Efficiency**:\n- **gemini-2.5-flash-lite**: 1 request → 1,087 input tokens + 86 output tokens\n- **gemini-3-flash-preview**: 11 requests → 81,037 input tokens (207,415 from cache) + 1,412 output tokens\n\n**Savings Highlight**: 207,415 (71.6%) of input tokens were served from cache, directly demonstrating the lazy-loading efficiency of the MCP approach.\n\n## Development Commands\n\n```bash\nnpm run demo\nnpm run mcp:server\nnpm run mcp:server:dev\nnpm test\nnpm run build\n```\n\n## Priorities and Next Steps\n\n1. Replace the mock loop in `src/demo.ts` with a live tool-calling flow so the model can decide when to call `read_file`.\n2. Add query intent routing (planned classifier layer) to scope indexing by domain before parsing, reducing initial index size.\n3. Improve index fidelity with richer class details (constructors, overloads, visibility filters) while preserving compact output.\n4. Expand tests for `src/LazyFileReader.ts`, especially path boundary checks, symbols-only output, and line-range edge cases.\n5. Add optional TTL/size limits and cleanup for `.cache/repo-index` in long-running environments.\n6. Document a release checklist for publishing this extension with versioned GitHub releases.",
  "bytes": 12506,
  "sha": "e677a0b82fd7b91f145712d5a6087c9bf7fef11f4a5b64057677a1c470290c90",
  "repo_slug": "prajanmanojkumarrekha/code-analyzer",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_prajanmanojkumarrekha_code_analyzer_164e32b7/readme"
}