{
  "markdown": "# memex\n\n[![GitHub release](https://img.shields.io/github/v/release/ayushagrawal288/memex)](https://github.com/ayushagrawal288/memex/releases)\n[![Python](https://img.shields.io/badge/python-3.12-blue)](https://www.python.org)\n[![FastAPI](https://img.shields.io/badge/FastAPI-0.115-green)](https://fastapi.tiangolo.com)\n[![MCP](https://img.shields.io/badge/MCP-Streamable%20HTTP-purple)](https://modelcontextprotocol.io)\n[![License: MIT](https://img.shields.io/badge/license-MIT-brightgreen)](LICENSE)\n[![ayushagrawal288/memex MCP server](https://glama.ai/mcp/servers/ayushagrawal288/memex/badges/score.svg)](https://glama.ai/mcp/servers/ayushagrawal288/memex)\n\n[![memex MCP server](https://glama.ai/mcp/servers/ayushagrawal288/memex/badges/card.svg)](https://glama.ai/mcp/servers/ayushagrawal288/memex)\n\nA production-grade persistent memory service for AI agents. Agents forget everything between sessions by default — memex fixes that. It stores, retrieves, and ranks conversation memory using semantic search with recency decay, so agents surface what's relevant *and* recent, not just what's semantically closest.\n\n```\nPOST /v1/memories          → store a memory, embed it, persist to Postgres\nPOST /v1/memories/search   → retrieve top-k memories ranked by similarity + recency\nDELETE /v1/memories/{id}   → forget a specific memory\nGET  /v1/memories/count    → how many memories does this agent/user have\nGET  /health               → liveness + DB connectivity check\nGET  /metrics              → Prometheus metrics\n```\n\n---\n\n## Architecture\n\n```\ncaller (agent / app)\n        │\n        ▼\n  FastAPI (async)\n        │\n   ┌────┴────┐\n   │         │\nembeddings  asyncpg pool (min=5, max=20)\n(fastembed  │\n ONNX,      ▼\n local)  PostgreSQL 16\n           pgvector extension\n           ivfflat index (cosine)\n```\n\n**Write path:** content → fastembed ONNX inference (local, ~12 ms CPU, `BAAI/bge-small-en-v1.5`) → INSERT with 384-dim vector → return memory ID.\n\n**Read path:** query → embed → pgvector cosine search (top_k × 3 candidates) → re-rank with recency decay in Python → return top_k results with scores.\n\n---\n\n## Design decisions\n\n### 1. Recency decay on top of semantic search\n\nPure vector similarity returns the most semantically similar memories, not the most useful ones. A fact from 90 days ago that's a 0.95 similarity match is often less useful than a 0.80 match from yesterday.\n\nScore formula:\n\n```\nscore = α × cosine_similarity + (1 − α) × exp(−λ × age_days)\n```\n\nWhere `λ = ln(2) / half_life_days` (default: 30 days, so a 30-day-old memory has 50% recency weight).\n\n`α` is configurable per request (default 0.7). Task-focused agents use higher α (semantic dominates). Conversational agents use lower α (recency matters more).\n\n### 2. Fetch 3× candidates, re-rank in Python\n\nThe pgvector query returns `top_k × 3` candidates sorted by pure similarity. Python re-ranks with the decay formula and slices to `top_k`. This prevents recency decay from starving high-similarity older memories — they're still in the candidate pool.\n\nAt 10× scale (>1M memories per agent): push the scoring into a Postgres function using `pg_proc` to eliminate the Python re-ranking round-trip.\n\n### 3. asyncpg + explicit pool sizing over SQLAlchemy async\n\nSQLAlchemy adds ORM overhead on every query. The hot retrieval path — embed, query, re-rank — needs to be tight. asyncpg gives direct control over pool min/max (same instinct as tuning HikariCP in Java). pgvector queries require raw SQL for the `<=>` operator anyway.\n\nPool defaults: `min=5, max=20`. Right-size for a single-instance deployment. Override via `DB_MAX_POOL_SIZE` env var.\n\n### 4. Rate limiting in Postgres, not Redis\n\nSliding window counter via upsert. One fewer dependency. Correct under concurrent requests (transactional upsert). At 10× scale with distributed deployments: replace with Redis `INCR + EXPIRE` — atomic operations, no lock contention.\n\n### 5. ivfflat index, not HNSW\n\n`ivfflat` has lower build cost and lower memory footprint — the right tradeoff at small-to-medium scale (<1M vectors). `lists=100` works well up to ~1M rows. At 10× scale: switch to HNSW (`m=16, ef_construction=64`) for better recall at the cost of higher memory and build time.\n\n---\n\n## Running locally\n\n**Prerequisites:** Docker and Docker Compose. No API keys required — the entire stack runs locally.\n\n```bash\ngit clone https://github.com/ayushagrawal288/memex\ncd memex\ndocker compose up\n```\n\nThe API is live at `http://localhost:8000`. Interactive docs at `http://localhost:8000/docs`.\n\n---\n\n## API reference\n\n### Store a memory\n\n```bash\ncurl -X POST http://localhost:8000/v1/memories \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"agent_id\": \"my-agent\",\n    \"user_id\": \"user-123\",\n    \"content\": \"User prefers concise responses and dislikes verbose explanations.\",\n    \"memory_type\": \"semantic\",\n    \"importance\": 1.2\n  }'\n```\n\n```json\n{\n  \"id\": \"3fa85f64-5717-4562-b3fc-2c963f66afa6\",\n  \"agent_id\": \"my-agent\",\n  \"user_id\": \"user-123\",\n  \"content\": \"User prefers concise responses and dislikes verbose explanations.\",\n  \"importance\": 1.2,\n  \"memory_type\": \"semantic\",\n  \"created_at\": \"2026-05-26T10:30:00Z\",\n  \"score\": null\n}\n```\n\n### Search memories\n\n```bash\ncurl -X POST http://localhost:8000/v1/memories/search \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"agent_id\": \"my-agent\",\n    \"user_id\": \"user-123\",\n    \"query\": \"how does this user like to communicate\",\n    \"top_k\": 5,\n    \"alpha\": 0.7\n  }'\n```\n\n```json\n{\n  \"results\": [\n    {\n      \"id\": \"3fa85f64-...\",\n      \"content\": \"User prefers concise responses and dislikes verbose explanations.\",\n      \"memory_type\": \"semantic\",\n      \"created_at\": \"2026-05-26T10:30:00Z\",\n      \"score\": 0.8921\n    }\n  ],\n  \"query\": \"how does this user like to communicate\",\n  \"total\": 1\n}\n```\n\n### Memory types\n\n| Type | Use for |\n|---|---|\n| `episodic` | Specific events, past conversations |\n| `semantic` | Facts, preferences, general knowledge |\n| `procedural` | Workflows, how-to instructions |\n\n---\n\n## Load test results\n\nRun on a MacBook M-series, Docker Desktop, single Postgres instance:\n\n```bash\nlocust -f scripts/load_test.py --host=http://localhost:8000 \\\n       --headless -u 50 -r 10 -t 60s\n```\n\n**Realistic load** (50 users, 100–300 ms think time — models actual agent traffic):\n\n| Endpoint | RPS | p50 (ms) | p95 (ms) | p99 (ms) | Error rate |\n|---|---|---|---|---|---|\n| POST /v1/memories (write) | 27 | 160 | 270 | 330 | 0% |\n| POST /v1/memories/search | 83 | 110 | 200 | 250 | 0% |\n| Aggregated | **113** | 120 | 230 | 300 | **0%** |\n\n**Saturation test** (500 users, minimal think time — finds the throughput ceiling):\n\n| Endpoint | RPS (plateau) | p50 (ms) | p99 (ms) | Error rate |\n|---|---|---|---|---|\n| POST /v1/memories (write) | 28 | 3,900 | 6,100 | **0%** |\n| POST /v1/memories/search | 91 | 3,600 | 5,800 | **0%** |\n| Aggregated | **~120** | 3,700 | 5,900 | **0%** |\n\n> Run on MacBook M-series, Docker Desktop (4 CPUs), 4 uvicorn workers, 16 threads/worker.  \n> Embeddings: local ONNX (`BAAI/bge-small-en-v1.5`) — zero external API calls, zero cost.\n\n**Why the ceiling is ~120 RPS:**  \nEvery write and every search requires one ONNX inference (~10–15 ms on CPU). With 4 Docker CPUs: `4 cores / 12 ms ≈ 333 embeddings/s` theoretical max. After Python overhead, DB queries, and asyncio scheduling: ~120 RPS actual.\n\n**Path to higher throughput:**\n\n| Approach | Expected gain | Complexity |\n|---|---|---|\n| Embedding cache (Redis, key = SHA256 of text) | 2–3× (40–60% hit rate on repeated agent queries) | Low |\n| Horizontal scaling (N replicas behind a load balancer) | N× linear | Medium |\n| GPU inference (swap ONNX runtime → CUDA) | 10–50× | Medium |\n| Voyage-3 API (offload to Anthropic's inference fleet) | Scales to thousands of RPS, limited by API quota | Low code change |\n\n---\n\n## Project structure\n\n```\nmemex/\n├── app/\n│   ├── main.py                  # REST API — FastAPI, lifespan, router registration\n│   ├── mcp_server.py            # MCP server — single-worker FastAPI on port 8001\n│   ├── core/\n│   │   └── config.py            # All settings, loaded from env\n│   ├── db/\n│   │   └── pool.py              # asyncpg pool, migrations\n│   ├── models/\n│   │   └── schemas.py           # Pydantic request/response models\n│   ├── services/\n│   │   ├── embeddings.py        # fastembed ONNX inference (local, zero API calls)\n│   │   ├── local_summarizer.py  # Extractive summariser — Jaccard dedup + TF scoring\n│   │   ├── memory.py            # Core write/search/scoring logic\n│   │   ├── metrics.py           # Prometheus metric definitions\n│   │   ├── summarizer.py        # Background summarisation job\n│   │   └── rate_limit.py        # Sliding window rate limiter\n│   └── api/routes/\n│       ├── memories.py          # Memory endpoints\n│       ├── health.py            # Health + readiness\n│       └── mcp_tools.py         # MCP tool definitions (store, search, delete, count)\n├── scripts/\n│   └── load_test.py             # Locust load test\n├── docker-compose.yml\n├── Dockerfile\n└── requirements.txt\n```\n\n---\n\n## Observability\n\n`docker compose up` starts Prometheus and Grafana alongside the API:\n\n| Service | URL | Credentials |\n|---|---|---|\n| REST API docs | http://localhost:8000/docs | — |\n| MCP server | http://localhost:8001/mcp/ | — |\n| Prometheus | http://localhost:9090 | — |\n| Grafana | http://localhost:3000 | admin / admin |\n\nThe Grafana dashboard is provisioned automatically. Panels:\n\n- **HTTP request rate + latency p50/p99** — from `prometheus-fastapi-instrumentator`\n- **Embedding API latency p50/p99** — per-attempt histogram by operation (`embed` / `embed_batch`)\n- **Memory operations/s** — create, search, delete throughput\n- **DB pool utilisation** — active vs idle connections (update interval: 15 s)\n- **Summariser activity** — memories condensed per hour, run outcomes\n- **Embedding errors/min** — by operation and error type\n\nCustom metrics are in `app/services/metrics.py` and exposed on `/metrics` alongside the standard FastAPI instrumentator metrics.\n\n---\n\n## MCP endpoint\n\nmemex exposes itself as an [MCP](https://modelcontextprotocol.io) server so any MCP-aware agent (Claude Desktop, Claude Code, custom agents) can store and retrieve memories without custom HTTP integration.\n\n**Transport:** Streamable HTTP (MCP 2024-11-05 spec). Single-worker process on port 8001 — session state is in-process, so a separate service avoids sticky-session complexity while keeping the REST API's multi-worker throughput.\n\n**Tools:**\n\n| Tool | Description |\n|---|---|\n| `store_memory` | Embed + persist a memory (type, importance configurable) |\n| `search_memories` | Semantic + recency ranked retrieval with configurable alpha |\n| `delete_memory` | Forget a specific memory by UUID |\n| `count_memories` | How many memories an agent/user pair has |\n\n### Connect from Claude Desktop\n\nAdd to `~/.config/claude/claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"memex\": {\n      \"type\": \"streamable-http\",\n      \"url\": \"http://localhost:8001/mcp/\"\n    }\n  }\n}\n```\n\n### Connect from Claude Code\n\n```bash\nclaude mcp add --transport http memex http://localhost:8001/mcp/\n```\n\n### Design: why a separate service\n\nThe MCP Streamable HTTP transport is session-stateful — `initialize`, `tools/list`, and `tools/call` must all reach the **same server process**. The REST API runs 4 uvicorn workers with round-robin routing; routing different MCP requests to different workers breaks session state.\n\nRunning a dedicated single-worker MCP service on port 8001 avoids sticky-session infrastructure (nginx `ip_hash`, Redis session store) while keeping the REST API fully multi-worker.\n\n---\n\n## Memory summarisation\n\nRuns as a background asyncio task on a configurable interval (default: every 5 minutes). Finds any `(agent_id, user_id)` pair where episodic memory count exceeds a threshold, condenses the oldest batch into a single `semantic` memory, then deletes the originals. **Fully local — no LLM API calls.**\n\n**How it summarises:** Pure Python extractive algorithm. Sentences are deduplicated by Jaccard similarity (≥ 0.7 threshold), scored by word frequency (TF), and the top-N are returned in original order. ~1 ms per summarisation, zero dependencies beyond the standard library.\n\n**Why episodic-only:** Episodic memories are conversation events with natural time-based obsolescence. Semantic and procedural memories encode facts and skills — silently condensing them risks precision loss; they age out via recency decay instead.\n\n**Concurrency safety:** Uses `pg_try_advisory_xact_lock` keyed on `hashtext(agent_id|user_id)`. The lock is held only during the DB write transaction, not during the embedding call.\n\nTune via env vars:\n\n| Var | Default | Description |\n|---|---|---|\n| `SUMMARIZATION_ENABLED` | `true` | Toggle the background job |\n| `SUMMARIZATION_THRESHOLD` | `100` | Episodic count to trigger per pair |\n| `SUMMARIZATION_BATCH_SIZE` | `50` | Oldest N memories to condense per run |\n| `SUMMARIZATION_INTERVAL_SECONDS` | `300` | How often the job wakes up |\n\n---\n\n## What's next\n\n- [x] **Memory summarisation** — background job to condense old episodic memories (local extractive algorithm, zero API calls) when count exceeds threshold\n- [x] **Prometheus + Grafana** — p50/p99 latency dashboards, embedding API call duration, pool saturation\n- [x] **MCP-compatible endpoint** — Streamable HTTP server on port 8001; 4 tools (store, search, delete, count); connects to Claude Desktop and Claude Code\n- [ ] **HNSW index option** — flag to switch from ivfflat to HNSW for deployments with >1M vectors\n- [ ] **Importance-weighted retrieval** — factor `importance` score into ranking formula alongside similarity and recency\n\n---\n\n## Tech stack\n\n| Layer | Choice | Why |\n|---|---|---|\n| API | FastAPI + uvicorn | Async-first, fast, excellent OpenAPI generation |\n| Embeddings | fastembed ONNX (`BAAI/bge-small-en-v1.5`) | Local, zero API calls, ~12 ms CPU inference, 384-dim |\n| Database | PostgreSQL 16 + pgvector | Relational + vector in one system, no extra infra |\n| Vector index | ivfflat | Lower build cost than HNSW at this scale |\n| Pool | asyncpg | Direct control, zero ORM overhead |\n| Summariser | Pure Python extractive | Jaccard dedup + TF scoring, zero ML deps, ~1 ms |\n| Retry | tenacity | Jitter-based backoff on transient errors |\n| Metrics | Prometheus + prometheus-fastapi-instrumentator | Standard observability |\n| Load testing | Locust | Python-native, realistic user simulation |\n",
  "bytes": 14489,
  "sha": "e4870e63d03c41aa4334ef222f86cd666307827ccceaff719088e590c55a8e44",
  "repo_slug": "ayushagrawal288/memex",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_ayushagrawal288_memex_c2e09464/readme"
}