\n \n**If you find this project helpful, please give us a โญ๏ธ on GitHub for the latest update.**\n\n_๐ค Contributions welcome! Feel free to open an issue or submit a pull request._\n\n
\n\nTeleMem is an agent memory management layer that can be used as **a high-performance drop-in replacement for [Mem0](https://mem0.ai/)** with one line of code (`import telemem as mem0`), deeply optimized for complex scenarios involving **multi-turn dialogues**, **character modeling**, **long-term information storage**, and **semantic retrieval**.\n\nThrough its unique **context-aware enhancement mechanism**, TeleMem provides conversational AI with core infrastructure offering **higher accuracy**, **faster performance**, and **stronger character memory capabilities**.\n\nBuilding upon this foundation, TeleMem implements **video understanding, multimodal reasoning, and visual question answering** capabilities. Through a complete pipeline of video frame extraction, caption generation, and vector database construction, AI Agents can effortlessly **store, retrieve, and reason over video content** just like handling text memories.\n\nThe ultimate goal of the TeleMem project is to _use an agent's hindsight to improve its foresight_. \n\n**TeleMem, where memory lives on and intelligence grows strong.**\n\n### Why TeleMem?\n\n- ๐ญ **Character memory done right** โ the only open-source memory layer that automatically builds **isolated, per-character memory profiles**, built for role-play, companion AI, NPCs, and multi-persona assistants.\n- ๐ฌ **Memory for video, not just text** โ a full video โ frames โ captions โ vector DB pipeline with **ReAct-style multi-step video QA**.\n- ๐ **Fully local by default** โ runs end-to-end on your hardware (Qwen + FAISS); no cloud service, no paid tier, no data leaving your machine.\n- ๐ **mem0-compatible API** โ `add()` / `search()` accept the same arguments and return the same `{\"results\": [...]}` shapes, so existing Mem0 code keeps working.\n\n---\n\n## ๐ข Latest Updates\n- **[2026-08-15] ๐ TeleMem [v1.10.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.10.0) adds first-class [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) support: an opt-in Cordis/MCP patch starts TeleMem with `uvx`, registers all 8 memory tools under `mcp__telemem__*`, and securely forwards provider configuration. See the [MCP server docs](docs/MCP.md#deepseek-harness).**\n- **[2026-08-06] ๐ TeleMem [v1.9.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.9.0) is on the latest MCP standard: migrated to the official [MCP Python SDK v2](https://github.com/modelcontextprotocol/python-sdk) (spec 2026-07-28) โ all 8 tools now declare titles, behavior annotations, and structured output, while staying compatible with older MCP clients. See the [MCP server docs](docs/MCP.md).**\n- **[2026-07-11] ๐ TeleMem [v1.8.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.8.0) โ the \"claims = contracts\" release: character-memory extraction fix, `infer=False`/`prompt`/`memory_type` now fully honored, offline contract test suite, telemetry disabled by default, and a [multi-NPC demo](examples/multi_npc.py)!**\n- **[2026-06-12] ๐ TeleMem [v1.7.1](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.7.1) is live on the [official MCP registry](https://registry.modelcontextprotocol.io) โ run the memory server with zero install: `uvx telemem`! Also new: [evaluation principles](https://teleai-uagi.github.io/telemem/evaluation/) and a LongMemEval harness with built-in baselines.**\n- **[2026-06-12] ๐ TeleMem is now on PyPI: `pip install telemem`! [v1.6.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.6.0) adds Ollama/DeepSeek/Kimi configs, LangChain & LlamaIndex examples, and a [documentation site](https://teleai-uagi.github.io/telemem/).**\n- **[2026-06-12] ๐ TeleMem [v1.5.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.5.0) has been released: true mem0 drop-in API, lightweight core install, and CI!**\n- **[2026-06-11] ๐ TeleMem [v1.4.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.4.0) has been released with [MCP support](docs/MCP.md)!**\n- **[2026-01-28] ๐ TeleMem [v1.3.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.3.0) has been released!**\n- **[2026-01-22] ๐ TeleMem [Tech Report](https://arxiv.org/abs/2601.06037) has been updated to its 4th version!**\n- **[2026-01-13] ๐ TeleMem [Tech Report](https://arxiv.org/abs/2601.06037) has been released on arXiv!**\n- **[2026-01-09] ๐ TeleMem [v1.2.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.2.0) has been released!**\n- **[2025-12-31] ๐ TeleMem [v1.1.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.1.0) has been released!**\n- **[2025-12-05] ๐ TeleMem [v1.0.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.0.0) has been released!**\n\n---\n\n## ๐ฅ Research Highlights\n\n* **Significantly improved memory accuracy**: Achieved **86.33%** accuracy on the ZH-4O Chinese multi-character long-dialogue benchmark, **19% higher** than Mem0.\n* **Doubled speed performance**: Millisecond-level semantic retrieval enabled by efficient buffering and batch writing.\n* **Greatly reduced token cost**: Optimized token usage delivers the same performance with significantly lower LLM overhead.\n* **Precise character memory preservation**: Automatically builds independent memory profiles for each character, eliminating confusion.\n* **Automated Video Processing Pipeline**: From raw video โ frame extraction โ caption generation โ vector database, fully automated\n* **ReAct-Style Video QA**: Multi-step reasoning + tool calling for precise video content understanding\n\n---\n\n## ๐ Table of Contents\n\n* [Project Introduction](#project-introduction)\n* [TeleMem vs Mem0: Core Advantages](#telemem-vs-mem0-core-advantages)\n* [Experimental Results](#experimental-results)\n* [Quick Start](#quick-start)\n* [Project Structure](#project-structure)\n* [Core Functions](#core-functions)\n* [Multimodal Extensions](#multimodal-extensions)\n* [MCP Server](#mcp-server)\n* [Framework Integrations](#framework-integrations)\n* [Data Storage Explanation](#data-storage)\n* [Telemetry & Privacy](#telemetry--privacy)\n* [Development and Contribution](#development-and-contribution)\n* [Acknowledgements](#acknowledgements)\n* [Citation](#-citation)\n\n---\n\n## Project Introduction\n\nTeleMem enables conversational AI to maintain stable, natural, and continuous worldviews and character settings during long-term interactions through a deeply optimized pipeline of **character-aware summarization โ semantic clustering deduplication โ efficient storage โ precise retrieval**.\n\n```mermaid\nflowchart LR\n A[\"Dialogue messages\"] --> B[\"Character-aware summarization (global + per-character)\"]\n B --> C[\"Embedding + similar-memory retrieval\"]\n C --> D[\"Write buffer (batch flush)\"]\n D --> E[\"LLM semantic clustering & fusion\"]\n E --> F[(\"FAISS index + JSON metadata\")]\n Q[\"Query\"] --> S[\"Vector search + rerank\"]\n F --> S\n S --> R[\"results\"]\n```\n\n### Features\n\n- **Automatic memory extraction**: Extracts and structures key facts from dialogues.\n- **Semantic clustering & deduplication**: Uses LLMs to semantically merge similar memories, reducing conflicts and improving consistency.\n- **Character-profiled memory management**: Builds independent memory archives for each character in a dialogue, ensuring precise isolation and personalized management.\n- **Efficient asynchronous writing**: Employs a buffer + batch-flush mechanism for high-performance, stable persistence.\n- **Precise semantic retrieval**: Combines **FAISS + JSON dual storage** for fast recall and human-readable auditability.\n\n### Applicable Scenarios\n\n* Multi-character virtual agent systems\n* Long-memory AI assistants (e.g., customer service, companionship, creative co-pilots)\n* Complex narrative/world-building in virtual environments\n* Dialogue scenarios with strong contextual dependencies\n* Video content QA and reasoning\n* Multimodal agent memory management\n* Long video understanding and information retrieval\n \n \n\n---\n\n## TeleMem vs Mem0: Core Advantages\n\nTeleMem deeply refactors Mem0 to address **characterization**, **long-term memory**, and **high performance**. Key differences:\n\n| Capability Dimension | Mem0 | TeleMem |\n| -------------------------- | --------------------------- | ------------------------------------------------------------ |\n| Multi-character separation | โ Not supported | โ Automatically creates **independent memory profiles** per character |\n| Summary quality | Basic summarization | โ **Context-aware + character-focused prompts** covering key entities, actions, and timestamps |\n| Deduplication mechanism | Vector similarity filtering | โ **LLM-based semantic clustering**: merges similar memories via LLM |\n| Write performance | Streaming, single writes | โ **Batch flush + concurrency**: 2โ3ร faster writes |\n| Storage format | SQLite / vector DB | โ **FAISS + JSON metadata dual-write**: fast retrieval + human-readable |\n| Multimodal Capability | Single image to text only | โ **Video Multimodal Memory**: Full video processing pipeline + ReAct multi-step reasoning QA |\n---\n\n## Experimental Results\n\n### Dataset\n\nWe evaluate the ZH-4O Chinese long-character dialogue dataset constructed in the paper [MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialogues](https://arxiv.org/abs/2509.11860):\n\n- Average dialogue length: **600 turns per conversation**\n- Scenarios: daily interactions, plot progression, evolving character relationships\n\nMemory capability was assessed via QA benchmarks, e.g.:\n\n```json\n{\n\"question\": \"What is Zhao Qi's nickname for Bai Yulan? A Xiaobai B Xiaoyu C Lanlan D Yuyu\",\n\"answer\": \"A\"\n},\n{\n\"question\": \"What is the relationship between Zhao Qi and Bai Yulan? A Classmates B Teacher and student C Enemies D Neighbors\",\n\"answer\": \"B\"\n}\n```\n\n### Experimental Configuration\n\n- LLM: [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) (thinking mode disabled)\n- Embedding model: [Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B)\n- Metric: QA accuracy\n\n | Method | Overall(%) |\n |:--------------------------------------------------------- |:---------- |\n | RAG | 62.45 |\n | _[Mem0](https://github.com/mem0ai/mem0)_ | _70.20_ |\n | [MOOM](https://github.com/cows21/MOOM-Roleplay-Dialogue) | 72.60 |\n | [A-mem](https://github.com/agiresearch/A-mem) | 73.78 |\n | [Memobase](https://github.com/memodb-io/memobase) | 76.78 |\n | **[TeleMem](https://github.com/TeleAI-UAGI/TeleMem)** | **86.33** |\n\n\n\n---\n\n## Quick Start\n\n### Installation\n\n```shell\npip install telemem # core (text memory)\npip install \"telemem[mcp]\" # + MCP server\npip install \"telemem[video]\" # + video/multimodal pipeline\npip install \"telemem[all]\" # everything\n```\n\n### Development Environment\n\nUsing [uv](https://docs.astral.sh/uv/) (recommended โ creates `.venv` from the committed `uv.lock` for a reproducible environment):\n\n```shell\nuv sync --all-extras # install TeleMem (editable) + all extras, incl. MCP\nuv run python examples/quickstart.py\n```\n\nOr with conda + pip:\n\n```shell\n# Create and activate virtual environment\nconda create -n telemem python=3.10\nconda activate telemem\n# Install from source (editable), with the extras you need\npip install -e \".[all]\"\n```\n\n### Example\n\nSet your OpenAI API key:\n```shell\nexport OPENAI_API_KEY=\"your-openai-api-key\"\n```\n\n```python\n# python examples/quickstart.py\nimport telemem as mem0\n\nmemory = mem0.Memory()\n\nmessages = [\n {\"role\": \"user\", \"content\": \"Jordan, did you take the subway to work again today?\"},\n {\"role\": \"assistant\", \"content\": \"Yes, James. The subway is much faster than driving. I leave at 7 o'clock and it's just not crowded.\"},\n {\"role\": \"user\", \"content\": \"Jordan, I want to try taking the subway too. Can you tell me which station is closest?\"},\n {\"role\": \"assistant\", \"content\": \"Of course, James. You take Line 2 to Civic Center Station, exit from Exit A, and walk 5 minutes to the company.\"}\n]\n\nmemory.add(messages=messages, user_id=\"Jordan\")\nresults = memory.search(\"What transportation did Jordan use to go to work today?\", user_id=\"Jordan\")\nfor hit in results[\"results\"]: # same result shape as mem0\n print(hit[\"memory\"])\n```\n\n`Memory()` uses the default provider settings inherited from `mem0ai`. To use the repository's local Qwen + FAISS configuration, load `config/config.yaml` explicitly:\n\n```python\nfrom telemem.utils import load_config\nimport telemem as mem0\n\nconfig = load_config(\"config/config.yaml\")\nmemory = mem0.Memory(config=config)\n```\n\nThe runnable examples also honor the same configuration through `TELEMEM_CONFIG`:\n\n```shell\nTELEMEM_CONFIG=config/config.yaml python examples/quickstart.py\n```\n\n### Using MiniMax as the LLM Provider\n\nTeleMem supports [MiniMax](https://api.minimax.io) as an LLM backend via its OpenAI-compatible API.\nA ready-to-use example config is provided at `config/config.minimax.yaml`.\n\n```shell\nexport MINIMAX_API_KEY=\"your-minimax-api-key\"\nexport OPENAI_API_KEY=\"your-openai-api-key\" # still needed for embeddings\n```\n\n```python\nfrom telemem.utils import load_config\nimport telemem as mem0\n\nconfig = load_config(\"config/config.minimax.yaml\")\nmemory = mem0.Memory(config=config)\n```\n\nKey points for MiniMax usage:\n- **LLM**: MiniMax M3 (1M context, default) via `https://api.minimax.io/v1`; MiniMax M2.7 (204,800 context) is also available. MiniMax-M3 accepts text, image and video input and supports adaptive thinking; MiniMax-M2.7 is text-only with always-on thinking\n- **Regional endpoints**: use `https://api.minimax.io/v1` (global) or `https://api.minimaxi.com/v1` (China) as `openai_base_url`\n- **Temperature**: must be in **(0.0, 1.0]** โ set explicitly (e.g. `0.7`) to avoid out-of-range errors\n- **Embeddings**: MiniMax does not provide a public embedding API; configure a separate embedder (e.g. `text-embedding-3-small`) in the `embedder` section\n\n### More LLM Providers\n\nTeleMem works with **any OpenAI-compatible endpoint**. Ready-to-use config examples ship in `config/`:\n\n| Provider | Config file | LLM | Embeddings | Notes |\n| -------- | ----------- | --- | ---------- | ----- |\n| **Ollama** (fully local) | [`config.ollama.yaml`](config/config.ollama.yaml) | any local model (e.g. `qwen3:8b`) | `nomic-embed-text`, local | **No API key, no cloud** โ everything runs on your machine |\n| **DeepSeek** | [`config.deepseek.yaml`](config/config.deepseek.yaml) | `deepseek-chat` / `deepseek-reasoner` | external (e.g. OpenAI) | `DEEPSEEK_API_KEY` + `OPENAI_API_KEY` |\n| **Moonshot (Kimi)** | [`config.moonshot.yaml`](config/config.moonshot.yaml) | `kimi-k2-0905-preview` | external (e.g. OpenAI) | `.cn` and `.ai` endpoints supported |\n| **MiniMax** | [`config.minimax.yaml`](config/config.minimax.yaml) | `MiniMax-M3` | external (e.g. OpenAI) | see section above |\n\n```shell\nTELEMEM_CONFIG=config/config.ollama.yaml python examples/quickstart.py # 100% local memory\n```\n\n---\n\n## Project Structure\n\n\nExpand/Collapse Directory Structure\n\n```\ntelemem/\nโโโ assets/ # Documentation assets and figures\nโโโ baselines/ # Baseline implementations for comparative evaluation\nโ โโโ RAG # Retrieval-Augmented Generation baseline\nโ โโโ MemoBase # MemoBase memory management system\nโ โโโ MOOM # MOOM dual-branch narrative memory framework\nโ โโโ A-mem # A-mem agent memory baseline\nโ โโโ Mem0 # Mem0 baseline implementation\nโโโ config/ \nโ โโโ config.yaml # TeleMem default configuration\nโ โโโ config.minimax.yaml # MiniMax provider example configuration\nโโโ data/ # Small sample datasets for evaluation or demonstration\nโโโ examples/ # Code examples and tutorial demos\nโ โโโ quickstart.py ย ย ย ย ย ย ย # Quick start\nโ โโโ quickstart_mm.py # Quick start (multimodal)\nโ โโโ mcp_client.py # Quick start over MCP (stdio client)\nโ โโโ mcp_config.json # MCP config snippet for Claude Desktop / Cursor\nโ โโโ deepseek-harness.cordis.yml # DeepSeek Harness memory patch\nโโโ docs/\nโ โโโ MCP.md # MCP server reference\nโ โโโ TeleMem_Tech_Report.pdf\nโโโ telemem/ # Telemem code\nโ โโโ mcp/ # Model Context Protocol server\nโโโ tests/ # Telemem test\nโโโ README.md ย ย ย ย ย ย ย ย ย ย ย ย ย # English README\nโโโ README-ZH.md ย ย ย ย ย ย ย ย ย ย ย # Chinese README\nโโโ pyproject.toml # Python environment\n```\n\n\n\n---\n\n## Core Functions\n\n### Add Memory (add)\n\nThe `add()` method injects one or more dialogue turns into the memory system.\n\n```python\ndef add(\n self,\n messages,\n *,\n user_id: Optional[str] = None,\n agent_id: Optional[str] = None,\n run_id: Optional[str] = None,\n metadata: Optional[Dict[str, Any]] = None,\n infer: bool = True,\n memory_type: Optional[str] = None,\n prompt: Optional[str] = None,\n batch: bool = False,\n)\n```\n\n#### ๐ Parameter Description\n\n| Parameter | Type | Required | Description |\n| ------------- | ------------------------------- | -------- | ------------------------------------------------------------ |\n| `messages` | `str` or `List[Dict[str, str]]` | โ Yes | A single statement, or a list of dialogue messages with `role` (`user`/`assistant`) and `content` |\n| `user_id` | `Optional[str]` | โ No | Character/user to attribute the memory to; TeleMem keeps an **independent memory profile per `user_id`**. Omit it to store shared conversation-event memories |\n| `agent_id` / `run_id` | `Optional[str]` | โ No | Additional mem0-compatible scopes (e.g. one `run_id` per session) |\n| `metadata` | `Optional[Dict[str, Any]]` | โ No | Arbitrary metadata stored with each memory |\n| `infer` | `bool` | โ No | Extract salient facts with the LLM (default: `True`); `False` stores message contents verbatim with no LLM call |\n| `memory_type` | `Optional[str]` | โ No | Pass `\"procedural_memory\"` to create procedural memories via mem0's pipeline; omit for conversational memories |\n| `prompt` | `Optional[str]` | โ No | Custom extraction prompt (replaces the optimized default as the system prompt) |\n| `batch` | `bool` | โ No | Route through the high-throughput batched pipeline (`add_batch`) |\n\n**Returns** the mem0-compatible shape: `{\"results\": [{\"id\": \"...\", \"memory\": \"...\", \"event\": \"ADD\"}, ...]}`\n\n#### ๐ Internal Workflow of `add()`\n\n1. **Message preprocessing**: Merge consecutive messages from the same speaker; normalize turn structure.\n2. **Multi-perspective summarization**:\n - Global event summary\n - Character 1โs perspective (actions, preferences, relationships)\n - Character 2โs perspective\n3. **Vectorization & similarity search**: Generate embeddings and retrieve existing similar memories.\n4. **Batch processing**: When buffer threshold is reached, invoke LLM to **semantically merge** similar memories.\n5. **Persistence**: Dual-write to **FAISS (for retrieval)** and **JSON (for metadata)**.\n\n> ๐ญ **Multi-character demo**: [examples/multi_npc.py](examples/multi_npc.py) runs five tavern\n> NPCs through one scene โ a single `add_batch(scene, user_id=[...])` call gives each NPC a\n> private memory profile plus a shared `\"events\"` world-state, and each NPC then recalls the\n> scene from their own perspective.\n\n---\n\n### Search Memory (search)\n\nPerforms semantic vector-based retrieval of relevant memories with context-aware recall.\n\n```python\ndef search(\n self,\n query: str,\n *,\n user_id: Optional[str] = None,\n agent_id: Optional[str] = None,\n run_id: Optional[str] = None,\n limit: int = 100,\n filters: Optional[Dict[str, Any]] = None,\n threshold: Optional[float] = None,\n rerank: bool = True,\n)\n```\n\n#### ๐ Parameter Description\n\n| Parameter | Type | Required | Description |\n| ----------- | ------------------ | -------- | ------------------------------------------------- |\n| `query` | `str` | โ Yes | Natural language query |\n| `user_id` | `Optional[str]` | โ No | Character/user profile to search. The shared event memories (pseudo-user `\"events\"`) are always searched as well |\n| `agent_id` / `run_id` | `Optional[str]` | โ No | Additional mem0-compatible scope filters |\n| `limit` | `int` | โ No | Max number of results (default: 100) |\n| `threshold` | `Optional[float]` | โ No | Similarity threshold (0โ1; auto-tuned if omitted) |\n| `filters` | `Dict[str, Any]` | โ No | Custom filters (e.g., by character, time range) |\n| `rerank` | `bool` | โ No | Whether to rerank results (default: `True`) |\n\n**Returns** the mem0-compatible shape: `{\"results\": [{\"id\": \"...\", \"memory\": \"...\", \"score\": ..., ...}, ...]}`\n\n> ๐ Search is based on FAISS vector retrieval, supporting millisecond-level responses.\n\n---\n\n## Multimodal Extensions\n\nBeyond text memory, TeleMem further extends multimodal capabilities. Drawing inspiration from [Deep Video Discovery](https://github.com/microsoft/DeepVideoDiscovery)'s Agentic Search and Tool Use approach, we implemented two core methods in the TeleMemory class to support intelligent storage and semantic retrieval of video content.\n\n| Method | Description |\n|------|----------|\n| `add_mm()` | Process video into retrievable memory (frame extraction โ caption generation โ vector database) |\n| `search_mm()` | Query video content using natural language, supporting ReAct-style multi-step reasoning |\n\n### Add Multimodal Memory (add_mm)\n\n```python\ndef add_mm(\n self,\n video_path: str,\n output_dir: str,\n clip_secs: int | None = None,\n emb_dim: int | None = None,\n subtitle_path: str | None = None,\n)\n```\n\n#### ๐ Parameter Description\n\n| Parameter | Type | Required | Description |\n|--------|------|----------|------|\n| video_path | str | โ Yes | Source video file path, e.g., `\"video/3EQLFHRHpag.mp4\"` |\n| output_dir | str | โ Yes | Root output directory. Artifacts are written under `frames/`, `captions/`, and `vdb/` subdirectories |\n| clip_secs | int | โ No | Reserved parameter; clip length is currently read from `config.vlm[\"CLIP_SECS\"]` |\n| emb_dim | int | โ No | Embedding dimension, reads from config by default |\n| subtitle_path | str | โ No | Subtitle file path (.srt), optional |\n\n#### ๐ add_mm() Internal Flow\n\n1. **Frame Extraction**: `decode_video_to_frames` - Decodes video to JPEG frames at configured FPS\n2. **Caption Generation**: `process_video` - Uses VLM (e.g., Qwen3-Omni) to generate detailed descriptions for each clip\n3. **Vector Database Construction**: `init_single_video_db` - Generates embeddings for semantic retrieval\n\n> ๐ก **Smart Caching**: If the target file for a stage already exists, that stage is automatically skipped to save computational resources.\n\n#### Return Value Example\n\n```python\n{\n \"output_dir\": \"/abs/path/to/output_dir\"\n}\n```\n\n---\n\n### Search Multimodal Memory (search_mm)\n\n```python\ndef search_mm(\n self,\n question: str,\n output_dir: str,\n max_iterations: int = 15,\n)\n```\n\n#### ๐ Parameter Description\n\n| Parameter | Type | Required | Description |\n|--------|------|----------|------|\n| question | str | โ Yes | Question string (supports A/B/C/D multiple choice format) |\n| output_dir | str | โ Yes | The same root output directory used by `add_mm`; it must contain exactly one `captions/*/captions.json` and one `vdb/*/*_vdb.json` |\n| max_iterations | int | โ No | Maximum MMCoreAgent reasoning iterations (default 15) |\n\n#### ๐ ๏ธ ReAct-Style Reasoning Tools\n\n`search_mm` internally uses `MMCoreAgent`, employing a THINK โ ACTION โ OBSERVATION loop with three specialized tools:\n\n| Tool Name | Function |\n|--------|------|\n| `global_browse_tool` | Get global overview of video events and themes |\n| `clip_search_tool` | Search for specific content using semantic queries |\n| `frame_inspect_tool` | Inspect frame details within a specific time range |\n\n---\n\n### Multimodal Example\n\nRun the multimodal demo:\n\n```bash\npython examples/quickstart_mm.py\n```\n\nOn the first run, frames, captions and VDB JSON will be generated under the chosen `output_dir`. The repository ships a small sample video; generating captions and the video database still requires configured VLM and embedding services unless you already have these artifacts locally.\n\nComplete code example:\n\n```python\nimport telemem as mem0\nfrom pathlib import Path\nfrom telemem.mm_utils.core import extract_choice_from_msg\n\n# Initialize\nmemory = mem0.Memory()\n\n# Define paths\nrepo_root = Path(__file__).resolve().parents[1]\nvideo_path = repo_root / \"data\" / \"samples\" / \"video\" / \"3EQLFHRHpag.mp4\"\nvideo_name = video_path.stem\noutput_dir = video_path.parent\n\n\n# Step 1: Add video to memory (auto-processing)\nvdb_json_path = output_dir / \"vdb\" / video_name / f\"{video_name}_vdb.json\"\nif not vdb_json_path.exists():\n result = memory.add_mm(\n video_path=str(video_path),\n output_dir=str(output_dir),\n )\n print(f\"Video processing complete: {result}\")\nelse:\n print(f\"VDB already exists: {vdb_json_path}\")\n\n# Step 2: Query video content\nquestion = \"\"\"The problems people encounter in the video are caused by what?\n(A) Catastrophic weather.\n(B) Global warming.\n(C) Financial crisis.\n(D) Oil crisis.\n\"\"\"\n\nmessages = memory.search_mm(\n question=question,\n output_dir=str(output_dir),\n max_iterations=15,\n)\n\n# Extract final answer\nanswer = extract_choice_from_msg(messages)\nprint(f\"Answer: ({answer})\")\n```\n\n---\n\n## MCP Server\n\n[](https://mcpvault.io/servers/telemem/health?utm_source=external_badge&utm_medium=referral&utm_campaign=mcp_health_report)\n\nTeleMem ships a [Model Context Protocol](https://modelcontextprotocol.io) (MCP) server, so any MCP-compatible client โ Claude Desktop, Claude Code, Cursor, custom agents โ can use TeleMem as its long-term memory.\n\n```shell\npip install telemem\n\ntelemem-mcp # stdio (default)\ntelemem-mcp --transport streamable-http # Streamable HTTP on :8421\nTELEMEM_CONFIG=config/config.yaml telemem-mcp # custom TeleMem config\nuvx telemem # zero-install run (stdio)\n```\n\nBuilt on the official MCP Python SDK v2, the server implements the current MCP\nspecification (2026-07-28) while remaining compatible with older clients; every tool\ndeclares titles, behavior annotations (read-only/destructive hints), and structured output.\n\nThe server exposes eight tools: `add_memory`, `search_memories`, `get_memories`, `get_memory`, `update_memory`, `delete_memory`, `delete_all_memories`, and `memory_history`. Calls without an explicit scope default to `TELEMEM_DEFAULT_USER_ID` (`telemem-mcp`); destructive bulk deletion always requires an explicit scope.\n\nClaude Desktop / Cursor configuration ([examples/mcp_config.json](examples/mcp_config.json)):\n\n```json\n{\n \"mcpServers\": {\n \"telemem\": {\n \"command\": \"telemem-mcp\",\n \"env\": {\n \"TELEMEM_CONFIG\": \"/absolute/path/to/config/config.yaml\",\n \"OPENAI_API_KEY\": \"sk-...\"\n }\n }\n }\n}\n```\n\nOr drive it programmatically over stdio โ the quickstart flow as MCP tool calls:\n\n```shell\npython examples/mcp_client.py\n```\n\nSee [docs/MCP.md](docs/MCP.md) for the full tool reference, transports, and client setup.\n\n---\n\n## Framework Integrations\n\nTeleMem drops into any agent framework with the same two calls โ `search()` before answering, `add()` after each exchange:\n\n| Framework | Example | Install |\n| --------- | ------- | ------- |\n| **LangChain** | [examples/langchain_memory.py](examples/langchain_memory.py) | `pip install langchain-core langchain-openai` |\n| **LlamaIndex** | [examples/llamaindex_memory.py](examples/llamaindex_memory.py) | `pip install llama-index-llms-openai` |\n| **Claude Desktop / Cursor / any MCP client** | [MCP Server](#mcp-server) | `pip install \"telemem[mcp]\"` |\n\nBecause TeleMem is mem0 API-compatible, any framework adapter written for Mem0's OSS client also works โ point it at `telemem.Memory` instead.\n\n---\n\n## Data Storage\n\n### Text Memory Storage\n\nTeleMem automatically creates a structured storage layout under `./faiss_db/`, organized by session and character:\n\n```\nfaiss_db/\nโโโ session_001_events.index\nโโโ session_001_events_meta.json \nโโโ session_001_person_1.index \nโโโ session_001_person_1_meta.json \nโโโ session_001_person_2.index \nโโโ session_001_person_2_meta.json \n```\n\n### ๐ Metadata Example (_meta.json)\n\n```json\n{\n \"summary\": \"Characters discussed the upcoming action plan.\",\n \"sample_id\": \"session_001\",\n \"round_index\": 3,\n \"timestamp\": \"2024-01-01T00:00:00Z\",\n \"user\": \"Jordan\" // Only present in person_*.json\n}\n```\n\n> All memories include summary, round number, timestamp, and character, facilitating auditing and debugging.\n\n------\n\n### Multimodal Memory Storage\n\nTeleMem generates video-related storage files in the `.data/samples/video/` directory:\n\n```\nvideo/\nโโโ frames/\nโ โโโ /\nโ โโโ frames/\nโ โโโ frame_000001_n0.00.jpg\nโ โโโ frame_000002_n0.50.jpg\nโ โโโ ...\nโโโ captions/\nโ โโโ /\nโ โโโ captions.json # Clip descriptions + subject registry\nโ โโโ ckpt/ # Checkpoint for resume\nโ โโโ 0_10.json\nโ โโโ 10_20.json\nโโโ vdb/\n โโโ /\n โโโ _vdb.json # Semantic retrieval vector database\n```\n\n#### ๐ captions.json Structure\n\n```json\n{\n \"0_10\": {\n \"caption\": \"The narrator discusses climate data, showing melting glaciers...\"\n },\n \"10_20\": {\n \"caption\": \"Scene shifts to coastal communities affected by rising sea levels...\"\n },\n \"subject_registry\": {\n \"narrator\": {\n \"name\": \"narrator\",\n \"appearance\": [\"professional attire\"],\n \"identity\": [\"climate scientist\"],\n \"first_seen\": \"00:00:00\"\n }\n }\n}\n```\n\n------\n\n## Telemetry & Privacy\n\nTeleMem itself collects no telemetry. The underlying `mem0ai` library ships\nanonymized PostHog usage telemetry, which TeleMem **disables by default**\n(`import telemem` sets `MEM0_TELEMETRY=False` unless you have already set it).\nTo opt back in:\n\n```bash\nexport MEM0_TELEMETRY=true\n```\n\n------\n\n## Development and Contribution\n\n* Issues and pull requests are welcome โ see the [Contributing Guide](CONTRIBUTING.md) to get started.\n* Changes between releases are tracked in the [Changelog](CHANGELOG.md).\n* CI runs the offline test suite (`uv run pytest tests/ -q`) on Python 3.10โ3.12 for every PR.\n* Chinese documentation: [README-ZH.md](README-ZH.md)\n\n---\n## License\n\n[Apache 2.0 License](LICENSE)\n\n---\n\n## Acknowledgements\n\nTeleMemโs development has been deeply inspired by open-source communities and cutting-edge research. We extend our sincere gratitude to the following projects and teams:\n\n- **[Mem0](https://github.com/mem0ai/mem0)**\n- **[Memobase](https://github.com/memodb-io/memobase)**\n- **[MOOM](https://github.com/cows21/MOOM-Roleplay-Dialogue)**\n- **[DVD](https://github.com/microsoft/DeepVideoDiscovery)**\n- **[Memento](https://github.com/Agent-on-the-Fly/Memento)**\n- **[Momento-Skills](https://github.com/Memento-Teams/Memento-Skills)**\n\n---\n\n## ๐ Citation\n\nIf you find TeleMem useful to your research or development, please cite our arXiv paper:\n\n> Chunliang Chen, Ming Guan, Xiao Lin, Jiaxu Li, Luxi Lin, Qiyi Wang, Xiangyu Chen, Jixiang Luo, Changzhi Sun, Dell Zhang, Xuelong Li. **TeleMem: Building Long-Term and Multimodal Memory for Agentic AI.** arXiv:2601.06037, 2026. https://arxiv.org/abs/2601.06037\n\n```bibtex\n@article{chen2026telemem,\n title = {TeleMem: Building Long-Term and Multimodal Memory for Agentic AI},\n author = {Chen, Chunliang and Guan, Ming and Lin, Xiao and Li, Jiaxu and Lin, Luxi and Wang, Qiyi and Chen, Xiangyu and Luo, Jixiang and Sun, Changzhi and Zhang, Dell and Li, Xuelong},\n journal = {arXiv preprint arXiv:2601.06037},\n year = {2026},\n url = {https://arxiv.org/abs/2601.06037}\n}\n```\n\nCitation metadata is also available in [CITATION.cff](CITATION.cff) (GitHub's \"Cite this repository\" button).\n\n---\n\n\n\n
\n\n**If you find this project helpful, please give it a โญ๏ธ โ starring also keeps updates in your GitHub feed.**\n\nMade with โค๏ธ by [Bloo-Mind AI Ltd](https://www.bloo-mind.ai/) and the Ubiquitous AGI team at TeleAI.\n\n
![](\"./assets/TeleMem.png\")
\n \n![\"arXiv\"](\"https://img.shields.io/badge/arXiv-Paper-red\"){kind=icon}
![\"CI\"](\"https://github.com/TeleAI-UAGI/telemem/actions/workflows/ci.yml/badge.svg\"){kind=icon}
\n \n \n ![\"PyPI\"](\"https://img.shields.io/pypi/v/telemem?color=blue\")
![\"GitHub](\"https://img.shields.io/github/stars/TeleAI-UAGI/TeleMem?style=social\")
![\"License:](\"https://img.shields.io/badge/license-Apache%20License%202.0-blue\"){kind=icon}
![\"Last](\"https://img.shields.io/github/last-commit/TeleAI-UAGI/TeleMem?color=blue\")
![\"PRs](\"https://img.shields.io/badge/PRs-Welcome-red\"){kind=icon}
![\"MCPVault:](\"https://mcpvault.io/badge/telemem.svg\"){kind=icon}
\n \n![\"Bloo-Mind](\"assets/bloo-mind.png\")
\n \n ![\"TeleAI](\"assets/TeleAI.png\")
\n