{
  "markdown": "# colabfit-mcp\n\nAn MCP server for discovering [ColabFit](https://materials.colabfit.org) datasets and training MACE interatomic potentials using [KLIFF](https://kliff.readthedocs.io/) and [KLAY](https://github.com/openkim/klay).\n\n## Overview\n\nThis is a **Model Context Protocol (MCP) server** that gives AI assistants the ability to:\n- Search and download scientific datasets from [ColabFit](https://materials.colabfit.org)\n- Train MACE interatomic potentials on your local hardware (GPU or CPU)\n- Run energy/forces calculations and validate models with OpenKIM test drivers\n\nIt bridges conversational AI and local compute — the AI agent searches for data, trains\nmodels, and runs simulations on your machine through this server.\n\n## Prerequisites\n\n- **Docker and Docker Compose v2** — for the containerized server\n- **Git** — for cloning the repository\n- **make** — for the quick-start commands (optional; manual steps are documented below)\n- **(Optional) NVIDIA GPU + drivers** — for GPU-accelerated training\n- **(Optional) [nvidia-container-toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html)** — required for Docker to access the GPU\n\nFor local (non-Docker) installation, only Python 3.10+ is required. See [Local Installation](#local-installation-without-docker).\n\n## Setup\n\n### Quick Start (Recommended)\n\n```bash\ngit clone https://github.com/colabfit/colabfit-mcp.git\ncd colabfit-mcp\n\n# One-time setup: creates data directories and .env file\nmake setup\n\n# Build Docker images with your user ID for proper permissions\nmake build\n```\n\nThen register the MCP server with your client (see [Register the MCP server](#register-the-mcp-server) below) and restart your client. The container starts automatically when your AI client connects.\n\nRun `make help` to see all available commands.\n\n### Manual Setup\n\nIf you prefer not to use the Makefile:\n\n#### 1. Configure environment\n\n```bash\ncp example.env .env\n# Edit .env to customize data directory location if desired\n```\n\n#### 2. Create data directories\n\n```bash\n# Default location\nmkdir -p ./colabfit_data/models ./colabfit_data/datasets ./colabfit_data/inference_output ./colabfit_data/test_driver_output\n\n# Or custom location (must match COLABFIT_DATA_ROOT in .env)\n# mkdir -p /your/custom/path/{models,datasets,inference_output,test_driver_output}\n```\n\n#### 3. Build with user ID mapping\n\n```bash\n# This ensures the container user matches your host user and selects the right\n# Dockerfile for your platform (CPU-only on macOS, GPU on Linux with NVIDIA)\nUSER_ID=$(id -u) GROUP_ID=$(id -g) ./start.sh build\n```\n\n### Register the MCP server\n\n`start.sh` automatically detects NVIDIA GPU availability and enables GPU passthrough when present, falling back to CPU otherwise.\n\n**Claude Code:**\n\n```bash\nclaude mcp add colabfit-mcp -- /path/to/colabfit-mcp/start.sh\n```\n\nReplace `/path/to/colabfit-mcp` with the absolute path to this repository.\nThen restart Claude Code for the new server to take effect.\n\n**Claude Desktop:**\n\nAdd to your Claude Desktop config (`Settings > Developer > Edit Config`):\n\n```json\n{\n  \"mcpServers\": {\n    \"colabfit-mcp\": {\n      \"command\": \"/path/to/colabfit-mcp/start.sh\",\n      \"args\": [\"run\", \"--rm\", \"-i\", \"server\"]\n    }\n  }\n}\n```\n\n**OpenAI Agent (API-based, not ChatGPT app):**\n\nOpenAI agents that support MCP can connect to this server over `stdio` by launching the same command used above.\n\nUse this command as the MCP server entrypoint:\n\n```bash\n/path/to/colabfit-mcp/start.sh\n```\n\nIf your agent framework requires explicit command/args fields, use:\n\n```json\n{\n  \"command\": \"/path/to/colabfit-mcp/start.sh\",\n  \"args\": [\"run\", \"--rm\", \"-i\", \"server\"]\n}\n```\n\nNotes:\n\n- This is for OpenAI API-based agent runtimes that support MCP server registration.\n- The ChatGPT consumer app (including non-Pro accounts) does not provide local `stdio` MCP server registration in the same way as developer agent runtimes.\n- Replace `/path/to/colabfit-mcp` with the absolute path to this repository.\n\n### Generic MCP Client Setup\n\nThe server uses standard MCP `stdio` transport and works with any MCP-compatible client.\n\n**Entry point** (after pip install or in the Docker container):\n\n```bash\ncolabfit-mcp          # registered console script\n# or\npython -m colabfit_mcp\n```\n\n**Testing with mcp-cli:**\n\n```bash\npip install mcp-cli\nmcp-cli run colabfit-mcp -- colabfit-mcp\n```\n\n**Any stdio MCP client** (Gemini, OpenAI agents, Cursor, etc.) can register the server using the same `command` / `args` pattern as Claude Desktop above. The protocol is standardized — all tools use MCP `stdio` transport, no HTTP server or open port is required.\n\n**Python SDK client example:**\n\n```python\nfrom mcp import ClientSession, StdioServerParameters\nfrom mcp.client.stdio import stdio_client\n\nparams = StdioServerParameters(\n    command=\"/path/to/colabfit-mcp/start.sh\",\n    args=[\"run\", \"--rm\", \"-i\", \"server\"],\n)\n\nasync with stdio_client(params) as (read, write):\n    async with ClientSession(read, write) as session:\n        await session.initialize()\n        tools = await session.list_tools()\n        result = await session.call_tool(\"check_status\", {})\n        print(result)\n```\n\nInstall the client library with `pip install mcp`. The server uses JSON-RPC 2.0 over stdio — raw `subprocess.Popen` with hand-crafted JSON will not work; use a proper MCP client library.\n\n> Note: Docker is required for training and inference (heavy dependencies). The `search_datasets`, `check_local_datasets`, `download_dataset`, `build_dataset`, and `check_status` tools work without Docker via a plain pip install.\n\n## Tools\n\n| Tool | Description |\n|------|-------------|\n| `search_datasets` | Search ColabFit database by text, elements, properties, software |\n| `check_local_datasets` | Scan local data directory for downloaded datasets, filter by elements/properties |\n| `download_dataset` | Download a dataset from HuggingFace via KLIFF |\n| `train_mace` | Train a MACE-style KLAY model from scratch using KLIFF |\n| `use_model` | Run energy/forces/relax calculations with a trained KLAY model, or generate a Python snippet |\n| `check_status` | Check GPU, packages, disk, existing models and datasets |\n| `list_test_drivers` | List available kimvv test drivers, optionally filtered by property keyword |\n| `run_test_driver` | Run a kimvv test driver against a trained KLAY model; saves `structures.extxyz` + `results.json` in a timestamped subdirectory; supports multiple structures per call with optional `repeat` for supercell sizing and `async_mode` for slow drivers |\n| `check_test_driver_result` | Check status of an async test driver job and return inline results when complete |\n\n### Available Test Drivers (kimvv)\n\n| Test Driver | Description | Properties |\n|---|---|---|\n| `EquilibriumCrystalStructure` | Equilibrium lattice parameters and cohesive energy | lattice-constant, cohesive-energy |\n| `ElasticConstantsCrystal` | Full elastic constants tensor at zero temperature | elastic-constants |\n| `CrystalStructureAndEnergyVsPressure` | Crystal structure and energy as a function of pressure | energy-vs-pressure |\n| `GroundStateCrystalStructure` | Lowest energy crystal structure among candidates | ground-state-structure |\n| `VacancyFormationEnergyRelaxationVolumeCrystal` | Vacancy formation energy and relaxation volume | vacancy-formation-energy, relaxation-volume |\n| `ClusterEnergyAndForces` | BFGS relaxation of an atomic cluster in a non-periodic box. Use for molecular/non-periodic models. | energy, atomic-forces, relaxed-positions |\n\n## Typical Workflow\n\n1. `search_datasets` — find datasets with the elements/properties you need\n2. `download_dataset` — download from HuggingFace (cached locally for reuse)\n3. `train_mace` — train a MACE-style KLAY model on the downloaded data\n4. `use_model` — run energy/forces/relax calculations or generate a Python snippet\n5. `run_test_driver` — validate the model against OpenKIM-style property tests\n\n## Sample Prompts\n\nThe following prompts work directly in Claude Code or Claude Desktop once the MCP server is registered.\n\n**Explore available data:**\n\n> Search ColabFit for silicon datasets that include forces. Which ones look best for training an interatomic potential?\n\n> What datasets do I have downloaded locally? Do any contain iron with stress data?\n\n**End-to-end training:**\n\n> Find a dataset for copper, download it, and train a MACE model on it. Use default settings.\n\n> I need a potential for lithium phosphate. Search ColabFit for Li and P datasets, pick the most suitable one, and start training.\n\n**Run inference:**\n\n> Use my model at /home/mcpuser/colabfit/models/cu_mace/cu_mace__MO_000000000000_000 to calculate the energy and forces on bulk copper in FCC structure.\n\n> Relax an FCC aluminum structure with my trained model and report the final energy and cell parameters.\n\n> Generate a Python snippet to run the energy calculation on bulk silicon using my KLAY model.\n\n**Validate with test drivers:**\n\n> What test drivers are available for validating my model?\n\n> Run the ElasticConstantsCrystal test driver on my silicon model at /home/mcpuser/colabfit/models/si_mace/si_mace__MO_000000000000_000.\n\n> Run the EquilibriumCrystalStructure and VacancyFormationEnergyRelaxationVolumeCrystal tests on my copper FCC model.\n\n**Check status:**\n\n> Check my GPU status and list all the models and datasets I have locally.\n\n**End-to-end workflow:**\n\n> Search ColabFit for silicon datasets with forces, download the best one, train a MACE model, calculate energy and forces on bulk diamond-cubic silicon, then run the ElasticConstantsCrystal and EquilibriumCrystalStructure test drivers to validate the model. Report the elastic constants and equilibrium lattice parameter when done.\n\n## Stopping / Canceling Training\n\nThe MCP server runs via `docker compose run` (not `docker compose up`), so\n`docker compose down` alone will **not** stop an active training container.\nUse the methods below to stop the server including any in-progress training job.\n\n### Using Makefile\n\n```bash\nmake stop\n```\n\n### Without Makefile\n\n```bash\n# Stop all containers belonging to this project (catches both 'up' and 'run' containers)\ndocker ps -q --filter \"label=com.docker.compose.project=colabfit-mcp\" | xargs -r docker stop\ndocker compose down\n```\n\nIf the project directory is not named `colabfit-mcp`, replace the filter value with your\ndirectory name (lowercased). You can check the label on a running container with:\n\n```bash\ndocker inspect <container-id> --format '{{ index .Config.Labels \"com.docker.compose.project\" }}'\n```\n\n> Training progress is saved as `training.log` inside the model's KIM subdirectory\n> (`<model_name>__MO_000000000000_000/training.log`). Stopping mid-training discards any\n> in-progress epoch; completed epochs and their checkpoints are preserved on disk.\n\n## Monitoring Training Progress\n\nView training output in the following ways:\n\n### 1. Real-time Container Logs (Recommended)\n\nView live training output as it happens:\n\n```bash\n# Using Makefile\nmake logs\n\n# Or directly with docker compose\ndocker compose logs -f server\n```\n\nPress `Ctrl+C` to exit (training continues in background).\n\n### 2. Persistent Log Files\n\nTraining writes log files inside the model's KIM subdirectory:\n\n```bash\n./colabfit_data/models/<model_name>/<model_name>__MO_000000000000_000/training.log\n```\n\n## GPU Support\n\n`start.sh` automatically detects your GPU:\n\n- **NVIDIA GPU present**: starts with `compose.nvidia.yaml` overlay, enabling CUDA passthrough via nvidia-container-toolkit\n- **No NVIDIA GPU**: starts without the overlay; the container selects the best available device (MPS or CPU) automatically at runtime\n\nThe pip-installed version handles GPU detection purely in Python via `detect_device()` — no shell wrapper needed, since PyTorch can see the host GPU directly.\n\n## Local Installation (without Docker)\n\n### Install\n\n```bash\npip install colabfit-mcp\n```\n\nThis enables `search_datasets`, `check_local_datasets`, `download_dataset`, `build_dataset`,\nand `check_status`. Training and inference require Docker — the full dependency stack\n(CUDA, kim-api, PyG wheels) is only supported via the Docker build.\n\n### Register with Claude Code\n\n```bash\nclaude mcp add colabfit-mcp -- colabfit-mcp\n```\n\n### Register with Claude Desktop\n\nAdd to your Claude Desktop config (`Settings > Developer > Edit Config`):\n\n```json\n{\n  \"mcpServers\": {\n    \"colabfit-mcp\": {\n      \"command\": \"colabfit-mcp\"\n    }\n  }\n}\n```\n\n### Data directory\n\nBy default, datasets and models are stored under `~/colabfit/`. Override with:\n\n```bash\nexport COLABFIT_DATA_ROOT=/your/preferred/path\n```\n\nSubdirectories are created automatically the first time each tool writes data.\n\n### Requirements\n\n- Python 3.10+\n- CUDA 12.x + nvidia drivers (for GPU training; CPU fallback works without CUDA)\n\n## Architecture\n\n```\nserver container\n├── MCP server (FastMCP, stdio)\n├── KLIFF (dataset loading, training orchestration)\n├── KLAY (MACE-style model construction)\n└── Training via KLIFF GNNLightningTrainer\n```\n\nDatasets are downloaded from HuggingFace (`colabfit/` org) as parquet/arrow files via KLIFF's\n`Dataset.from_huggingface` and cached locally. Models are MACE-style graphs\nbuilt with KLAY and trained with KLIFF's Lightning trainer.\n\nContainer managed by Docker Compose:\n- **server** — MCP server + ML training\n\n## Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `COLABFIT_DATA_ROOT` | `./colabfit_data` | Host-side bind-mount source directory. Inside the container the data root is always `/home/mcpuser/colabfit`. |\n| `USER_ID` | `1000` | User ID for container (should match host user) |\n| `GROUP_ID` | `1000` | Group ID for container (should match host user) |\n| `KLIFF_BATCH_SIZE` | `4` | Training batch size. Decrease if OOM. |\n| `KLIFF_NUM_WORKERS` | `0` | DataLoader worker processes. Keep at 0 to avoid CUDA fork deadlocks. |\n| `TRAIN_SIZE` | `0` | Number of training configs (0 = auto 90% split) |\n| `VAL_SIZE` | `0` | Number of validation configs (0 = auto 10% split) |\n| `KLIFF_DTYPE` | `float32` | Training precision (`float32` default; use `float64` for higher accuracy) |\n| `COLABFIT_BASE_URL` | `https://materials.colabfit.org` | ColabFit API base URL (used by search) |\n| `COLABFIT_AUTH_USER` | `mcp-tool` | ColabFit API auth username (used by search) |\n| `COLABFIT_AUTH_PASS` | `mcp-secret` | ColabFit API auth password (used by search) |\n\n**Data Storage:**\n\nBy default, models and datasets are stored in `./colabfit_data/` (relative to the\nproject root), making data portable with the project. `COLABFIT_DATA_ROOT` controls\nonly the **host-side** bind-mount source — the container-internal data root is always\n`/home/mcpuser/colabfit` regardless of this setting. To use a fixed host location that\npersists across project clones, set `COLABFIT_DATA_ROOT` in `.env`:\n\n```bash\ncp example.env .env\n# Edit .env and set: COLABFIT_DATA_ROOT=/home/yourusername/ml_data\n```\n\n```\nHost machine                        Docker container\n─────────────                       ────────────────\n${COLABFIT_DATA_ROOT}/              /home/mcpuser/colabfit/\n├── datasets/          ← bind mount →  ├── datasets/\n├── models/            ← bind mount →  ├── models/\n├── inference_output/  ← bind mount →  ├── inference_output/\n└── test_driver_output/← bind mount →  └── test_driver_output/\n```\n\n**User ID Mapping:**\n\nThe `USER_ID` and `GROUP_ID` variables ensure the container user matches your host\nuser, preventing permission issues with bind-mounted directories. The Makefile\nautomatically detects your IDs, but you can override them in `.env` if needed.\n\n## Requirements\n\nSee [Prerequisites](#prerequisites) for the full list. In short: Docker + Compose v2 for the containerized server, or Python 3.10+ for local installation.\n\n> **HPC / cluster users:** Docker is typically unavailable on HPC systems. Apptainer (formerly Singularity) can pull and convert Docker images (`apptainer pull docker://...`), but the Docker Compose lifecycle and `start.sh` MCP registration do not translate directly to an HPC environment. Native Apptainer/Podman support is a planned future goal.\n\n## Troubleshooting\n\n**`torch_scatter` fails to install with \"torch not found\"**: When installing into an existing\nPython environment (e.g. a KDP container or a system Python), pip's build isolation prevents\nthe build from seeing an already-installed `torch`. Use `--no-build-isolation`:\n\n```bash\npython -m pip install --no-build-isolation torch-scatter\n```\n\nThen reinstall the package to pick up the newly available extension:\n\n```bash\npip install -e \".[full]\"\n```\n\n**GPU not detected in container**: Ensure `nvidia-container-toolkit` is\ninstalled and the Docker daemon has been restarted. Verify with\n`docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi`.\nIf no NVIDIA GPU is present, use `./start.sh` which falls back to CPU automatically.\n\n**MCP server not responding**: The server uses stdio transport, not HTTP. It\nmust be launched via `docker compose run --rm -i server`, not accessed\nover a network port.\n\n---\n\n## Manual Usage: Running Inference with a Trained KLAY Model\n\nAfter training, the model directory (`model_path` returned by `train_mace`) contains\n`model.pt` and `kliff_graph.param`. Use these directly with PyTorch and KLIFF.\n\n### Loading and Running the Model\n\n```python\nimport numpy as np\nimport torch\nfrom torch_scatter import scatter_add\nfrom kliff.dataset import Configuration\nfrom kliff.transforms.configuration_transforms.graphs.generate_graph import RadialGraph\nfrom ase.build import bulk\n\natoms = bulk(\"Si\", \"diamond\", a=5.43)\n\nmodel_dir = \"/home/mcpuser/colabfit/models/colabfit_mace/colabfit_mace__MO_000000000000_000\"\n\n# Load model (tries TorchScript first, falls back to torch.load)\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\ntry:\n    model = torch.jit.load(f\"{model_dir}/model.pt\", map_location=device)\nexcept Exception:\n    model = torch.load(f\"{model_dir}/model.pt\", map_location=device, weights_only=False)\nmodel.eval()\nmodel_dtype = next(model.parameters()).dtype  # match training precision (float32 or float64)\n\n# Build graph — read species/cutoff from kliff_graph.param\ntransform = RadialGraph(species=[\"Si\"], cutoff=5.0, n_layers=1)\nconfig = Configuration(\n    cell=atoms.cell.array,\n    species=list(atoms.get_chemical_symbols()),\n    coords=atoms.get_positions(),\n    PBC=list(atoms.get_pbc()),\n    energy=0.0,\n    forces=np.zeros((len(atoms), 3)),\n)\ngraph = transform(config)\n\ncoords = graph.coords.clone().detach().to(model_dtype).to(device).requires_grad_(True)\nenergy = model(\n    species=graph.species.to(device),\n    coords=coords,\n    edge_index0=graph.edge_index0.to(device),\n    contributions=graph.contributions.to(device),\n)\nprint(f\"Energy: {energy.sum().item():.4f} eV\")\n\n# Forces via autograd\n(grad,) = torch.autograd.grad(energy.sum(), coords)\nforces = -scatter_add(grad, graph.images.to(device), dim=0)[:len(atoms)]\nprint(f\"Forces (eV/Å):\\n{forces.detach().cpu().numpy()}\")\n```\n\n### Geometry Optimization with ASE\n\nThe `use_model` tool's `_KliffInlineCalculator` wraps the KLAY model as an ASE\ncalculator. For custom scripts, replicate the same pattern:\n\n```python\nfrom ase.optimize import BFGS\n\n# (attach _KliffInlineCalculator from use_model module, or replicate the pattern)\nopt = BFGS(atoms, trajectory=\"relax.traj\")\nopt.run(fmax=0.01)  # converge forces below 0.01 eV/Å\n```\n\n[mcp-name: io.github.colabfit/colabfit-mcp]: #\n",
  "bytes": 19475,
  "sha": "e25ebb476a35563cf432bc806bce1d71ac6763cf8e9408e35e0ed848cd43865a",
  "repo_slug": "colabfit/colabfit-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_colabfit_colabfit_mcp_e2b02fe6/readme"
}