{
  "markdown": "# vllm-ops-mcp\n\n[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n[![tests](https://img.shields.io/badge/tests-102%20%2895%20passing%2C%207%20live--skipped%29-brightgreen)](tests)\n[![CI](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml)\n\nRead-only ops/health MCP server for a local, WSL2/systemd-managed vLLM\nserver -- liveness vs. real-completion health tiers, GPU/VRAM status,\nsystemd service status, and live serve-flag introspection. Built to the same\nstandard as [mcp-factory](https://github.com/jaimenbell/mcp-factory) and\n[desktop-mcp](https://github.com/jaimenbell/desktop-mcp), and other local\ntooling in this operator's portfolio: own pyproject, own fastmcp server,\nhonest README, real test suite, no product code before a spec was confirmed.\n\nThis is **not** a container-lifecycle manager. It assumes vLLM already runs\nas a bare-metal `systemd` unit inside WSL2 (this operator's actual setup --\nsee [vllm-autostart](../vllm-autostart)), not inside Docker/Podman.\n\n## Quickstart\n\n```jsonc\n// ~/.claude.json (or any MCP-host stdio client config)\n{\n  \"mcpServers\": {\n    \"vllm-ops-mcp\": {\n      \"type\": \"stdio\",\n      \"command\": \"C:\\\\Users\\\\<you>\\\\projects\\\\vllm-ops-mcp\\\\.venv\\\\Scripts\\\\python.exe\",\n      \"args\": [\"C:\\\\Users\\\\<you>\\\\projects\\\\vllm-ops-mcp\\\\run_server.py\"]\n    }\n  }\n}\n```\n\n## Tools (Phase 1 -- all read-only, no gating required beyond a throughput cap)\n\n| Tool | What it does | What it can't do |\n|---|---|---|\n| `check_health(deep: bool = False)` | `deep=False`: GET `/v1/models` liveness check (same signal as `llm_router.LocalLLM.is_healthy()`). `deep=True`: additionally fires one minimal real `/v1/chat/completions` call (mirrors `check-vllm.cmd`'s two-stage probe) to confirm the server actually *generates*, not just *reports a model loaded*. | Cannot fix a degraded server -- reports ground truth only. `deep=True` is rate-limited (default 20/min, see below). |\n| `list_models` | `/v1/models` passthrough. | No detail beyond what vLLM's API itself exposes. |\n| `test_completion(prompt, max_tokens=16)` | On-demand real completion with a caller-supplied prompt, for manual sanity checks. | Rate-limited (same bucket as `check_health(deep=True)`); refuses if the liveness check fails first (no point firing a completion at a server that isn't even listening). `max_tokens` is clamped server-side to 1024 (`config.MAX_TEST_COMPLETION_TOKENS`) and the prompt is capped at 8000 chars (`config.MAX_TEST_COMPLETION_PROMPT_CHARS`, oversized prompts are rejected, not truncated) -- the rate limiter only bounds call *frequency*, not the cost of a single call. |\n| `get_gpu_status` | `nvidia-smi` wrapper: per-GPU VRAM used/total + utilization%, plus per-process VRAM via `--query-compute-apps`. | **Per-process VRAM attribution does not work on this operator's actual WSL2 setup** -- live-verified empty even while the vLLM process holds 13.7GB (see Limitations). Treat `processes` as best-effort/often-empty, not a reliable per-PID breakdown. |\n| `get_service_status` | `systemctl show <unit> -p ...` (structured property output, not free-text parsing) via `wsl -d <distro>` from the Windows host, or native `systemctl` if this server itself runs inside WSL2/Linux. Returns load/active/sub state, restart count, main PID, last-active timestamp. | Read-only -- cannot start/stop/restart the unit. No computed uptime duration (deliberately -- see Limitations, clock-discipline note). |\n| `get_serve_config` | Read-only launch-flag introspection. **Prefers the LIVE process's actual argv** (`/proc/<pid>/cmdline` of the systemd unit's `MainPID` -- ground truth of what's running right now), falling back to the static exec script (local mirror via `VLLM_OPS_MCP_SERVE_CONFIG_PATH`, else a live `cat` of the WSL-side exec script) only when the process isn't up. | vLLM doesn't expose its own launch flags over the API, so this never queries the server itself -- it's always shelling out. Exec-script fallback parsing is best-effort shell-text parsing (less precise than the live-argv path). |\n\n### Future extension: `restart_service` (Phase 2, not built)\n\nDeliberately **not implemented in this release**. Per the spec: vLLM is\nshared infra other bots/tools depend on (options-bot, `qwen_cli.py`\ncallers, DeerFlow fallback), so a restart tool needs its own gating design\n(off by default, `VLLM_OPS_MCP_ENABLE_RESTART=1`, refuse to touch a healthy\ninstance unless `force=True`, rate-capped) before it ships -- not\nretrofitted onto a read-only MVP. Tracked as a v2 idea, not a current\ncapability. If built, it should wrap `run-vllm.cmd`'s exact recipe (`systemctl\nstart vllm` only, never blind `stop`/`restart`) rather than reinvent it.\n\n## The differentiator: probe HTTP, not the listener\n\nEvery health signal in this server comes from an actual HTTP request to the\nOpenAI-compatible API (`urllib.request`, not a socket/connect check). This\nis deliberate, not incidental: `Get-NetTCPConnection` and other\nlistener-based checks **miss WSL2-mirrored-networking-relayed ports\nentirely** -- a genuinely healthy `127.0.0.1:8000` can be invisible to a\nlistener probe run from the Windows host. `check-vllm.cmd` already embodies\nthis lesson; this server generalizes it into two tiers (liveness vs. real\ngeneration) that nothing else in this operator's fleet exposes as MCP tools.\n\n## Security\n\nRead this before you register vllm-ops-mcp. Honesty about blast radius is\nthe point.\n\n- **This server shells out to `nvidia-smi`, `systemctl` (via `wsl -d`), and\n  reads `/proc/<pid>/cmdline`.** All six tools are read-only -- nothing here\n  starts, stops, restarts, or kills a process, and no tool accepts an\n  argument that gets passed to a shell unescaped (unit/distro names come\n  from server-side config, not caller input; `shlex.quote` wraps the one\n  caller-adjacent path that does reach a shell string, the unit name in\n  `systemctl show`).\n- **`check_health(deep=True)` and `test_completion` fire a real inference\n  call against a shared GPU server.** Rate-limited (default 20 calls/min,\n  `VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN`) specifically because this vLLM\n  instance is shared infra other bots/tools depend on -- a caller hammering\n  this MCP could starve or slow other consumers even though every call\n  individually looks \"read-only.\"\n- **`get_serve_config`'s live-process path reads `/proc/<pid>/cmdline`,**\n  which can include secrets if any were ever passed as CLI flags (none are,\n  in this operator's actual setup -- the model path and flags are all\n  non-secret; auth, if any, is environment-injected inside the exec script,\n  not passed as an argv flag). Defense-in-depth: any argv entry whose\n  preceding flag name matches a known-sensitive pattern (`token`, `key`,\n  `secret`, `password`, `passwd`, `credential` -- `config.\n  SENSITIVE_ARGV_FLAG_PATTERNS`) has its **value** redacted to\n  `[REDACTED]` before being returned, in both the structured `argv`/`flags`\n  fields and `raw_text`; the flag name itself is left visible so the shape\n  is still diagnosable. This applies across all three sources\n  (`live_process`, `exec_script`, `local_config_mirror`).\n- **No tool has any write/mutate capability in this release.** There is no\n  env var that turns one on -- `restart_service` doesn't exist in this\n  codebase yet (see Future extension above), so there's nothing to\n  accidentally enable.\n\n## Honest-capabilities table\n\n| Claim | Implementation | Verified by |\n|---|---|---|\n| Liveness health tier (`/v1/models`) | `vllm_ops_mcp/probes.py::_probe_models`, `check_health` | `tests/test_probes.py::TestCheckHealthLiveness`; live: `tests/test_live_smoke.py::test_live_check_health_liveness` |\n| Deep/real-completion health tier | `vllm_ops_mcp/probes.py::_probe_completion`, `check_health(deep=True)` | `tests/test_probes.py::TestCheckHealthDeep`; live: `test_live_check_health_deep` |\n| Model list passthrough | `probes.py::list_models` | `tests/test_probes.py::TestListModels`; live: `test_live_list_models` |\n| On-demand completion sanity check | `probes.py::test_completion` | `tests/test_probes.py::TestTestCompletion`; live: `test_live_test_completion` |\n| GPU/VRAM status via nvidia-smi | `probes.py::get_gpu_status` | `tests/test_probes.py::TestGetGpuStatus`; live: `test_live_get_gpu_status` |\n| Per-process VRAM attribution | `probes.py::get_gpu_status` (`--query-compute-apps`) | Unit-tested with a mocked non-empty response; **live-verified to return EMPTY on this operator's actual WSL2 setup** -- see Limitations |\n| systemd unit status via structured `systemctl show` | `probes.py::get_service_status` | `tests/test_probes.py::TestGetServiceStatus`; live: `test_live_get_service_status` |\n| Live-process launch-flag introspection via `/proc/<pid>/cmdline` | `probes.py::get_serve_config`, `parse_launch_flags` | `tests/test_probes.py::TestGetServeConfig`, `TestParseLaunchFlags`; live: `test_live_get_serve_config` -- live-confirmed it recovers flags (`--enable-auto-tool-choice --tool-call-parser qwen3_xml`) that were **not** in this repo's own grounding spec, proving the live-argv approach beats trusting a static doc |\n| Exec-script fallback when the unit is down | `probes.py::get_serve_config`, `_parse_exec_script_text` | `tests/test_probes.py::TestGetServeConfig::test_falls_back_to_exec_script_when_service_down` |\n| Local config-mirror override path | `probes.py::get_serve_config` + `config.serve_config_override_path` | `tests/test_probes.py::TestGetServeConfig::test_uses_local_override_file`, `test_override_file_missing_reports_error` |\n| Sensitive-flag-value redaction in `get_serve_config` | `probes.py::redact_argv`, `redact_raw_text`, `config.SENSITIVE_ARGV_FLAG_PATTERNS` | `tests/test_probes.py::TestGetServeConfig::test_live_process_redacts_sensitive_flag_values`, `test_exec_script_raw_text_redacts_sensitive_flag_values`, `test_local_override_file_redacts_sensitive_flag_values` |\n| Deep-inference rate limiting (shared-infra protection) | `config.py::TokenBucket`, `DeepRateLimiter`, `check_deep_rate_limit` | `tests/test_config.py::TestTokenBucket`, `TestDeepRateLimiter` |\n| WSL-vs-native shell-out abstraction | `probes.py::_shell_args`, `config.runs_in_wsl` | `tests/test_config.py::TestRunsInWsl`; `tests/test_probes.py::TestGetServiceStatus::test_uses_wsl_prefix_on_windows`, `test_no_wsl_prefix_when_native` |\n| All 6 tools registered, no Phase-2 tool present | `vllm_ops_mcp/server.py` | `tests/test_server.py` |\n\n## Limitations (read before relying on this)\n\n- **Per-process VRAM attribution is unreliable under this operator's actual\n  WSL2 GPU-passthrough config.** `get_gpu_status`'s live smoke test (real\n  `nvidia-smi.exe` against the real running vLLM process) returned an\n  **empty** `processes` list even though the same call's `gpus[0].\n  memory_used_mib` correctly showed 13.7GB in use by that process. This is a\n  documented `nvidia-smi --query-compute-apps` limitation under some\n  WSL2/passthrough configurations, not a bug in this server's parsing --\n  the tool reports the empty result honestly (`ok: true, processes: []`)\n  rather than fabricating attribution. Don't rely on `processes` for\n  VRAM-overshoot root-causing on this setup; `gpus[].memory_used_mib` is the\n  reliable signal.\n- **No computed service uptime.** `get_service_status` returns the raw\n  `ActiveEnterTimestamp` string from `systemctl` (already timezone-labeled,\n  e.g. `MDT`) rather than computing an elapsed duration server-side --\n  deliberately, per this operator's own clock-discipline rule (never state\n  an elapsed/ETA time from estimation; artifact timestamps are often UTC\n  while local is UTC-6/-7). Compute the delta at the call site if you need\n  it, with an explicit timezone conversion.\n- **`get_serve_config`'s exec-script fallback is best-effort shell-text\n  parsing**, not an exact argv array -- it handles `\\` line continuations\n  and basic quoting via `shlex.split`, but a sufficiently unusual exec\n  script could parse incorrectly. The live-process path (`/proc/<pid>/\n  cmdline`) does not have this limitation (it's the kernel's exact argv,\n  null-separated) and is always preferred when the unit is up.\n- **WSL2-specific by default, but the seam is explicit.** `get_service_status`\n  and `get_serve_config`'s live-process path assume this server runs on the\n  Windows host and shells into WSL2 via `wsl -d <distro>`. Set\n  `VLLM_OPS_MCP_RUNS_IN_WSL=1` if this server process itself runs inside\n  WSL2/Linux natively (native `systemctl`/`cat`, no `wsl -d` hop) -- both\n  paths are unit-tested (`TestGetServiceStatus::test_uses_wsl_prefix_on_windows`\n  / `test_no_wsl_prefix_when_native`), but only the Windows-host path has\n  been live-verified against this operator's actual setup.\n- **Single vLLM instance, single GPU, this operator's setup.** No\n  multi-instance/multi-GPU aggregation; `get_gpu_status` returns whatever\n  `nvidia-smi --query-gpu` enumerates, which is every GPU visible to the\n  process, not scoped to \"the GPU vLLM is using\" if more than one exists.\n- **No lifecycle control whatsoever in this release.** See \"Future\n  extension\" above -- this is intentional, not an oversight.\n\n## Env vars\n\n| Var | Effect | Default |\n|---|---|---|\n| `VLLM_OPS_MCP_BASE_URL` | OpenAI-compatible base URL | `http://127.0.0.1:8000/v1` (explicit `127.0.0.1`, not `localhost` -- see `qwen_cli.py`'s IPv6-hang footgun note) |\n| `VLLM_OPS_MCP_MODEL` | served-model-name, used in completion payloads | `qwen3-14b` |\n| `VLLM_OPS_MCP_WSL_DISTRO` | WSL distro hosting vLLM | `Ubuntu-22.04` |\n| `VLLM_OPS_MCP_SERVICE_UNIT` | systemd unit name | `vllm` |\n| `VLLM_OPS_MCP_EXEC_SCRIPT_PATH` | WSL-side path to the serve exec script (fallback source) | `~/vllm-systemd-exec.sh` |\n| `VLLM_OPS_MCP_SERVE_CONFIG_PATH` | local file mirror of the exec script; if set, `get_serve_config` reads it directly instead of shelling into WSL as a fallback source | unset |\n| `VLLM_OPS_MCP_NVIDIA_SMI_PATH` | `nvidia-smi` binary override | `nvidia-smi` (Windows host) / `nvidia-smi.exe` (native WSL2) |\n| `VLLM_OPS_MCP_RUNS_IN_WSL` | `1` if this server process itself runs inside WSL2/Linux (skips the `wsl -d` hop) | auto-detected via `sys.platform` |\n| `VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN` | cap on `check_health(deep=True)` + `test_completion` calls/min | `20` |\n| `VLLM_OPS_MCP_LIVE` | `1` to run real-infra smoke tests (see Testing) | unset (skip) |\n\n## Usage examples\n\n```jsonc\n// A tool call from the MCP host, illustrative -- not a shell command.\n{\"tool\": \"check_health\", \"arguments\": {\"deep\": true}}\n// -> {\"ok\": true, \"status\": \"up\", \"tier\": \"deep\", \"model_id\": \"qwen3-14b\",\n//     \"latency_ms\": 32.0, \"completion_s\": 0.218, \"error\": \"\"}\n\n{\"tool\": \"get_serve_config\", \"arguments\": {}}\n// -> {\"ok\": true, \"source\": \"live_process\", \"pid\": 5645,\n//     \"flags\": {\"model\": \"Qwen/Qwen3-14B-AWQ\", \"served_model_name\": \"qwen3-14b\",\n//               \"port\": \"8000\", \"max_model_len\": \"8192\",\n//               \"gpu_memory_utilization\": \"0.72\", \"max_num_seqs\": \"8\",\n//               \"enable_auto_tool_choice\": true, \"tool_call_parser\": \"qwen3_xml\", ...}}\n\n// rate limit exceeded (21st deep call within a minute):\n{\"tool\": \"test_completion\", \"arguments\": {\"prompt\": \"hi\"}}\n// -> {\"ok\": false, \"error\": {\"type\": \"rate_limited\", \"group\": \"deep_inference\",\n//     \"tool\": \"test_completion\", \"limit_per_min\": 20, \"retry_after_s\": 4.87}}\n```\n\n## Testing\n\nCI (`.github/workflows/ci.yml`) runs this suite on every push/PR (ubuntu-latest --\nthe unit suite mocks HTTP/subprocess, no live server/GPU/WSL required) and\nfails the build if the Tests badge above drifts from what the suite\nactually reports -- see `scripts/check_readme_counts.py`.\n\n```\n# unit suite (mocked HTTP/subprocess, no live server/GPU/WSL required)\n.venv\\Scripts\\python.exe -m pytest -q\n# -> 95 passed, 7 skipped\n\n# handshake check -- prints every registered tool name\n.venv\\Scripts\\python.exe scripts\\list_tools.py\n\n# real-infra smokes (real vLLM completion, real nvidia-smi, real systemctl\n# via WSL, real /proc/<pid>/cmdline read) -- read-only, safe to run anytime\n# vLLM is up\nVLLM_OPS_MCP_LIVE=1 .venv\\Scripts\\python.exe -m pytest -v -m live\n# -> 7 passed (all live, against the actual running qwen3-14b instance)\n```\n\n## Install\n\n```\npython -m venv .venv\n.venv\\Scripts\\python.exe -m pip install -r requirements.txt\n```\n\nRegistered in `~/.claude.json` as `vllm-ops-mcp` (stdio, own `.venv`, no env\noverrides needed for this operator's default setup).\n\n## Commercial support\n\nMaintained by [Jaimen Bell](https://jaimenbell.dev). For production MCP\nintegrations, custom servers, or agent-reliability work, see\n[jaimenbell.dev](https://jaimenbell.dev).\n\nBuilding your own MCP server? The [MCP Starter Kit](https://jaimenbell.gumroad.com/l/adnojp)\nhas templates, a build playbook, and packaging war-stories from shipping this one.\n\n<!-- MCP registry ownership marker -->\nmcp-name: io.github.jaimenbell/vllm-ops-mcp\n",
  "bytes": 16969,
  "sha": "07402cd80bea9a06a28bae98a7c0462eec627f5af6f25c216524ccf813f9bc0e",
  "repo_slug": "jaimenbell/vllm-ops-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_jaimenbell_vllm_ops_mcp_5fd762e1/readme"
}