{
  "markdown": "<!-- mcp-name: io.github.kanishka089/realhands -->\n\n# computer-use-mcp (\"realhands\")\n\nAn MCP server that lets **Claude operate your real computer** the way a human does —\nmoving the **actual mouse**, clicking, typing, and reading the **actual screen**.\n\nUnlike OpenAI Operator, browser-use, or Playwright agents (which spin up a separate,\nisolated, logged-out Chrome), this drives the **physical OS cursor and keyboard**. So it\nworks in **your own Chrome with your own logged-in sessions** — and in every other app —\nbecause it's just a human at the keyboard, as far as any website can tell.\n\n**Status: LIVE and battle-tested.** Registered with Claude Code as the user-scope MCP\n**`realhands`** (tool `mcp__realhands__computer`) and ✓Connected since 2026-06-02.\nOn 2026-06-09 it drove the user's real, logged-in Chrome through a **complete Google\nPlay Console deployment** (app upload, release notes, submission) end-to-end.\n\n> The server is registered as `realhands` rather than `computer-use` because the name\n> \"computer-use\" is reserved in Claude Code.\n\n## How it works\n\nClaude (Desktop or Code) is the agent loop. You type a task; Claude calls the single\n`computer` tool in a see → think → act cycle:\n\n> **See** — `screenshot` returns the real screen (downscaled to ~1280px for grounding accuracy)\n> → **Think** — Claude picks the next action + pixel coordinates\n> → **Act** — the server moves the real mouse / types on the real keyboard\n> → a fresh screenshot comes back automatically after every action, and it repeats.\n\nTwo Windows-specific details make clicks land accurately (`src/screen.py`):\n\n- **DPI awareness** — `SetProcessDpiAwareness(2)` is set at import time so screenshot\n  pixels == pyautogui cursor coordinates even under display scaling (125% / 150% / …).\n- **Stateless coordinate scaling** — screenshots are downscaled (LANCZOS) to at most\n  `COMPUTER_USE_MAX_DIM` on the longest side before sending; incoming click coordinates\n  are scaled back up to real pixels. The scale factor is a pure function of monitor\n  geometry + `MAX_DIM`, so mapping never depends on which screenshot ran last.\n  Coordinates are clamped inside the target monitor so a stray click can't fly off-screen.\n  Each axis is rounded down to a whole 28px vision patch, so the two axes can scale very\n  slightly differently (<2.2%); `to_real()` maps each with its own factor and stays exact.\n\n**Multi-monitor:** every call takes an optional `monitor` index (1 = primary, 2.. =\nothers, 0 = the whole virtual desktop). `action=\"monitors\"` enumerates the setup.\nOrigins may be negative for screens left/above the primary — `to_real()` handles the\noffset. Use the **same** monitor for a click as for the screenshot you're clicking on.\n\n## Architecture\n\n```\nsrc/realhands/\n  server.py   FastMCP server \"computer-use\"; the single `computer` tool (action enum\n              modeled on Anthropic's reference computer_20250124 tool); runs one\n              action or a batch of `steps`, then returns status text + at most one\n              screenshot\n  screen.py   DPI awareness, mss capture, patch-aligned downscale, model-space ->\n              real-pixel mapping, unchanged-screen detection\n  input.py    pyautogui mouse/keyboard execution; xdotool-style key-name translation\n              (Return, Page_Down, ctrl+a, super, ...); clipboard-paste fast path for\n              long/Unicode/multiline typing (preserves your existing clipboard);\n              activate_window via win32 AttachThreadInput\n  safety.py   kill switches + lazy arm / stand-down lifecycle\n  config.py   .env-driven configuration (all defaults are sensible; .env is optional)\ninstall.py    one-shot installer: venv, deps, .env, Claude Desktop registration\n```\n\nStack: Python 3.10/3.11 · `mcp` (FastMCP, stdio) · `pyautogui` · `mss` · `pillow` ·\n`pynput` · `keyboard` · `pyperclip` · `python-dotenv` — plus `pygetwindow` and `pywin32`\nfor `activate_window`.\n\n## The `computer` tool\n\nA single tool with an `action` parameter:\n\n| Action | What it does |\n|---|---|\n| `screenshot` | Capture the screen (always start a task with this) |\n| `cursor_position` | Report the real mouse position |\n| `monitors` | List detected monitors (for multi-screen setups) |\n| `mouse_move` | Glide the cursor to `coordinate` |\n| `left_click` / `right_click` / `middle_click` / `double_click` / `triple_click` | Click at `coordinate` (or current position) |\n| `left_click_drag` | Drag from `text=\"x1,y1\"` to `coordinate=[x2,y2]` |\n| `left_mouse_down` / `left_mouse_up` | Press / release the left button |\n| `scroll` | Scroll at `coordinate` (`scroll_direction` + `scroll_amount` notches) |\n| `type` | Type `text` (clipboard-paste path for long/Unicode/multiline) |\n| `key` | Press a key or chord — `\"Return\"`, `\"ctrl+s\"`, `\"alt+Tab\"` |\n| `hold_key` | Hold keys for `duration` seconds |\n| `activate_window` | Bring an app to the front by title substring (beats Windows' foreground-lock; far more reliable than clicking the taskbar) |\n| `wait` | Sleep `duration` seconds, then screenshot |\n| `stop` | Stand down: close the STOP overlay + release the panic hotkey (call as the final action) |\n\nCoordinates are in the pixel space of the most recent screenshot; its size is reported\nwith every capture. After every non-screenshot action the tool waits ~0.4s for the UI\nto settle and returns a fresh screenshot.\n\n### Token economy\n\nScreenshots dominate the cost of driving a desktop, and not just once: every image\nstays in the conversation and is re-sent as history on each later turn. Claude bills\nvision in 28×28 patches — `tokens = ⌈w/28⌉ × ⌈h/28⌉` — so this server does four things\nto keep the bill down.\n\n**Batch steps.** Pass `steps` (a list of action dicts) instead of one call per action\nand the whole run shares **one** screenshot at the end:\n\n```jsonc\n{\"steps\": [{\"action\": \"left_click\", \"coordinate\": [420, 300]},\n           {\"action\": \"type\",  \"text\": \"hello@example.com\"},\n           {\"action\": \"key\",   \"text\": \"Tab\"},\n           {\"action\": \"type\",  \"text\": \"secret\"},\n           {\"action\": \"key\",   \"text\": \"Return\"}]}\n```\n\nThat is 1 125 visual tokens instead of 5 625, and one round trip instead of five — the\nlarger saving, since each avoided turn also avoids re-sending the entire transcript.\nA failing step stops the run, reports which step failed, and still returns the screen.\nAdd `\"screenshot\": false` to skip the trailing image too.\n\n**Patch-aligned downscaling.** A dimension that isn't a multiple of 28 pays for a\npartial patch row/column carrying almost no pixels. From a 1920×1080 primary the\ndefault 1260×700 is exactly 45×25 patches = 1 125 tokens, versus 1 196 for 1280×720 —\n6% off for 1.5% fewer pixels.\n\n**Unchanged-screen suppression.** If under `CHANGE_THRESHOLD` of pixels moved since the\nlast image sent, the reply is a line of text instead of a screenshot. A real desktop\nnever produces two byte-identical frames (clock, caret, hover states), so this is a\nthreshold, not an equality check. After `MAX_SKIPS` suppressions in a row it force-sends\none, so the model can't fly blind if it lost the earlier image to context compaction.\n\n**Right-sized images.** `COMPUTER_USE_MAX_DIM` trades grounding accuracy against cost —\nfrom a 1920×1080 primary: 1792 → 2 304 tokens, 1260 → 1 125 (default), 1036 → 777,\n896 → 576. Keep the long edge ≤ 2576 px: an image returned inside a `tool_result` is\n*rejected* rather than downscaled when it exceeds the model's limit.\n\n> `COMPUTER_USE_IMAGE_FORMAT` is **not** a token lever — Claude bills by pixel\n> dimensions, so a JPEG and a PNG of the same screenshot cost exactly the same. JPEG\n> only cuts payload bytes (~977 KB → ~141 KB here), which helps latency at some risk\n> to small-text legibility.\n\n## Safety — it controls your REAL machine\n\nThis is **fully autonomous**: it does not ask before each action. Three independent\nkill switches (`src/safety.py`):\n\n1. **Fail-safe corner** — slam the mouse into the **top-left corner** → pyautogui raises\n   `FailSafeException` and the action aborts instantly.\n2. **Panic hotkey** — **Ctrl+Alt+Q** (configurable) → hard-kills the server process\n   (`os._exit(1)`).\n3. **STOP overlay** — an always-on-top window (top-right) showing the current action,\n   with a big red **■ STOP AGENT** button that also hard-kills the process.\n\n**Lazy arm / stand-down:** the overlay and the global panic hotkey are armed lazily on\nthe **first action** of a task, not at server startup — idle sessions show nothing and\ngrab no hotkeys. They stand down when the agent calls `action=\"stop\"` at the end of a\ntask, and re-arm automatically on the next action. (The STOP overlay is a single\npersistent window that is *hidden* when dormant, never destroyed — recreating it was a\ncrash hazard.) An optional idle auto-stand-down is available via\n`COMPUTER_USE_IDLE_STOP` but is **disabled by default**: an agent's thinking time\nbetween tool calls easily exceeds any short idle window, so a non-zero value would stand\nthe agent down mid-task.\n\nPacing also helps you stay in control: every action is followed by a configurable pause\n(`COMPUTER_USE_PAUSE`) and the cursor glides rather than teleports\n(`COMPUTER_USE_MOVE_DURATION`), so you can watch and interrupt.\n\n**Don't leave it unsupervised on anything that can spend money, send messages, or\ndelete data.**\n\n## Install\n\nRequires **Python 3.10 or 3.11** (3.13+ untested; avoid the 3.14 beta).\n\n### Recommended: `uvx` (always the latest version)\n\nNothing to install up front, and **you get every release automatically** — `uvx`\nresolves the newest published version each time the server starts, so a restart of your\nMCP client is the whole upgrade process. Requires [uv](https://docs.astral.sh/uv/).\n\n```powershell\nuvx realhands@latest\n```\n\nDrop the `@latest` (`uvx realhands`) if you would rather let uv reuse whatever version\nit already has cached.\n\n### From PyPI (pinned)\n\n```powershell\npip install realhands\n```\n\nThis installs the `realhands` console script and the importable `realhands`\npackage. Run the server with either `realhands` or `python -m realhands.server`.\n\nPick this over `uvx` when you want a **pinned** version that never changes underneath\nyou, or when the machine may be offline at startup — `uvx realhands@latest` needs to\nreach PyPI each time it launches. The trade-off is that upgrades become manual:\n\n```powershell\npip install --upgrade realhands\n```\n\nThen restart your MCP client — clients bind their servers at session start, so a\nrunning session keeps the old code until it restarts.\n\n### From source (with Claude Desktop registration)\n\n```powershell\ngit clone https://github.com/kanishka089/computer-use-mcp\ncd computer-use-mcp\npy -3.10 install.py\n```\n\nThis creates `.venv/`, installs the package + deps (editable), copies `.env.example` to\n`.env` if missing, and registers the server in Claude Desktop's config (backing up any\nexisting config). **Restart Claude Desktop**, then look for the `computer-use` tool.\n\n### Claude Code\n\nRegister it as a **user-scope** stdio server named `realhands`. This form auto-upgrades\n— each new session resolves the latest release, so you never have to think about it:\n\n```powershell\nclaude mcp add realhands --scope user -- uvx realhands@latest\n```\n\nThe tool then appears as `mcp__realhands__computer` in every project.\n\nIf you installed with `pip` instead and want a pinned version, point it at the Python\nthat has the package (and run `pip install --upgrade realhands` yourself to move up):\n\n```powershell\nclaude mcp add realhands --scope user -- python -m realhands.server\n```\n\n> **Already registered on an older version?** Re-point it at the auto-upgrading form:\n> ```powershell\n> claude mcp remove realhands --scope user\n> claude mcp add realhands --scope user -- uvx realhands@latest\n> ```\n> Removing and re-adding an MCP mid-session drops this session's connection to it — the\n> new registration is picked up on the next Claude Code start.\n\n## Upgrading\n\n> **On 0.1.x?** Upgrade. `mcp` 2.0 removed the API 0.1.x imports, so while an install\n> made back when it resolved `mcp` 1.x still runs, reinstalling or moving machines will\n> produce a server that cannot start. 0.2.1 and later handle both. See\n> [v0.2.1](https://github.com/kanishka089/computer-use-mcp/releases/tag/v0.2.1).\n\n### Best: switch to the auto-upgrading form (once)\n\nThen you never do this again — every client restart picks up the newest release:\n\n```powershell\nclaude mcp remove realhands --scope user\nclaude mcp add realhands --scope user -- uvx realhands@latest\n```\n\nRestart Claude Code. (Removing and re-adding an MCP mid-session drops this session's\nconnection to it; the new registration takes effect on the next start.)\n\n### Staying on pip\n\n**Upgrade the same interpreter the server actually runs** — not whatever `pip` happens\nto resolve. If you installed into a virtualenv, or have several Pythons, a bare\n`pip install --upgrade realhands` will cheerfully upgrade a different environment and\nleave the server on its old version. This is the most common reason an upgrade\n\"doesn't take\".\n\nFirst, find the interpreter:\n\n```powershell\nclaude mcp get realhands\n```\n\nRead the `Command:` line — that is your Python. Then upgrade with it explicitly:\n\n```powershell\n& \"C:\\path\\from\\that\\Command line\\python.exe\" -m pip install --upgrade realhands\n```\n\nIf `Command:` is a bare `python`, `pip install --upgrade realhands` is fine.\n\nConfirm it took, then restart Claude Code:\n\n```powershell\n& \"C:\\path\\to\\python.exe\" -c \"import realhands; print(realhands.__version__)\"\n```\n\n### Installed from source?\n\n`git pull` is enough — `install.py` installs the package as editable, so the checkout\n*is* the installed version. Restart your client.\n\n## Use\n\nJust ask. For example:\n\n> *Take a screenshot, open Chrome, go to YouTube, and search for \"lofi\".*\n\nWatch your real cursor move and your logged-in Chrome respond. Real-world proof: it has\nautonomously completed a full Google Play Console release flow in the user's own\nsigned-in Chrome session.\n\n## Configuration (`.env`, optional — defaults are fine)\n\n| Var | Default | Meaning |\n|-----|---------|---------|\n| `COMPUTER_USE_MAX_DIM` | `1260` | Longest screenshot side sent to Claude; the main accuracy-vs-cost dial (see [Token economy](#token-economy)) |\n| `COMPUTER_USE_PATCH_ALIGN` | `1` | Round each axis down to a whole 28px patch so no tokens go on a partial patch (~6% off every shot) |\n| `COMPUTER_USE_CHANGE_THRESHOLD` | `0.002` | Skip the screenshot when less than this fraction of pixels moved (`0` = always send) |\n| `COMPUTER_USE_MAX_SKIPS` | `6` | Force a real screenshot after this many suppressed in a row |\n| `COMPUTER_USE_MONITOR` | `1` | Default monitor (1 = primary, 2.. = others, 0 = all screens); overridable per call |\n| `COMPUTER_USE_IMAGE_FORMAT` | `png` | `png` (crisp text) or `jpeg` (smaller payload). **Not** a token lever — cost is by pixel dimensions, not bytes |\n| `COMPUTER_USE_PAUSE` | `0.15` | Delay after each pyautogui action (interruptibility) |\n| `COMPUTER_USE_PANIC_HOTKEY` | `ctrl+alt+q` | Global hard-stop hotkey |\n| `COMPUTER_USE_OVERLAY` | `1` | Show the STOP overlay window |\n| `COMPUTER_USE_MOVE_DURATION` | `0.4` | Cursor glide time (human-like movement) |\n| `COMPUTER_USE_IDLE_STOP` | `0` | Auto stand-down after this many idle seconds (`0` = never; stand down only on `action=\"stop\"`) |\n\n## Known gotchas\n\n- **MCP connection drops when the agent idles between turns.** The stdio connection to\n  `realhands` can silently die while Claude is thinking/waiting between turns. Fix: issue\n  a `screenshot` action — it silently reconnects. Importantly, an action that \"failed\"\n  with *Connection closed* **often still executed** on the real machine — take a\n  screenshot and check the actual screen state before retrying, or you may double-click /\n  double-submit.\n- **Click coordinates must match the screenshot's monitor.** If you screenshot\n  `monitor=2` and then click without passing `monitor=2`, the click lands on the primary.\n- **`activate_window` beats the taskbar.** Windows' foreground-lock makes taskbar clicks\n  unreliable (the icon just flashes). `activate_window` uses `AttachThreadInput` +\n  z-order toggling + a minimize/restore fallback, so prefer it for app switching.\n- **Don't run with Python 3.13/3.14.** Tested on 3.10/3.11 only; the installer warns.\n- **Typing long/Unicode text uses the clipboard.** Your clipboard is saved and restored,\n  but anything watching the clipboard will see the pasted text momentarily.\n\n## Self-test\n\n```powershell\npython -m realhands.screen\n```\n\nCaptures the screen, prints real vs. sent dimensions and the scale factor, writes\n`test_capture.png`, and runs a coordinate round-trip check (center + both corners).\n",
  "bytes": 16667,
  "sha": "f5b5ee351871050b84a65785320c087886b9f42a9a9f3316205746de1e01e076",
  "repo_slug": "kanishka089/computer-use-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_kanishka089_realhands_e14fd6f2/readme"
}