{
  "markdown": "<p align=\"center\">\n  <img src=\"https://raw.githubusercontent.com/woladi/macos-vision-mcp/master/.github/assets/logo-128.png\" alt=\"macos-vision-mcp logo — a viewfinder locked onto an eye\" width=\"96\" height=\"96\">\n</p>\n\n# macos-vision-mcp\n\n<p align=\"center\">\n  <img src=\"https://raw.githubusercontent.com/woladi/macos-vision-mcp/master/.github/assets/hero.jpg\" alt=\"A Mac with documents and an app window inside a glowing boundary; only a small JSON card crosses it toward the cloud\" width=\"1200\">\n</p>\n<p align=\"center\">\n  <sub><b>Your documents and your screen stay on the machine.</b> Only extracted text and structured JSON ever reach the model.</sub>\n</p>\n\nLocal, private, offline OCR **and UI testing** for any MCP client — no API keys, no uploads.\nCut document token costs by ~97%, and let an agent see and click your Mac's UI without a single screenshot leaving the machine.\n\n[![npm version](https://img.shields.io/npm/v/macos-vision-mcp?style=flat-square&logo=npm&logoColor=white)](https://www.npmjs.com/package/macos-vision-mcp)\n[![npm downloads](https://img.shields.io/npm/dm/macos-vision-mcp?style=flat-square&logo=npm&logoColor=white)](https://www.npmjs.com/package/macos-vision-mcp)\n[![GitHub stars](https://img.shields.io/github/stars/woladi/macos-vision-mcp?style=flat-square&logo=github)](https://github.com/woladi/macos-vision-mcp/stargazers)\n[![License: MIT](https://img.shields.io/badge/License-MIT-ffd60a?style=flat-square)](LICENSE)\n[![macOS 13.0+](https://img.shields.io/badge/macOS-13.0%2B-0078d7?logo=apple&logoColor=white&style=flat-square)](https://developer.apple.com/documentation/vision)\n[![No API Key](https://img.shields.io/badge/no%20API%20key-required-brightgreen?style=flat-square)](#privacy-layer)\n[![Offline](https://img.shields.io/badge/offline-yes-blue?style=flat-square)](#what-you-get)\n[![Glama](https://glama.ai/mcp/servers/woladi/macos-vision-mcp/badges/score.svg)](https://glama.ai/mcp/servers/woladi/macos-vision-mcp)\n\nPre-extracts text and image data locally before your AI ever sees it — cutting token usage by ~97% on real documents and returning structured paragraphs, lines, and bounding boxes so the model can reconstruct the document into Markdown, HTML, DOCX, or any other format. Files never leave your Mac: no cloud API, no API keys, no network requests.\n\n> <sub>**How the ~97% is measured:** a 44-page scanned PDF sent as page images costs ~73,500 tokens; the same file run through `analyze_document` returns ~2,400 tokens of extracted text and structure (raw page-image tokens vs. extracted-text tokens). Your numbers vary with page density and tokenizer — treat 97% as the order of magnitude, not a guarantee.</sub>\n\n**Contents:** [Quick Start](#quick-start) · [What you get](#what-you-get) · [What agents use this for](#what-agents-use-this-for) · [UI testing](#ui-testing-without-sending-screenshots-anywhere) · [Why it's different](#why-its-different) · [Available Tools](#available-tools) · [Usage](#usage) · [Example workflows](#example-workflows) · [Configuration](#configuration) · [Privacy layer](#privacy-layer)\n\n## What you get\n\n- OCR for images and PDFs (JPG, PNG, HEIC, TIFF, multi-page PDF) via Apple Vision Framework.\n- ~97% token reduction: a 44-page PDF costs ~2,400 tokens instead of ~73,500.\n- Reading-order paragraphs + raw text blocks with bounding boxes — rich structure for the model to reconstruct the document into any output format (Markdown, HTML, DOCX, JSON), not a lossy plain-text dump.\n- Face detection, barcode/QR reading, and image classification — all on-device.\n- Full document pipeline: OCR + faces + barcodes + rectangles in a single tool call.\n- Works with Claude Code, Claude Desktop, and Cursor — any MCP-compatible client.\n- No files uploaded to any server — processing stays entirely on your Mac.\n- **UI testing for agents**: screenshot a window locally, find an element by its visible text, get back click coordinates, and assert what's on screen — all without uploading the screenshot.\n- 100% offline after `npm install` — powered by Apple Vision Framework, same engine as Live Text in Photos.app.\n\n## ❌ Without / ✅ With\n\n❌ **Without macos-vision-mcp:**\n\n- Sending a 44-page PDF costs ~73,500 tokens\n- Every image, invoice, or contract goes through a cloud API\n- Sensitive documents leave your machine on every request\n\n✅ **With macos-vision-mcp:**\n\n- Local Apple Vision pre-extracts text before Claude ever sees it\n- ~2,400 tokens for the same 44-page PDF — 97% fewer\n- Files never leave your Mac\n\n## What agents use this for\n\nMost work an agent does on a Mac needs no deep understanding of a layout. It needs to see what is\non screen, find the thing it is looking for, act on it, and confirm what happened. That loop —\n`list_windows` → `find_element` → click (via any input driver) → `assert_text` — covers a lot,\nand every step of it runs on the machine.\n\n- **Drive an app that has no API.** Native tools, Electron apps, internal software, anything with\n  a GUI and no scripting interface. `find_element(\"Export\")` returns the point to click.\n- **Read what is on screen right now.** A dialog, an error banner, a notification, a progress\n  state — including a window sitting behind others, without bringing it to the front.\n- **Confirm an action actually worked.** `assert_text` is string matching after unicode\n  normalisation, so it answers pass/fail the same way every time instead of asking a model to\n  judge a picture.\n- **Pull data out of software that will not export it.** OCR a window, get the text, move on.\n- **Audit accessibility.** `ui_snapshot` reports every piece of visible text the accessibility\n  tree does not account for — unlabelled controls and custom-drawn text, with coordinates.\n- **Test a UI for regressions.** The case this started as, and still a good one — see the chapter\n  below.\n- **Work through documents.** The original job: invoices, contracts, scans, PDFs.\n\n### Why doing it locally is better, not just different\n\n**Cheaper.** A verdict costs ~240 tokens against ~6,900 for the screenshot it replaces — about\n**29×**. Over a twenty-step task that is ~4,800 tokens instead of ~138,000. It is the difference\nbetween an agent that can afford to check its work after every step and one that cannot.\n\n**More private, and this is the part that never shows up on a bill.** A screenshot is not a neat\ncrop of the button you cared about. It carries whatever else was on screen: another window, a\npassword manager, an open inbox, a customer's record. Sending one to a third party is a\ndisclosure you cannot withdraw, and it repeats on every single step. Here the image is written to\na temp file, read by a model on the Neural Engine, and never serialised into the conversation.\nThat invariant is enforced in the code, not promised in this README: **no tool returns image\nbytes.**\n\n**Faster in practice, and predictable, which matters more.** `find_element` takes 1.17–1.25 s end\nto end on an M1 Pro — capture 0.31–0.41 s, OCR ~1.04 s, matching under a millisecond. There is no\nnetwork term at all: no ~750 KB upload before inference can start, no rate limit, no provider\nunder load, no failure when the Wi-Fi drops. The same call costs the same on a plane as it does\nat a desk.\n\n> **What this does not do: click.** It is eyes, not hands, and therefore never asks for control of\n> your machine. Pair it with an input driver — `cliclick`, a `macos-mcp`-style automation server,\n> or anything that accepts screen coordinates — and hand it the `clickPoint` that `find_element`\n> returns. The split is deliberate: seeing and acting are different permissions, and this server\n> only ever asks for the first.\n\n## UI testing without sending screenshots anywhere\n\nThe usual way to let an agent work with a GUI is to screenshot the screen and upload it to a\nvision model. That is one network round trip, one image-token bill, and one copy of whatever was\non screen — per step. A ten-step flow means ten uploads of your desktop.\n\nThis server does the seeing locally. Apple's Vision framework runs on the Neural Engine, so the\nscreenshot stays on disk and only text, geometry, and verdicts reach the model.\n\n<p align=\"center\">\n  <img src=\"https://raw.githubusercontent.com/woladi/macos-vision-mcp/master/.github/assets/ui-inspect.jpg\" alt=\"An app window with measurement rectangles snapped around each element, a crosshair on the Save button labelled x 812 y 540, and a card of extracted structured data\" width=\"1000\">\n</p>\n<p align=\"center\">\n  <sub><b>What the agent actually receives.</b> Element boxes, roles and labels — measured, not guessed — and the exact point to click. The screenshot itself never leaves the Mac.</sub>\n</p>\n\n```\nfind_element(query: \"Save\", app: \"MyApp\")\n  → { found: true, matches: [{ text: \"Save\", method: \"exact\",\n        clickPoint: { x: 812, y: 556 }, bbox: {...} }] }\n\n# hand clickPoint to any input driver — macos-mcp, cliclick, CGEvent\n# then verify, again locally:\n\nassert_text(expect: \"Saved\", app: \"MyApp\")  → { pass: true, ... }\n```\n\n`clickPoint` is in global screen points with a top-left origin — the same space click drivers\nuse, so it goes straight to a driver with no conversion. This server deliberately does not click:\nit is eyes, not hands, and therefore never asks for control of your machine.\n\n### Is it actually cheaper, safer, and faster?\n\nMeasured on an **Apple M1 Pro (2021, 16 GB)** against a 2992×1734 Retina window capture of a\nreal, text-dense app — median of five runs each.\n\n|                          | Local (this server)                                 | Screenshot → cloud vision API        |\n| ------------------------ | --------------------------------------------------- | ------------------------------------ |\n| **Tokens per step**      | ~240 (an `assert_text` verdict)                     | ~6,900 (image tokens for 2992×1734)  |\n| **Data leaving the Mac** | none                                                | ~750 KB PNG of your screen, per step |\n| **Network**              | none — works offline, on a plane, behind an air gap | one round trip per step              |\n| **Latency**              | 1.17–1.25 s end-to-end for `find_element`           | upload + inference + return          |\n| **Cost**                 | $0                                                  | per-image, per-step, forever         |\n\n> <sub>Image tokens are estimated with Anthropic's `width × height / 750` rule; other providers\n> tile differently, so the exact figure moves but the order of magnitude does not. Local token\n> counts are the actual JSON payloads the tools returned, at ~4 characters per token.</sub>\n\n<p align=\"center\">\n  <img src=\"https://raw.githubusercontent.com/woladi/macos-vision-mcp/master/.github/assets/token-cost.jpg\" alt=\"Side by side: uploading a screenshot to a cloud costs a tall stack of 6,900 tokens; the same check run locally costs a tiny stack of 240\" width=\"1000\">\n</p>\n<p align=\"center\">\n  <sub><b>Same question, same answer, ~29× the price.</b> Measured on an M1 Pro against a 2992×1734 window capture.</sub>\n</p>\n\nThe three claims behind that table — cheaper, safer, faster — are argued in\n[What agents use this for](#why-doing-it-locally-is-better-not-just-different) above. What this\nsection adds is the measurement: the numbers are a median of five runs on an Apple M1 Pro\n(2021, 16 GB) against a 2992×1734 Retina capture of a real, text-dense window.\n\nWe have not benchmarked any specific vision provider, so treat the right-hand column as structure\nrather than a measured figure. What can be stated is that the local path has no variance from\nbandwidth, rate limits or provider load, and does not fail when the network does.\n\nTwo honest caveats. Targeting a single region instead of a whole window cuts the OCR term\nsharply, since cost scales with pixels searched. And the first call after install spends ~2 s\ncompiling a small Swift helper; every call after that is warm.\n\n### What it is good at — and what it is not\n\nGood at: **native macOS apps, Electron apps with poor accessibility, canvas/WebGL UIs, games,\nand design mockups** — anything where there is no DOM to query. Also good when you want a\ndeterministic assertion rather than a model's opinion: `assert_text` is string matching after\nunicode normalisation, so it returns the same answer every time.\n\nNot the right tool for a plain web page: Playwright or the DOM will be faster and more precise\nthere. And OCR only sees what is rendered, so it cannot read a control's `enabled` state or its\naccessibility role.\n\nText matching is normalised before comparison — NFC, collapsed whitespace, unicode dashes and\nquotes folded — then tried exact → substring → fuzzy (Levenshtein). When a match is rejected it\nis still reported under `nearMisses`, so \"the label is there but OCR read _Zapisr_ for _Zapisz_\"\nis distinguishable from \"the label is genuinely absent\".\n\nSubstring hits are graded rather than treated alike: a query that stands on word boundaries,\nopens the label, and covers more of it scores higher. That ordering matters when the answer is\na button — for the query `Save`, `Save Changes` must outrank `Don't Save`. Each match reports\n`wholeWord`, and `assert_text` decides its verdict only on matches where it is true: `Save`\ninside `Unsaved changes` is a coincidence of spelling, and it neither proves a Save button is on\nscreen nor proves it is gone. Such hits are still listed, under `incidental`.\n\n### Requirements\n\n- **Screen Recording** permission for the app hosting the MCP server (Terminal, Claude Desktop,\n  Cursor): System Settings → Privacy & Security → Screen Recording, then restart that app.\n  No compiler or Xcode tooling is needed — the native helper arrives prebuilt.\n\n  The grant is per **host process**, not per package: the same server can be fully working under\n  one client and blind under another on the same Mac. Without it, capture fails outright and\n  macOS additionally withholds every window title, so `list_windows` reports `title: \"\"` for\n  everything — that case is flagged as `titlesAvailable: false` rather than left to look like a\n  screen full of untitled windows. Run `vision_capabilities` first: `ready` and `blockers` say\n  what works right now.\n\n- An **unlocked** Mac. On a locked machine window and region capture fail outright and a\n  full-screen capture returns only the lock screen; `vision_capabilities` reports `screenLocked`\n  so an agent can check before it starts rather than guessing at a failure afterwards.\n\n## Why it's different\n\nMost OCR options for LLMs either ship your documents to a cloud vision API or make you stand up and tune your own engine. This runs on Apple's on-device Vision framework — the same engine behind Live Text in Photos.app — so extraction is free, private, and instant.\n\n|                | macos-vision-mcp                                            | Cloud vision OCR (GPT-4o, Google Vision, Mistral OCR) | Tesseract-based MCP               |\n| -------------- | ----------------------------------------------------------- | ----------------------------------------------------- | --------------------------------- |\n| **Cost**       | $0 — no per-page or per-token fees                          | Per-call / per-page billing                           | $0, but self-hosted               |\n| **Offline**    | Yes, after install                                          | No — every page hits the network                      | Yes                               |\n| **Privacy**    | Files never leave your Mac                                  | Documents uploaded to a third party                   | Local                             |\n| **Setup**      | One command, no keys                                        | API key + billing account                             | Install + language data + tuning  |\n| **Quality**    | Apple Vision (strong on clean scans, receipts, screenshots) | Generally high                                        | Varies; weaker on poor scans      |\n| **UI testing** | Built in — capture, locate, assert, no uploads              | Possible, but every step uploads your screen          | OCR only; no capture or targeting |\n\nThe trade-off is honest: it's macOS-only, and on heavily skewed or low-contrast scans a cloud model may still read more. For the common case — invoices, contracts, receipts, screenshots, clean PDFs — you get cloud-grade extraction with zero cost, zero setup, and nothing leaving your machine.\n\n## Privacy layer\n\nmacos-vision-mcp acts as a local pre-processing layer between your documents and the cloud. Useful for:\n\n- Legal documents, contracts, NDAs\n- Financial reports, invoices, internal spreadsheets\n- Medical records or any GDPR-sensitive content\n- Any situation where you want to extract structured data locally before deciding what (if anything) to send upstream\n\nInstead of sending the raw document to your AI, you extract the text and structure locally first. The model then works only with the extracted text — never the original file.\n\nThe same applies to your screen. A screenshot taken for one small check still carries everything\nelse that happened to be visible — other windows, a password manager, a customer record, an open\ninbox. The UI-testing tools keep that image on disk and return only paths, geometry, and text, so\na UI assertion does not become an unplanned disclosure. No tool in this server returns image\nbytes to the model.\n\n## Quick Start\n\n**Add to your MCP client (example for Claude Code):**\n\n```sh\nclaude mcp add macos-vision-mcp -- npx -y macos-vision-mcp\n```\n\nUsing **Claude Desktop** or **Cursor**? [Jump to Configuration ↓](#configuration)\n\nRestart your client. `npx` fetches the package on first run, caches it, and the tools appear automatically — no separate install step. This is the convention used by most MCP servers and recommended by Anthropic, Cursor, and other clients.\n\n> **Note:** On first run, the package downloads prebuilt Swift helper binaries (`vision-helper`, `pdf-helper`, `ui-helper`, `ax-helper`) from its GitHub Releases (~276 KB compressed, ~1–2s). Subsequent invocations hit the npx cache and start instantly. Xcode Command Line Tools are only required as a fallback when the download can't reach the network — set `MACOS_VISION_SKIP_DOWNLOAD=1` to force local compilation with `swiftc`.\n\n> **Prefer instant cold-starts (no npx cache lookup)?** Install globally with `npm install -g macos-vision-mcp` and use the alternative config shown at the bottom of [Configuration](#configuration).\n\n## Available Tools\n\n| Tool               | What it does                                                                                                                                                                                                                                                   | Example prompt                                       |\n| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |\n| `ocr_image`        | Extract text from an image or PDF (JPG, PNG, HEIC, TIFF, PDF). Returns plain text, or per-page paragraphs + text blocks with `lineId` / `paragraphId` and bounding boxes. Accepts `start_page` / `max_pages` for partial PDF OCR.                              | \"Read the text from ~/Desktop/screenshot.png\"        |\n| `detect_faces`     | Detect human faces and return their count and positions.                                                                                                                                                                                                       | \"How many people are in this photo?\"                 |\n| `detect_barcodes`  | Read QR codes, EAN, UPC, Code128, PDF417, Aztec, and other 1D/2D codes.                                                                                                                                                                                        | \"What does the QR code in /tmp/qr.jpg say?\"          |\n| `detect_document`  | Detect the four corner points of a document in a photo (paper, receipt, ID). Useful as a crop / deskew hint before OCR.                                                                                                                                        | \"Find the document corners in ~/Desktop/receipt.jpg\" |\n| `classify_image`   | Classify image content into 1000+ categories with confidence scores.                                                                                                                                                                                           | \"What is in this image?\"                             |\n| `analyze_document` | Returns structured JSON with reading-order paragraphs, raw text blocks (bbox / confidence), faces, barcodes, and rectangles — ready for the model to reconstruct into Markdown, HTML, or anything else. Also accepts `start_page` / `max_pages` for long PDFs. | \"Reconstruct ~/Desktop/scan.pdf as clean Markdown\"   |\n\n### UI-testing tools (local, no screenshots sent to the cloud)\n\nThese tools let an agent **see and verify your Mac's UI without ever sending a screenshot to a\ncloud model**. Screenshots are captured locally, OCR runs on-device, and only paths, geometry,\nand extracted text are returned. `find_element` gives click coordinates in screen points, ready\nto hand to any input driver (this server deliberately does not click — eyes, not hands).\n\n| Tool                  | What it does                                                                                                                                                                                        | Example prompt                                        |\n| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |\n| `capture_screen`      | Screenshot the main display, a window (even occluded), an app's frontmost window, or a region. Returns the file path + screen-point frame — never the image bytes.                                  | \"Capture the Safari window\"                           |\n| `list_windows`        | List on-screen windows with global screen-point bounds, front-to-back.                                                                                                                              | \"What windows are open?\"                              |\n| `read_screen_text`    | Capture + OCR in one step — read what an app shows right now, fully offline.                                                                                                                        | \"What does the TestFlight window say?\"                |\n| `find_element`        | Find a UI element by visible text; returns `clickPoint {x,y}` in global screen points (exact → substring → fuzzy matching with near-miss reporting).                                                | \"Where is the Save button in MyApp?\"                  |\n| `assert_text`         | Local pass/fail assertion that text is present on / absent from the screen — the verdict is computed on your Mac, not by a cloud model.                                                             | \"Verify the dialog says 'Saved' after clicking Save\"  |\n| `vision_capabilities` | Report macOS version, Screen Recording / Accessibility permission state, and displays.                                                                                                              | \"Can this machine run UI tests?\"                      |\n| `ui_snapshot`         | Return the whole layout as JSON: every element's exact box, role, label and state from the accessibility tree, optionally with colours and fonts — plus visible text the tree does not account for. | \"Review this dialog's layout\" · \"What is unlabelled?\" |\n\n> Requires **Screen Recording** permission for the app hosting the MCP server (Terminal / Claude\n> Desktop / Cursor): System Settings → Privacy & Security → Screen Recording, then restart that\n> app. Nothing else to install — the native helper ships prebuilt with `macos-vision`.\n\n#### `ui_snapshot` — the layout, not just the text\n\n`find_element` answers \"where is X\". `ui_snapshot` answers \"what is on this screen\": every\nelement's **measured** box (from the accessibility API, not inferred from OCR), its role, label\nand enabled state, the parent/child structure, and optionally colours sampled from the capture\nand real font data.\n\n```jsonc\n{\n  \"app\": \"MyApp\",\n  \"window\": [0, 29, 1496, 867],\n  \"source\": \"ax+px\",\n  \"budget\": { \"elements\": 289, \"walked\": 400, \"capped\": false, \"elapsedMs\": 136 },\n  \"nodes\": [\n    {\n      \"id\": 42,\n      \"parent\": 7,\n      \"role\": \"Button\",\n      \"label\": \"Zapisz\",\n      \"box\": [812, 540, 96, 32],\n      \"style\": { \"bg\": \"#2F6FEB\", \"border\": \"#1B4FC4\", \"borderWidth\": 1 },\n      \"text\": { \"font\": \"SFPro-Semibold\", \"size\": 13, \"align\": \"center\" },\n    },\n  ],\n  \"unresolved\": [{ \"text\": \"Sprzedaż Q4\", \"box\": [420, 300, 88, 16], \"coveredByNode\": 17 }],\n  \"summary\": {\n    \"nodes\": 289,\n    \"labelled\": 240,\n    \"ocrBlocks\": 123,\n    \"unresolved\": 21,\n    \"axTextCoverage\": 0.83,\n  },\n}\n```\n\n`unresolved` is text Vision can read that no accessibility node accounts for. It completes the\npicture where AX is blind — canvas, WebGL, games, text baked into images — and each entry is an\naccessibility gap in the app: `coveredByNode` present means a control is there but unlabelled,\nabsent means nothing is exposed at all.\n\nRead it honestly: `budget.capped` means the tree is **incomplete**, and `summary.axTextCoverage`\nis `null` in that case on purpose — a capped walk measures how much was visited, not how\naccessible the app is. Colours come from pixels, so an occluded element reports whatever is drawn\non top; `borderWidth` is inferred and there is no padding or margin. This is not the CSS box\nmodel.\n\nNeeds **Accessibility** permission in addition to Screen Recording, and an unlocked Mac — a\nlocked screen exposes no accessibility windows at all.\n\n## Usage\n\nUse the tool name explicitly in your prompt to guarantee local processing:\n\n**Extract text from an image or PDF:**\n\n```\nUse ocr_image to extract text from ~/Desktop/invoice.pdf\n```\n\n**Detect faces in a photo:**\n\n```\nUse detect_faces on ~/Photos/team.jpg and tell me how many people are in it\n```\n\n**Classify image content:**\n\n```\nUse classify_image on ~/Downloads/unknown.jpg\n```\n\n**Full document analysis + reconstruction:**\n\n```\nUse analyze_document on ~/Desktop/report.pdf and reconstruct it as clean Markdown\n```\n\nThe tool returns structured JSON; the model picks the output format you ask for (Markdown, HTML, DOCX outline, etc.) without any extra dependencies — no Ollama, no cloud LLM, no extra tooling.\n\n## Example workflows\n\nReal-world combinations that work out of the box once the server is connected:\n\n- **\"Convert PDF → clean Markdown for LLM\"** — `analyze_document` returns reading-order paragraphs and bounding boxes; the model renders Markdown ready to drop into a docs site, knowledge base, or RAG pipeline.\n- **\"Extract invoice data locally before sending to GPT\"** — pull line items, totals, vendor, and dates from the PDF locally with `analyze_document`, then send only the structured JSON upstream. The original document never leaves your Mac.\n- **\"Scan receipts → JSON → expense tracker\"** — `ocr_image` on a phone photo, the model normalizes amount / date / merchant, and pipes the result straight into your expense tool's API.\n- **\"Decode a QR code from a screenshot\"** — `detect_barcodes` returns the decoded value plus symbology in one round trip.\n- **\"Crop a photo of a paper form before OCR\"** — `detect_document` returns the four corner points so you (or a downstream tool) can deskew and crop the image before reading the text.\n- **\"Click the Save button in my app\"** — `find_element` returns `clickPoint` in screen points; hand it to a click driver (macos-mcp, cliclick). The screenshot never leaves the Mac.\n- **\"Check my app still renders correctly after this change\"** — `assert_text` gives a deterministic pass/fail on what is on screen, at ~240 tokens per check instead of ~6,900 for the screenshot.\n- **\"Read the error message in that background window\"** — `read_screen_text` captures a specific window, even one hidden behind others, and returns just the text.\n- **\"What is my app showing right now?\"** — `list_windows` to pick the target, `read_screen_text` to read it, without bringing the window to the front.\n\n### Output schema (analyze_document)\n\n```jsonc\n{\n  \"source\": { \"path\": \"...\", \"pageCount\": 1, \"isPdf\": false },\n  \"pages\": [\n    {\n      \"page\": 0,\n      // primary surface for reconstruction — reading-order paragraphs joined with \"\\n\"\n      \"paragraphs\": [\n        { \"paragraphId\": 0, \"lineIds\": [0], \"text\": \"ACME COFFEE\" },\n        { \"paragraphId\": 1, \"lineIds\": [1, 2], \"text\": \"12 Main St\\nPortland, OR\" },\n      ],\n      // spatial fallback — raw blocks with page-local 0–1 bbox, confidence, line/paragraph membership\n      \"textBlocks\": [\n        {\n          \"text\": \"ACME COFFEE\",\n          \"lineId\": 0,\n          \"paragraphId\": 0,\n          \"confidence\": 0.99,\n          \"bbox\": { \"x\": 0.21, \"y\": 0.04, \"width\": 0.58, \"height\": 0.06 },\n        },\n      ],\n      \"faces\": [],\n      \"barcodes\": [],\n      \"rectangles\": [],\n    },\n  ],\n  \"summary\": {\n    \"totalTextBlocks\": 8,\n    \"totalParagraphs\": 2,\n    \"totalFaces\": 0,\n    \"totalBarcodes\": 0,\n    \"totalRectangles\": 0,\n  },\n}\n```\n\nUse `paragraphs[].text` for the 95% case (rebuild Markdown/HTML/plain text directly). Reach for `textBlocks[]` when you need spatial context — multi-column layouts, tables, forms, IDs.\n\n**Notes:**\n\n- `ocr_image` in `blocks` mode returns the same per-page shape minus the detection sections: `{ pages: [{ page, paragraphs, textBlocks }] }`.\n- PDFs are processed page by page. All coordinates are page-local (0–1), and `paragraphId` / `lineId` reset on every page.\n- Face, barcode, and rectangle detection on PDFs is best-effort — the underlying binary analyzes the file as a whole rather than per page, so any detections returned are attached to page 0 only.\n- Paragraph grouping uses spatial heuristics. For multi-column layouts (magazine spreads, wiki pages with side panels) the heuristic can collapse the whole page into a single paragraph. When that happens, fall back to `textBlocks[]` and reconstruct from the bounding boxes.\n\n## Configuration\n\nAll examples below use `npx -y` — the recommended default. No prior `npm install` needed; the package is fetched and cached on first run, and updates pick up automatically when the npx cache rolls over.\n\n### Claude Code\n\n```sh\nclaude mcp add macos-vision-mcp -- npx -y macos-vision-mcp\n```\n\n### Claude Desktop\n\nEdit `~/Library/Application Support/Claude/claude_desktop_config.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"macos-vision-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"macos-vision-mcp\"]\n    }\n  }\n}\n```\n\n### Cursor\n\nAdd to `~/.cursor/mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"macos-vision-mcp\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"macos-vision-mcp\"]\n    }\n  }\n}\n```\n\n### Alternative: global install\n\nIf you'd rather skip the npx cache lookup on cold starts — or you want to pin a specific version — install once:\n\n```sh\nnpm install -g macos-vision-mcp\n```\n\n…then use `\"command\": \"macos-vision-mcp\"` (no `args`) in any of the JSON configs above, or `claude mcp add macos-vision-mcp -- macos-vision-mcp` for Claude Code. Note that global installs can break when switching Node versions with nvm / asdf / volta — re-run `npm install -g` after switching.\n\n## Support\n\nIf macos-vision-mcp saved you tokens or kept a document on your Mac, consider [starring the repo](https://github.com/woladi/macos-vision-mcp) — it helps others find it.\n\n## Contributing\n\nContributions are welcome. Please follow [Conventional Commits](https://www.conventionalcommits.org/) for commit messages.\n\nReleases run on [changesets](https://github.com/changesets/changesets). If your change is user-visible, add a changeset to the PR:\n\n```sh\nnpm run changeset   # pick patch / minor / major, describe the change\n```\n\nMerging to `master` then opens a \"version packages\" PR that bumps the version, `server.json` and the changelog; merging _that_ PR publishes to npm (Trusted Publishing, with provenance), tags the release, and refreshes the MCP registry entry.\n\n```sh\ngit clone <repo>\ncd macos-vision-mcp\nnpm install\nnpm run dev   # watch mode\n```\n\n## License\n\nMIT — Adrian Wolczuk\n",
  "bytes": 32808,
  "sha": "149ccde9906bc49374075d1c9a9b4bb9a65be7639cea7787915bb08d08b93f11",
  "repo_slug": "woladi/macos-vision-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_woladi_macos_vision_mcp_e1bb523c/readme"
}