ocular
Vision tools for coding agents: screenshots, OCR, UI diffs, errors, tables, and charts.
Open source Open in the app JSON README (API)
About
Vision tools for coding agents: screenshots, OCR, UI diffs, errors, tables, and charts.
Details
- Kind
- MCP servers
- Topic
- Files & documents
- Publisher
- xyun1996
- Origin
- official
- Category
- ferramentas
- Transport
- local
- Version
- 0.1.0
- Open pull requests
- 7
- Last push
- 2026-08-12T04:03:59Z
- Repository state
- ativo
- Language
- TypeScript
- License
- MIT
- Added
- 2026-08-29 04:01:42
- Updated
- 2026-08-29 04:01:42
- Origin id
io.github.xyun1996/ocular
README
# ocular
[](https://github.com/xyun1996/ocular/actions/workflows/ci.yml)
[](https://www.npmjs.com/package/ocular-mcp)
[](LICENSE)
[](https://nodejs.org/)
[](https://www.typescriptlang.org/)
[](https://modelcontextprotocol.io/)
**Vision for coding agents.**
`ocular` is an MCP server that lets text-first coding agents analyze screenshots, UI mockups, terminal errors, documents, tables, and charts through OpenAI-compatible vision models.
It is designed for both local stdio use and remote HTTP deployments. For remote agents, image bytes can travel through a binary upload side channel while MCP tool calls carry only a lightweight `file_id`, avoiding large inline base64 payloads.
> **Project status:** early-stage and actively evolving. Feedback, bug reports, integrations, and real-world usage reports are welcome.
## Why ocular?
Coding agents are good at reading source code but often lose context when the important evidence is visual: a broken layout, a terminal screenshot, an error dialog, a chart, or a design reference.
`ocular` turns those visual inputs into structured data an agent can reason about.
- **Agent-oriented output** — tools return structured JSON instead of prose-only descriptions.
- **8 focused vision tools** — general analysis, OCR, UI inspection, error diagnosis, UI comparison, table extraction, chart analysis, and upload orchestration.
- **OpenAI-compatible provider interface** — point ocular at a compatible multimodal endpoint and model; reproducibly tested configurations are tracked in [Provider compatibility](docs/provider-compatibility.md).
- **Remote-friendly uploads** — binary `PUT /upload` flow for large images with content-addressed `file_id` references.
- **Local or remote MCP** — stdio for local clients, HTTP for hosted/private deployments.
- **Caching and persistence** — deduplicated uploads plus result caching for repeated agent workflows.
## How it works
```mermaid
flowchart LR
A[Coding agent] -->|MCP tool call| B[ocular]
C[Image / screenshot] -->|binary upload or base64| B
B -->|OpenAI-compatible request| D[Vision model]
D -->|multimodal response| B
B -->|structured JSON| A
```
For remote HTTP deployments, the recommended path is:
```text
image bytes -> PUT /upload -> file_id -> MCP vision tool -> structured result
```
See [Architecture](docs/architecture.md) for the upload and caching model.
## Demo
Want to see the full handoff from screenshot to coding-agent evidence? Read the [end-to-end demo](docs/demo.md).
It walks through a remote image upload, a `diagnose_error_screenshot` call, the structured fields returned to the agent, and how that evidence is combined with repository context. Example model output is explicitly marked representative rather than presented as a benchmark.
## Quick start
### 1. Install
The published npm package is [`ocular-mcp`](https://www.npmjs.com/package/ocular-mcp). It installs the CLI command `ocular`.
Global install:
```bash
npm install -g ocular-mcp
```
Or run it without a global install:
```bash
npx -y ocular-mcp
```
To build from source instead:
```bash
git clone https://github.com/xyun1996/ocular.git
cd ocular
npm install
npm run build
```
### 2. Configure a vision provider
`ocular` requires an OpenAI-compatible multimodal endpoint, API key, and model name:
```env
OCULAR_BASE_URL=https://your-openai-compatible-endpoint.example/v1
OCULAR_API_KEY=your_api_key
OCULAR_MODEL=your_vision_model
```
For a local compatible endpoint, use that server's base URL and vision-capable model name. Compatibility depends on the endpoint/model combination; see [Provider compatibility](docs/provider-compatibility.md) for the reproducible smoke-test procedure and verified configurations.
### 3. Run in stdio mode
With a global install:
```bash
ocular
```
Or:
```bash
npx -y ocular-mcp
```
The server communicates over stdio, so it may appear idle when started directly. In normal use an MCP client launches it and exchanges protocol messages over stdin/stdout.
### 4. Connect an MCP client
Claude Code example:
```bash
claude mcp add ocular \
-e OCULAR_BASE_URL=https://your-openai-compatible-endpoint.example/v1 \
-e OCULAR_MODEL=your_vision_model \
-e OCULAR_API_KEY=your_api_key \
-- npx -y ocular-mcp
```
Avoid putting long-lived API keys directly in shell history on shared machines. Use your client's environment/secret-management mechanism when available.
For a generic MCP client:
```json
{
"mcpServers": {
"ocular": {
"command": "npx",
"args": ["-y", "ocular-mcp"],
"env": {
"OCULAR_BASE_URL": "https://your-openai-compatible-endpoint.example/v1",
"OCULAR_MODEL": "your_vision_model",
"OCULAR_API_KEY": "your_api_key"
}
}
}
}
```
See [Claude Code setup](examples/claude-code.md) for a fuller walkthrough.
## Example workflows
### Diagnose a screenshot
Ask your coding agent to inspect an error screenshot and extract the exact message, likely cause, and next checks.
```json
{
"file_id": "e21ba723...",
"task": "Extract the exact error and suggest the next debugging checks",
"project_context": "Node.js TypeScript project"
}
```
### Review a UI implementation
Use `analyze_ui_screenshot` to turn a screenshot into implementation-oriented observations about hierarchy, alignment, spacing, typography, contrast, and likely visual defects.
### Compare expected vs actual UI
Use `compare_ui_screenshots` with a reference screenshot and an implementation screenshot to identify regressions and layout differences.
See [Screenshot debugging example](examples/screenshot-debugging.md).
## Tools
| Tool | Purpose |
|---|---|
| `analyze_image` | General structured image analysis |
| `extract_text_from_image` | OCR with reading-order/layout awareness |
| `analyze_ui_screenshot` | UI hierarchy, spacing, typography and accessibility review |
| `diagnose_error_screenshot` | Extract and diagnose terminal/browser/build errors |
| `compare_ui_screenshots` | Compare reference and implementation screenshots |
| `extract_table_from_image` | Extract table data into structured output |
| `analyze_chart_image` | Analyze chart labels, values, trends and uncertainty |
| `create_upload_session` | Return upload endpoint and instructions for remote clients |
Every vision tool accepts `file_id`; local workflows can also use inline `image_base64` where appropriate.
## Remote deployment
Set HTTP transport and authentication:
```env
MCP_TRANSPORT=http
MCP_HTTP_HOST=127.0.0.1
MCP_HTTP_PORT=3000
MCP_HTTP_PATH=/mcp
MCP_AUTH_TOKEN=replace_with_a_long_random_token
MCP_AUTH_HEADER=authorization
MCP_AUTH_SCHEME=Bearer
```
Upload raw bytes:
```bash
curl --request PUT \
--data-binary @/path/to/image.png \
"https://your.host/upload" \
-H "Content-Type: image/png" \
-H "Authorization: Bearer your_mcp_auth_token"
```
The server returns a content-addressed `file_id`; pass that id to a vision tool instead of sending a large base64 string through MCP.
For reverse proxy and systemd examples, see [Deployment](docs/deployment.md).
## Configuration
Common variables:
| Variable | Purpose |
|---|---|
| `OCULAR_BASE_URL` | OpenAI-compatible API base URL |
| `OCULAR_API_KEY` | Provider API key |
| `OCULAR_MODEL` | Vision-capable model name |
| `OCULAR_HEADERS` | Optional custom provider headers as JSON |
| `OCULAR_TEMPERATURE` | Generation temperature |
| `OCULAR_MAX_TOKENS` | Maximum generated tokens |
| `OCULAR_TIMEOUT_MS` | Provider timeout |
| `OCULAR_MAX_IMAGE_MB` | Maximum image size |
| `OCULAR_CACHE_ENABLED` | Enable result cache |
| `OCULAR_CACHE_DIR` | Cache directory |
| `OCULAR_UPLOADS_DIR` | Persistent upload directory |
| `OCULAR_UPLOAD_URL_BASE` | Public base URL used in upload instructions |
See [.env.example](.env.example) for the full configuration surface.
## Verification and benchmarks
Provider compatibility claims are based on real endpoint/model smoke tests, not on API naming alone. See [Provider compatibility](docs/provider-compatibility.md).
The repository also includes synthetic, redistributable visual fixtures for repeatable project-level measurements. See [Benchmark fixtures](bench/README.md). The benchmark measures execution, structural JSON output, and timing; it is not presented as a broad model-quality ranking.
## Development
```bash
npm install
npm run build
npm test
npm run check
npm run dev
```
The repository includes tests for authentication, caching, image handling, MCP server behavior, provider payloads, tool execution, npm packaging, Registry metadata consistency, and release smoke checks.
## Security and privacy
Do not commit provider API keys or MCP authentication tokens. Public HTTP deployments should sit behind HTTPS and a reverse proxy; the Node process should generally bind to a private interface.
See [SECURITY.md](SECURITY.md) for vulnerability reporting guidance.
## Roadmap
Near-term areas where contributions are useful:
- Real-world MCP client integration and usage reports
- Provider/model compatibility verification
- Published fixture-based benchmark results from real endpoints
- Community-driven tool and prompt improvements
If you are using `ocular` in a real workflow, open a **Usage report** issue describing the client, provider/model, and use case. Public reports are useful even when nothing is broken and help keep compatibility/adoption claims grounded in real usage.
## Contributing
Contributions are welcome. Start with [CONTRIBUTING.md](CONTRIBUTING.md), run `npm run check` before opening a PR, and include reproduction details for behavior changes.
## Release and registry
The npm package is `ocular-mcp`; the installed CLI is `ocular`. Release automation uses npm Trusted Publishing rather than a long-lived repository token. See [Publishing](docs/publishing.md).
`ocular` is published in the official MCP Registry as [`io.github.xyun1996/ocular`](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.xyun1996%2Focular). See [MCP Registry](docs/mcp-registry.md) for the live identity and release flow.
## License
MIT — see [LICENSE](LICENSE).
If `ocular` is useful in your agent workflow, a GitHub star helps other developers discover the project.