io.github.sshwarts/napkin

Shared visual canvas (Excalidraw) for collaborative whiteboarding between AI agents and humans.

Open source Open in the app JSON README (API)

About

Shared visual canvas (Excalidraw) for collaborative whiteboarding between AI agents and humans.

Details

Kind

MCP servers

Topic

No topic detected

Publisher

sshwarts

Origin

official

Category

ferramentas

Transport

local

Version

0.1.0

Stars

1

Last push

2026-05-05T21:32:02Z

Repository state

ativo

Language

TypeScript

License

MIT

Added

2026-08-29 04:01:27

Updated

2026-08-29 04:01:27

Origin id

io.github.sshwarts/napkin

README

<p align="center">
  <img src="assets/NapkinLogo.png" alt="Napkin" width="300">
</p>

<p align="center">
Napkin is a shared Excalidraw canvas for agent + human collaboration.
</p>

## Why We Built It
I love using AI assistants for everyday tasks — design, programming, and brainstorming. I’ve always found visual thinking helpful, but there was a gap: I could describe something, and the agent could describe it back, but we couldn’t point to the same thing. What if my agent and I could share a whiteboard — where I draw, it responds, and we build together?

Napkin works with any MCP-capable agent. For true two-way canvas-to-agent, you'll also need a webhook.

## Examples (all drawn by Perry the agent, with Napkin)

Brainstorming

<p align="center">
  <img src="assets/brainstorm.png" alt="Brainstorm" width="300">
</p>

Gaming

<p align="center">
  <img src="assets/snake_game.png" alt="Games" width="300">
</p>

"Art" - Don't know why Perry invented a Platabird, but there you are.

<p align="center">
  <img src="assets/self_portrait.png" alt="Art" width="300">
</p>

Collaboration

<p align="center">
  <img src="assets/collab.png" alt="Collaboration" width="300">
</p>


## What It Is
Napkin is an [Excalidraw](https://excalidraw.com) whiteboard connected to an MCP server. Any MCP-capable agent can read the canvas, draw on it, annotate it, animate it, and respond to what you draw — all while you interact with the agent through your normal channel (Slack, WhatsApp, Telegram, Terminal, or any MCP client).

The conversation stays in your channel. The canvas is pure whiteboard.

<p align="center">
  <img src="assets/overview.png" alt="Overview" width="600">
</p>


## Quick Start

### 1) Start the MCP server

```bash
cd mcp
npm install
npm run build
npm start
```

Defaults:
- MCP HTTP: `http://localhost:3003`
- WebSocket: `ws://localhost:3002`

### 2) Start the browser UI

```bash
cd ui
npm install && npm run dev
# Open http://localhost:5173
```

### 3) Connect your agent

Napkin works with any MCP-capable agent. Add to your MCP client config:
```json
{
  "napkin": {
    "type": "http",
    "url": "http://localhost:3003"
  }
}
```

**Two-way vs one-way:** MCP alone gives you agent→canvas (read, write, animate). For canvas→agent (the agent waking up when *you* draw something), you need a webhook receiver on your agent's side.

| Mode | What you get | Requires |
|------|-------------|---------|
| MCP only | Agent can read and write the canvas | Any MCP client |
| MCP + webhook | Agent also reacts to human canvas activity | Webhook receiver (see below) |

### 4) Configure the webhook (two-way only)

When you draw on the canvas, Napkin POSTs a trigger to your agent's webhook endpoint. The agent wakes up, reads the canvas diff, and responds.

**OpenClaw** — built in. Set `NAPKIN_TRIGGER_WEBHOOK` in your environment and it routes automatically.

**NanoClaw** — add the webhook channel via [PR #1488](https://github.com/qwibitai/nanoclaw/pull/1488):

```bash
# .env additions
WEBHOOK_PORT=3200
WEBHOOK_LINKED_JID=<your channel JID>   # the chat where the agent should post
```

Then pass the webhook URL when starting your session:

```
start_session({
  session_id: "<your channel ID>",
  webhook_url: "http://localhost:3200/webhook"
})
```

**Custom / other frameworks** — any HTTP server that accepts `POST /webhook` with `{ message, sender? }` JSON and forwards to your agent works fine.


## Environment Variables

Common server variables:

- `NAPKIN_TRANSPORT` (`http` or `stdio`, default `http`)
- `NAPKIN_MCP_PORT` (default `3003`)
- `MCP_WS_PORT` (default `3002`)
- `AGENT_TRIGGER_DEBOUNCE_MS` (default `3000`)
- `NAPKIN_TRIGGER_WEBHOOK` (optional global webhook URL)
- `NAPKIN_COMPACT_TRIGGERS` (`true`/`false`, default `false`)
- `NAPKIN_TRIGGER_INCLUDE_CANVAS` (`true`/`false`, default `false`)
- `NAPKIN_SESSION_TTL_MS` (default `7200000`)
- `NAPKIN_EXPORT_DIR` (optional base dir for relative exports)
- `ANTHROPIC_API_KEY` (required for vision tools only - others may be used)

See `ARCHITECTURE.md` for full details.

## Core Concepts

### The Intent API

No coordinates. No boilerplate. Describe what you want.

```
add_node("Auth Service", shape: "rectangle", metadata: { intent: "entry point" })
add_node("Token Store")
connect("Auth Service", "Token Store", label: "issues token")
layout()
```

The server handles placement, bindings, and layout. A 3-node diagram takes one short exchange.

### Reading the Canvas

`get_canvas()` returns a semantic structure — nodes, edges, zones — not raw coordinates:

```json
{
  "nodes": [
    { "id": "abc", "label": "Auth Service", "type": "box",
      "metadata": { "intent": "entry point", "status": "wip" } }
  ],
  "edges": [
    { "id": "xyz", "from": "abc", "to": "def", "label": "issues token" }
  ]
}
```

For cheap reasoning passes, `get_canvas_summary()` returns nodes and edges only — no zones, sketches, or proximity properties.

### Metadata

Every element carries a `customData` object — invisible in the UI, readable by agents:

```
add_node("Deploy Production", metadata: {
  type: "task",
  step: 8,
  owner: "platform-team",
  status: "wip"
})
```

Agents use metadata to reason about *what* elements represent, not just what they look like. `trace_path()` can filter by metadata to find specific flows.

### Triggers and Webhooks

When you draw or annotate, Napkin notices. After a quiet period, it POSTs a trigger to your agent's webhook:

```json
{
  "session_id": "slack:D0AGREPG84R",
  "source": "debounce",
  "message": "[napkin] Canvas updated (idle)",
  "changed_element_ids": ["rect-42"],
  "changed_elements_compact": [
    { "id": "rect-42", "type": "rectangle", "label": "Change the timer to 120s?" }
  ],
  "change_summary": "added text \"Change the timer to 120s?\"",
  "change_type": "semantic"
}
```

The agent wakes up, reads what changed, and responds — on the canvas and in your channel.

**Trigger sources:**
- `debounce` — canvas went quiet (configurable per session, default 3s)
- `chat` — injected by your agent framework
- `reconnect` — browser connected or reconnected

**Echo suppression:** agent writes don't trigger webhooks. Only genuine human edits fire triggers.

### Thought Bubbles

Agents express tentative ideas as thought bubbles — dashed purple, clearly "agent suggestion":

```
add_thought_bubble("Should this connect to the cache layer?", near_node_id: "db-node")
```

You can confirm them (makes permanent) or dismiss them. Keeps agent suggestions visually distinct from established content.

### Batch Execution: `apply_intents`

Collapse many sequential calls into one:

```
apply_intents([
  { tool: "add_node", args: { label: "A" }, ref: "n1" },
  { tool: "add_node", args: { label: "B" }, ref: "n2" },
  { tool: "connect", args: { from_id: "$ref:n1.id", to_id: "$ref:n2.id" } },
  { tool: "layout", args: {} }
])
```

Ref substitution (`$ref:n1.id`) lets later ops use outputs from earlier ones — no round-trips needed to get IDs. A 10-node diagram with connections and layout ships in a single MCP call.

### Swimlanes: `zone` + `row`

Pin nodes to explicit layout rows independent of edge structure. Nodes sharing the same `zone` and `row` are snapped to the same rank band after Dagre runs — useful for shared infrastructure (databases, event buses) that should sit on a common row across subtrees:

```
add_node("API",         zone: "main", row: 0)
add_node("Worker",      zone: "main", row: 0)
add_node("Postgres",    zone: "main", row: 1)   # shared infra row
connect("API", "Postgres")
connect("Worker", "Postgres")
layout("TB")
```

Row index is the rank axis (top-to-bottom in `TB`, left-to-right in `LR`). Unzoned nodes fall back to natural Dagre placement.

### Path Traversal: `trace_path`

Find flows through the graph without manual graph walking:

```
trace_path({
  from_label: "Button 1",
  direction: "downstream",
  filter: { node_metadata: { status: "error" } }
})
```

Returns ordered nodes and edges. Filter by metadata to isolate happy paths, error paths, or any subset. Combine with `apply_intents` to animate the result.

---

## Tool Examples

### Flow Diagram

```
add_node("Input")
add_node("Process")
add_node("Output")
connect("Input", "Process")
connect("Process", "Output")
layout("LR")
```

### State Machine (with animation)

```python
# Build a traffic light state machine
apply_intents([
  { tool: "add_node", args: { label: "RED",    shape: "ellipse", style: { background: "#ff6b6b" } }, ref: "red" },
  { tool: "add_node", args: { label: "GREEN",  shape: "ellipse", style: { background: "#69db7c" } }, ref: "grn" },
  { tool: "add_node", args: { label: "YELLOW", shape: "ellipse", style: { background: "#ffd43b" } }, ref: "yel" },
  { tool: "connect",  args: { from_id: "$ref:red.id", to_id: "$ref:grn.id", label: "60s" } },
  { tool: "connect",  args: { from_id: "$ref:grn.id", to_id: "$ref:yel.id", label: "45s" } },
  { tool: "connect",  args: { from_id: "$ref:yel.id", to_id: "$ref:red.id", label: "5s"  } },
  { tool: "layout",   args: { style: "LR" } }
])

# Animate the cycle
apply_intents([
  { tool: "animate_element", args: { id: "$ref:red.id", to: { opacity: 100 }, duration_ms: 700 } },
  { tool: "animate_element", args: { id: "$ref:red.id", to: { opacity: 20  }, duration_ms: 200 } },
  { tool: "animate_element", args: { id: "$ref:grn.id", to: { opacity: 100 }, duration_ms: 700 } },
  ...
])
```

### CI/CD Pipeline (real example)

22 operations — 10 nodes, 11 connections, layout — in a single MCP call. The agent then calls `trace_path` to find the happy path and animates it with another single call.

*(See `cicd-pipeline-demo.excalidraw` for the saved canvas.)*

### Responding to Human Input

A human types "Change the timer to 120s?" directly on the canvas. The agent's webhook fires. The agent reads `changed_elements_compact`, finds the text, acts on it, and responds with a thought bubble confirming the change — all without leaving the whiteboard.

---

## Agent Instructions Template

Add this to your agent's instructions (adapt webhook URL for your framework):

```
## Napkin (Collaborative Canvas)

When the napkin MCP server is available, you have a shared Excalidraw whiteboard.

**First — always call:**
  start_session({ session_id: "<your channel ID>", webhook_url: "<your webhook>" })

**Drawing — use the intent API (no coordinates needed):**
  add_node, connect, move, resize, style, add_label, delete_element, layout

**For multi-step operations:**
  apply_intents([...ops...]) — build entire diagrams in one call, chain refs

**Reading:**
  get_canvas_summary() — cheap reasoning pass (nodes + edges only)
  get_canvas() — full semantic output including zones and metadata
  trace_path(from_label, direction, filter) — graph traversal

**On webhook trigger:**
  1. add_thought_bubble() — acknowledge visually before processing
  2. Use changed_elements_compact from the payload — no round-trip needed
  3. Skip triggers with change_type "cosmetic" if you only care about structure
  4. Skip triggers with source "reconnect"

**Metadata convention:**
  intent, notes, status (wip|review|done|parking_lot), owner
```

---

## Tool Reference

### Read
| Tool | Purpose |
|------|---------|
| `get_canvas` | Full semantic canvas — nodes, edges, zones, metadata |
| `get_canvas_summary` | Compact read — nodes and edges only |
| `get_canvas_raw` | Raw Excalidraw JSON |
| `get_canvas_diff` | Elements changed since a timestamp |
| `get_pending_triggers` | Poll for triggers (pull mode) |
| `trace_path` | Traverse graph from a node, with optional metadata filtering |
| `get_server_instructions` | Compact or verbose server-side guidance for agents |

### Write
| Tool | Purpose |
|------|---------|
| `apply_intents` | Execute ordered batch of operations in one call |
| `add_node` | Add labeled node — server handles placement. Optional `zone`+`row` pin nodes to a shared layout rank (see Swimlanes). |
| `connect` | Connect nodes with an arrow |
| `move` | Move element by offset (dx, dy) |
| `resize` | Resize element, center preserved |
| `style` | Apply color, fill, opacity, stroke |
| `add_label` | Floating text near an element |
| `delete_element` | Remove element and bound text |
| `patch_canvas` | Partial field updates on existing elements |
| `update_canvas` | Add elements with full definitions |
| `clear_canvas` | Remove everything |

### Layout
| Tool | Purpose |
|------|---------|
| `layout` | Auto-arrange via Dagre (`TB`, `LR`, `tree`, `hierarchy`) |

### Thought Bubbles
| Tool | Purpose |
|------|---------|
| `add_thought_bubble` | Dashed purple tentative suggestion |
| `confirm_thought_bubble` | Make permanent |
| `dismiss_thought_bubble` | Remove |
| `list_thought_bubbles` | List current bubbles |

### Vision
| Tool | Purpose |
|------|---------|
| `describe_elements` | Render to PNG, send to Claude vision for description |
| `describe_sketch` | Convenience wrapper for freehand elements |

### Animation & Export
| Tool | Purpose |
|------|---------|
| `animate_element` | Interpolate position/size/opacity/color over time |
| `export_canvas` | Save to `.excalidraw`, `.svg`, or `.png` |

### Sessions
| Tool | Purpose |
|------|---------|
| `start_session` | Begin session with channel ID, webhook, debounce override |
| `end_session` | End session |

---

## Troubleshooting

**Canvas appears blank after agent draws something**
The browser is the source of truth. If it reconnected between your agent's write and the next read, the server cache was reset. Call `start_session` again and redraw.

**Webhook isn't firing**
Confirm `start_session` was called with the correct `session_id` and `webhook_url`. Triggers are suppressed on agent writes — only human canvas edits fire them.

**Agent can't see what I drew**
Check `get_canvas_diff` with a recent timestamp. If the browser tab was closed and reopened, the canvas state was re-synced from the browser — call `get_canvas` for a fresh read.

**`export_canvas` produces a blank file**
Export immediately after drawing — don't let the browser reconnect between the write and export calls. Use an absolute path on the host machine.

## Design Decisions

**Canvas state lives in the browser.** The server caches it for agents but the browser is the source of truth. If the server restarts, the browser reconnects and re-syncs — no data loss as long as the tab is open. Export to `.excalidraw` for persistence.

**No chat UI.** The conversation lives in your agent's channel. Napkin is a visual tool. The trigger/webhook system routes canvas events back to the right conversation automatically.

**Echo suppression.** Agent writes don't trigger webhooks. The server tracks which element IDs were written by agents and ignores browser echoes of those writes for ~2s. Only genuine human edits fire triggers.

**Per-session isolation.** Each `start_session` call gets its own debounce timer, webhook URL, and trigger context. Multiple agents can share a canvas with independent notification channels.


## Contributing

PRs welcome. The most useful directions right now: additional layout engines, integrations with other agent frameworks, and tighter agentic loop patterns. Open an issue first for anything structural.

## Security

Please review `SECURITY.md` for reporting guidance.

## License

MIT

More

• More mcp servers on No topic detected
• Everything published by sshwarts
• Browse the whole catalog