Agent Watch
Monitoring for the agent economy — liveness, latency, trust scoring for MCP endpoints
Open source Repository Open in the app JSON README (API)
About
Monitoring for the agent economy — liveness, latency, trust scoring for MCP endpoints
Details
- Kind
- MCP servers
- Topic
- Cloud & DevOps
- Publisher
- entradox
- Origin
- official
- Category
- ferramentas
- Transport
- http
- Version
- 0.1.0
- Last push
- 2026-09-07T19:40:22Z
- Repository state
- ativo
- Language
- Python
- Added
- 2026-09-05 03:00:32
- Updated
- 2026-09-05 03:00:32
- Origin id
io.github.entradox/agent-watch
README
# Agent Watch — Project README **Status:** Two capabilities, two different maturity levels (see below). **Frame card:** `frame-card.md` (original MCP-monitoring scope). Grantwatch's own design/plan live in the parent AI-Workbench repo: `docs/superpowers/specs/2026-09-05-grantwatch-oauth-scope-monitoring-design.md` and `docs/superpowers/plans/2026-09-05-grantwatch-oauth-scope-monitoring.md`. **Sibling project:** `~/AI-Workbench/projects/perimeter-watch/` — Agent Watch reuses Perimeter Watch's live Stripe account (GASPERMIT acct) rather than standing up new billing. ## What this is Agent Watch monitors the agent economy from two directions: 1. **Supply side (original, live in production):** liveness, latency, schema-drift, price-integrity, and auth-posture monitoring for MCP registry endpoints and x402/paid agent services. "Is this endpoint alive and honest about what it does?" 2. **Demand side (Grantwatch, new, not yet customer-facing):** for an SMB's own Google Workspace tenant, which third-party AI agents/apps hold OAuth grants, whether granted scope matches actual usage, and whether any grant looks abandoned or still-active-when-it-shouldn't-be. "Which AI agents did your employees let in, and are they behaving?" Both are sold to the same buyer profile (SMB/MSP), reuse the same infrastructure pattern (probe/ingest → baseline → drift diff → alert), and are packaged under one product, not two. ## Capability 1: MCP/agent-endpoint monitoring (supply side) **Status:** deployed and registry-listed. No paid tier wired yet. | Piece | Status | Evidence | |---|---|---| | `probe_engine.py` — MCP handshake + latency + schema-hash + price-integrity + auth-metadata probe | LIVE | `compute_trust_score()` produces a 0-100/A-F trust score for a monitored endpoint | | REST API on Railway | LIVE | `https://agent-watch-api-production.up.railway.app` — `/health`, `/v1/probe`, `/v1/endpoints`, `/v1/alerts`, `/stats` | | CLI (`repo/cli/agent-watch.py`) | BUILT | `add/status/list/alert/history/scan` | | MCP server (`repo/mcp/aw_mcp_server.py`) | BUILT, registry-listed | `server.json` — `io.github.entradox/agent-watch`, streamable-http remote | | `llms.txt` (agent-facing API reference) | LIVE | `repo/llms.txt` | | Public status page | LIVE | `/status`, `status.html` | | Stripe billing | **NOT YET WIRED** | Frame card targets $9-99/mo metered tiers; no Stripe config exists in this repo yet — this is the actual gap between "built" and "monetized" for capability 1 | **Known gap:** the original frame card's `$9-99/mo metered SaaS` billing was never actually set up. Capability 1 is a live, working, registry-listed free tool with no revenue rail — worth deciding whether to wire billing here or let Grantwatch be the first paid tier under this product. ## Capability 2: Grantwatch — OAuth scope drift monitoring (demand side) **Status:** engine complete on `feature/grantwatch`, Turing-approved (27/27 tests passing), not yet merged, not yet customer-facing. | Piece | Status | |---|---| | Tenant authorization gate (`grantwatch/gate.py`) | DONE — mirrors Perimeter Watch's `domains_authorized.txt` pattern | | Scope-diff engine (`grantwatch/scope_diff.py`) | DONE — findings only, no score/grade by design | | Google Workspace connector (`grantwatch/connectors/google_workspace.py`) | DONE, mocked-tested only — real use requires Google OAuth restricted-scope verification (see below) | | Microsoft 365 connector (`grantwatch/connectors/microsoft365.py`) | DONE, built standalone, **not wired into the audit flow** — launch scope is Google Workspace only | | Coverage disclosure (`grantwatch/coverage.py`) | DONE — a monitoring gap is always stated, never hidden; no "cleared"/GREEN state anywhere in this codebase | | Free "Shadow Agent Audit" lead magnet (`grantwatch/shadow_audit.py`) | DONE | | REST API wiring (`/v1/grantwatch/audit` on the existing `api_server.py`) | DONE | | Per-tenant credential pointer storage (`grantwatch/credential_store.py`) | DONE — stores a path pointer, never the raw secret | | Binding kill-criteria check (`grantwatch/kill_criteria.py`) | DONE — K1: <5 free-audit-to-paid conversions by day 45; K2: <$200 MRR by day 90 | | Stripe billing (3 tiers: Starter $49, Growth $149, MSP $399/mo) | **NOT DONE** — manual Stripe CLI step, requires the operator | | Real-tenant end-to-end verification | **NOT DONE** — requires a real Google Workspace admin + service account setup | ### Hard rules (same discipline as Perimeter Watch) 1. **No customer tenant is ever scanned without written authority confirmation.** The scan gate (`tenants_authorized.txt`) is the enforcement point. 2. **Read-only, always.** Grantwatch never writes to a customer's tenant. 3. **No graded verdict, ever.** No score, no percentage, no Red/Amber/Green badge — findings only, plus explicit coverage disclosure of what wasn't checked. This was a deliberate architecture decision after an adversarial review found that "cryptographic proof of correct agent behavior" overclaims what's actually observable (you can prove a log wasn't tampered with, not that it's complete). 4. **Kill criteria (K1/K2 above) are binding**, matching Perimeter Watch's pattern. ### Blocking conditions before scanning any real customer tenant (Morgan gate, 2026-09-05) Logged to `~/.claude/team/gate-log.csv` as `decision=hold` (approved-with-conditions). None of these block further engineering — they block Task 11 (first real tenant scan): - **Google OAuth restricted-scope verification + CASA Tier 2 security assessment** — `admin.reports.audit.readonly` is a Google Restricted Scope; using it against any tenant beyond an internal dev/test account requires this. Weeks-long, real cost. This is the longest-lead item and the most likely to affect launch timing — start it independent of any code work. - **Microsoft Publisher Verification** — recommended before selling the M365 tier (not yet built into the flow anyway). - **Pre-consent clickwrap** on the connect flow (Morgan drafted starter language; not yet built as a UI/flow). - **"Limited Use" clause** in a public Grantwatch privacy policy (doesn't exist yet). - **Retention purge jobs implemented in code** — 90-day rolling window on raw audit events, 15-day purge for the free one-time audit, 30-day post-cancellation purge for the app list. Turing-reviewed, not just documented policy. - **Signed DPA** with the first paying customer (lightweight template OK for MVP; real counsel required once any tenant has EU/UK employees or at Growth/MSP tier). - **PII handling controls** in the connector layer — `actor_email` encrypted at rest/in transit, excluded from logs/observability tools, excluded from any public "anonymized findings" marketing content. ## Known gaps (honest, both capabilities) - Capability 1 has no billing wired despite being live — see above. - Grantwatch's Microsoft connector is fully built and tested but not reachable from any user-facing flow — deliberate v1 scoping, not an oversight. - Neither capability has been used against a real, paying customer yet.