{
  "markdown": "# SpecterQA\n\n[![PyPI version](https://img.shields.io/pypi/v/specterqa.svg)](https://pypi.org/project/specterqa/)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)\n[![CI](https://github.com/SyncTek-LLC/specterqa/workflows/CI/badge.svg)](https://github.com/SyncTek-LLC/specterqa/actions)\n[![MCP Compatible](https://img.shields.io/badge/MCP-compatible-blueviolet)](https://github.com/SyncTek-LLC/specterqa/blob/main/docs/for-agents.md)\n[![FTI Trust Score](https://forgeos-api.synctek.io/v1/badge/pypi/specterqa/flat)](https://forgeos-api.synctek.io/v1/trust/pypi/specterqa)\n\n**AI personas walk your app so real users don't trip.**\n\nSpecterQA sends AI personas through your application — they look at the screen, decide what to do, and interact like real humans. No test scripts. No selectors. You describe personas and journeys in YAML, and SpecterQA handles the rest.\n\n```\n$ specterqa run -p myapp\n\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ SpecterQA Run                                                    ┃\n┃ Product: myapp   Budget: $5.00   Viewport: 1280x720            ┃\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛\n\n  ✓ Step 1/4: Navigate to homepage       PASS   3.2s   $0.0081\n  ✓ Step 2/4: Click signup link          PASS   2.1s   $0.0043\n  ✓ Step 3/4: Fill registration form     PASS   8.7s   $0.0312\n  ✓ Step 4/4: Verify dashboard loads     PASS   4.5s   $0.0127\n\n┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ ALL TESTS PASSED                                                ┃\n┃ Steps: 4/4   Findings: 0   Duration: 18.5s   Cost: $0.0563     ┃\n┃ Run ID: GQA-RUN-20260222-143052-a1b2                            ┃\n┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛\n```\n\n---\n\n## What is this?\n\nTraditional E2E tests are brittle. You write selectors, they break. You maintain scripts, they rot. SpecterQA takes a different approach: AI vision models look at your actual UI and navigate it the way a person would.\n\nYou define **personas** (who is using your app) and **journeys** (what they're trying to do). SpecterQA's engine takes a screenshot, sends it to a Claude vision model, gets back a decision (\"click this button\", \"fill this field\"), executes it via Playwright, takes another screenshot, and repeats until the goal is achieved or something goes wrong.\n\nWhen something goes wrong, you get evidence: screenshots, UX observations, cost breakdowns, and findings categorized by severity.\n\n## Installation\n\nSpecterQA is distributed via PyPI and requires Python 3.10 or later.\n\n```bash\npip install specterqa\n```\n\nAfter installing, download the Playwright browser binaries:\n\n```bash\nspecterqa install\n```\n\nFor macOS native app testing and iOS Simulator support, install the optional `native` extra:\n\n```bash\npip install specterqa[native]\n```\n\nFor MCP server support (integrating SpecterQA as a tool in Claude Desktop, Cursor, or other MCP clients):\n\n```bash\npip install specterqa[mcp]\n```\n\nYou will also need an Anthropic API key to run tests:\n\n```bash\nexport ANTHROPIC_API_KEY=sk-ant-...\n```\n\nTo verify the installation:\n\n```bash\nspecterqa --version\nspecterqa init       # scaffold a sample project\nspecterqa run -p demo\n```\n\n## Quick Start\n\n```bash\npip install specterqa\nspecterqa install          # downloads Playwright browsers\nspecterqa init             # scaffolds .specterqa/ with sample configs\nspecterqa run -p demo      # runs the sample journey\n```\n\nYou'll need an Anthropic API key:\n\n```bash\nexport ANTHROPIC_API_KEY=sk-ant-...\n```\n\nThat's it. Three commands and an API key.\n\n## How It Works\n\nThe core loop is simple:\n\n```\nscreenshot --> vision model --> action decision --> execute --> repeat\n```\n\n1. **Screenshot** -- Playwright captures the current page state as a PNG\n2. **Decide** -- A Claude vision model receives the screenshot + persona context + goal, returns a structured JSON action (`click`, `fill`, `navigate`, `scroll`, `keyboard`, `wait`, `done`, or `stuck`)\n3. **Execute** -- Playwright performs the action (click at coordinates, type text, navigate to URL, etc.)\n4. **Repeat** -- Loop until the goal is achieved, the agent gets stuck, or the budget runs out\n\nThe persona's profile shapes how the AI behaves. A \"tech-savvy developer\" explores differently than a \"frustrated first-time user.\" Persona patience, tech comfort, and frustrations all influence the system prompt.\n\n**Model routing** keeps costs down. Simple actions (click, scroll) use Haiku. Complex actions (form filling, initial assessment) use Sonnet. You can also route simple actions to a local Ollama model (llava:13b) for zero API cost on straightforward navigation.\n\n## Features\n\n- **Persona-based testing** -- Define AI users with backgrounds, goals, frustrations, and tech comfort levels. They don't just follow scripts; they react to what they see.\n- **Vision-powered** -- No selectors, no DOM queries. The AI interprets screenshots like a human would. Catches visual/layout issues that selector-based tests miss entirely.\n- **YAML-configured** -- Products, personas, and journeys are all YAML files. PMs can read them. No code to maintain.\n- **Budget enforcement** -- Per-run, per-day, and per-month cost caps. The engine hard-stops if you hit the limit. No surprise bills.\n- **JUnit XML output** -- Drop `--junit-xml results.xml` and plug it into any CI system.\n- **Tiered model routing** -- Haiku for cheap navigation, Sonnet for complex reasoning, optional local Ollama for zero-cost simple actions.\n- **Multi-platform** -- Web apps (via Playwright), macOS native apps (via Accessibility API + pyobjc), iOS Simulator (via simctl). Same YAML format, different runners.\n- **Evidence collection** -- Every run produces screenshots, a findings report, cost breakdown, and a structured JSON result. Everything is saved to an evidence directory.\n- **Stuck detection** -- If the AI repeats the same action or the UI stops changing, the engine escalates to a stronger model, then aborts if nothing works. No infinite loops.\n- **Template variables** -- Use `{{persona.credentials.email}}` in your journey steps. Variables resolve from persona configs at runtime.\n- **Precondition checks** -- Verify services are up before running tests. Fail fast with clear errors instead of wasting API calls.\n\n## Configuration\n\nSpecterQA uses three types of YAML config files, all living in `.specterqa/`:\n\n### Product (`products/myapp.yaml`)\n\n```yaml\nproduct:\n  name: myapp\n  display_name: \"My Application\"\n  base_url: \"http://localhost:3000\"\n\n  services:\n    frontend:\n      url: \"http://localhost:3000\"\n      health_endpoint: /\n\n  viewports:\n    desktop:\n      width: 1280\n      height: 720\n    mobile:\n      width: 375\n      height: 812\n\n  cost_limits:\n    per_run_usd: 5.00\n```\n\n### Persona (`personas/alex-developer.yaml`)\n\n```yaml\npersona:\n  name: alex_developer\n  display_name: \"Alex Chen\"\n  role: \"Full-Stack Developer\"\n  age: 28\n  tech_comfort: high\n  patience: medium\n  preferred_device: desktop\n\n  goals:\n    - \"Evaluate the app from a developer's perspective\"\n    - \"Check for common UX anti-patterns\"\n\n  frustrations:\n    - \"Unclear error messages\"\n    - \"Missing loading indicators\"\n\n  credentials:\n    email: \"alex@example.com\"\n    password: \"TestPass123!\"\n```\n\n### Journey (`journeys/onboarding.yaml`)\n\n```yaml\nscenario:\n  id: onboarding-happy-path\n  name: \"Onboarding Happy Path\"\n  description: \"New user signs up, completes onboarding, reaches dashboard.\"\n  tags: [onboarding, critical_path, smoke]\n\n  personas:\n    - ref: alex_developer\n      role: primary\n\n  preconditions:\n    - service: frontend\n      check: /\n      expected_status: 200\n\n  steps:\n    - id: visit_homepage\n      mode: browser\n      goal: \"Navigate to the homepage and verify it loads\"\n      checkpoints:\n        - type: text_present\n          value: \"Welcome\"\n\n    - id: navigate_signup\n      mode: browser\n      goal: \"Find and click the signup link\"\n\n    - id: fill_signup_form\n      mode: browser\n      goal: \"Complete the signup form with test credentials\"\n\n    - id: verify_dashboard\n      mode: browser\n      goal: \"Verify signup succeeded and the dashboard loads\"\n```\n\nSee [docs/configuration.md](docs/configuration.md) for the full reference.\n\n## YAML schema support\n\nSpecterQA includes a JSON Schema for product YAML files at `schemas/product.schema.json`.\n\n```yaml\n# yaml-language-server: $schema=../../schemas/product.schema.json\n\n```\n## CI Integration\n\nSpecterQA is built for CI. It runs headless by default and returns proper exit codes.\n\n```bash\n# Basic CI run\nspecterqa run -p myapp --junit-xml results.xml\n\n# Smoke test (runs first scenario only, fast)\nspecterqa run -p myapp --level smoke --budget 2.00\n\n# JSON output for programmatic consumption\nspecterqa run -p myapp --output json > results.json\n```\n\n**Exit codes:**\n- `0` -- all tests passed\n- `1` -- one or more tests failed\n- `2` -- configuration error\n- `3` -- infrastructure error (missing dependencies, API unreachable)\n\nSee [docs/ci-integration.md](docs/ci-integration.md) for GitHub Actions, GitLab CI, and CircleCI examples.\n\n## Cost\n\nSpecterQA uses Anthropic's Claude API. Every run costs money. Here's what to expect:\n\n| Model | Role | Input (per 1M tokens) | Output (per 1M tokens) |\n|-------|------|-----------------------|------------------------|\n| Claude Haiku 4.5 | Simple navigation | $0.80 | $4.00 |\n| Claude Sonnet 4 | Complex reasoning | $3.00 | $15.00 |\n| Ollama llava:13b | Local fallback | Free | Free |\n\n**Typical costs per run:**\n- 3-step smoke test: ~$0.30-0.60\n- 5-step standard journey: ~$0.50-1.50\n- Complex 10-step journey with forms: ~$1.00-3.00\n\nThe default budget is **$5.00 per run**. The engine hard-stops if the budget is exceeded -- no silent overruns. You can set per-day and per-month caps too.\n\nYou can also set a default budget via an environment variable to avoid passing `--budget` every time:\n\n```bash\nexport SPECTERQA_BUDGET=2.00\nspecterqa run -p myapp          # uses $2.00 budget\nspecterqa run -p myapp --budget 5.00  # uses $5.00 budget (CLI flag wins)\n```\n\nModel routing helps: simple clicks and scrolls use Haiku (~$0.01 per action), while form fills and initial assessments use Sonnet (~$0.03-0.05 per action). If you have a local Ollama instance, simple actions can route there for zero API cost.\n\nSee [docs/cost-guide.md](docs/cost-guide.md) for detailed cost breakdowns and budgeting strategies.\n\n## Multi-Platform\n\nSpecterQA isn't web-only. The same persona/journey YAML format works across platforms:\n\n**Web apps** (default) -- Uses Playwright for browser automation.\n\n**macOS native apps** -- Uses the macOS Accessibility API via pyobjc. The AI reads the accessibility tree and screenshots, then executes clicks and keypresses through AX actions.\n\n```yaml\nproduct:\n  name: my-mac-app\n  app_type: native_macos\n  app_path: /Applications/MyApp.app\n  bundle_id: com.example.myapp\n```\n\n**iOS Simulator** -- Uses `simctl` for screenshots and touch simulation. Useful for testing iOS apps without a physical device.\n\n```yaml\nproduct:\n  name: my-ios-app\n  app_type: ios_simulator\n  bundle_id: com.example.myiosapp\n  simulator_device: \"iPhone 15 Pro\"\n  simulator_os: \"17.2\"\n```\n\nNative and simulator support require the `native` optional dependency:\n\n```bash\npip install specterqa[native]\n```\n\n## For AI Agents\n\nIf you're an AI agent or building agent tooling, SpecterQA provides structured interfaces for programmatic use.\n\n### CLI with JSON output\n\n```bash\nspecterqa run -p myapp --output json\n```\n\nReturns structured JSON to stdout:\n\n```json\n{\n  \"passed\": true,\n  \"run_id\": \"GQA-RUN-20260222-143052-a1b2\",\n  \"step_reports\": [\n    {\n      \"step_id\": \"visit_homepage\",\n      \"passed\": true,\n      \"duration_seconds\": 12.3\n    }\n  ],\n  \"findings\": [],\n  \"cost_usd\": 0.4521\n}\n```\n\n### Python API\n\n```python\nfrom specterqa.config import SpecterQAConfig\nfrom specterqa.engine.orchestrator import SpecterQAOrchestrator\n\nconfig = SpecterQAConfig()\nconfig.project_dir = Path(\".specterqa\")\nconfig.products_dir = Path(\".specterqa/products\")\nconfig.personas_dir = Path(\".specterqa/personas\")\nconfig.journeys_dir = Path(\".specterqa/journeys\")\nconfig.evidence_dir = Path(\".specterqa/evidence\")\nconfig.anthropic_api_key = \"sk-ant-...\"\nconfig.budget = 5.00\nconfig.headless = True\n\norchestrator = SpecterQAOrchestrator(config)\nreport_md, all_passed = orchestrator.run(product=\"myapp\", level=\"smoke\")\n```\n\n### Federated Protocol\n\nSpecterQA exposes a `protocols.py` module with Python Protocol classes (`AIDecider`, `ActionExecutor`) that let you swap in your own AI model or action backend:\n\n```python\nfrom specterqa.engine.protocols import AIDecider, Decision\n\nclass MyCustomDecider:\n    def decide(self, goal, screenshot_base64, **kwargs) -> Decision:\n        # Your logic here\n        ...\n```\n\n### MCP Server\n\nSpecterQA ships an MCP (Model Context Protocol) server. Any MCP-compatible agent (Claude Desktop, Cursor, Cline, custom agent tooling) can discover and invoke SpecterQA as a tool -- run tests, read results, manage configs -- without shelling out to the CLI.\n\n**Add to your MCP client config (`claude_desktop_config.json` or equivalent):**\n\n```json\n{\n  \"specterqa\": {\n    \"command\": \"specterqa-mcp\",\n    \"args\": []\n  }\n}\n```\n\n**Available tools:**\n\n| Tool | Description |\n|------|-------------|\n| `specterqa_run` | Execute behavioral tests against a product. Synchronous — may take 45-300s. Incurs API costs (default budget: $5.00). |\n| `specterqa_list_products` | List configured products and their available journeys |\n| `specterqa_get_results` | Retrieve full structured results from a previous run by run ID |\n| `specterqa_init` | Initialize a new SpecterQA project directory |\n\nSee [docs/for-agents.md](docs/for-agents.md) for the full programmatic API reference and MCP integration details.\n\n## API Reference\n\nThe complete API reference is available at **[specterqa.synctek.io/docs](https://specterqa.synctek.io/docs)**.\n\n### Key classes\n\n| Class | Module | Description |\n|-------|--------|-------------|\n| `SpecterQAConfig` | `specterqa.config` | Root configuration object. Set project dirs, API key, budget, and model routing preferences. |\n| `SpecterQAOrchestrator` | `specterqa.engine.orchestrator` | Main entry point for programmatic runs. Call `orchestrator.run(product, level)` to execute a journey. |\n| `AIDecider` | `specterqa.engine.protocols` | Protocol class. Implement to swap in a custom vision model or decision backend. |\n| `ActionExecutor` | `specterqa.engine.protocols` | Protocol class. Implement to swap in a custom action execution backend (e.g., replace Playwright). |\n| `RunReport` | `specterqa.models` | Structured result returned by `orchestrator.run()`. Contains step reports, findings, and cost breakdown. |\n| `Finding` | `specterqa.models` | Individual UX issue captured during a run. Includes severity, step ID, screenshot reference, and description. |\n\n### CLI reference\n\n| Command | Description |\n|---------|-------------|\n| `specterqa run -p PRODUCT` | Run all journeys for a product |\n| `specterqa run -p PRODUCT --level smoke` | Run only smoke-tagged journeys |\n| `specterqa run -p PRODUCT --junit-xml results.xml` | Emit JUnit XML for CI |\n| `specterqa run -p PRODUCT --output json` | Emit structured JSON to stdout |\n| `specterqa init` | Scaffold a `.specterqa/` project directory with sample configs |\n| `specterqa install` | Download Playwright browser binaries |\n| `specterqa list` | List configured products and journeys |\n| `specterqa results RUN_ID` | Print the full report for a previous run |\n| `specterqa-mcp` | Start the MCP server |\n\n### MCP tools\n\n| Tool | Description |\n|------|-------------|\n| `specterqa_run` | Execute behavioral tests. Parameters: `product` (str), `level` (str, optional), `directory` (str, optional). Returns a `RunReport` JSON object. |\n| `specterqa_list_products` | List all products and their configured journeys. No parameters required. |\n| `specterqa_get_results` | Retrieve a previous run report by `run_id`. |\n| `specterqa_init` | Initialize a new SpecterQA project at a given `directory`. |\n\nFor schema definitions, type stubs, and federated protocol details, see [docs/for-agents.md](docs/for-agents.md).\n\n## Security\n\n**Directory access:** When the environment variable `SPECTERQA_ALLOWED_DIRS` is unset, the SpecterQA MCP server permits the `directory` parameter of `specterqa_run` to point at **any path on the filesystem** accessible to the process. In shared or multi-user environments — or anywhere the MCP server is exposed to untrusted agents — you should set this variable to an explicit allowlist:\n\n```bash\nexport SPECTERQA_ALLOWED_DIRS=\"/home/user/projects:/ci/workspaces\"\n```\n\nWhen set, the MCP server rejects any `directory` value that is not under one of the listed prefixes. This mitigates the MCP directory traversal vector described in [SECURITY_ADVISORY.md](SECURITY_ADVISORY.md) (GHSA-SPECTERQA-001).\n\n**Command injection fix (v0.2.1):** The `check_command` field in product YAML service definitions has been removed. It was the source of a critical command injection vulnerability. Precondition checks are now limited to TCP connectivity and HTTP health endpoint checks, which are safe. See [SECURITY_ADVISORY.md](SECURITY_ADVISORY.md) for full details.\n\n**Credential scrubbing:** Run artifacts (JSON result files, log output) automatically scrub known credential patterns — API keys, tokens, passwords — from captured content before writing to disk.\n\n**Reporting vulnerabilities:** Do not open public issues for security bugs. Email **info@synctek.io** or see [SECURITY.md](SECURITY.md) for the full disclosure policy.\n\n## Limitations\n\nBe honest with yourself about what this is and isn't:\n\n- **Requires an Anthropic API key.** No API key, no testing. There's no free tier built into SpecterQA itself.\n- **Costs money.** Every run makes API calls. A typical 3-step journey costs $0.30-0.60. Budget enforcement prevents surprises, but the meter is always running.\n- **Vision models aren't perfect.** The AI sometimes misreads small text, clicks the wrong element, or gets confused by complex layouts. It's good, not infallible. You'll occasionally see false positives and false negatives.\n- **Not a replacement for unit tests.** SpecterQA tests behavioral UX flows. It doesn't test your business logic, data integrity, or edge case handling. Use it alongside your existing test suite, not instead of it.\n- **macOS native testing requires pyobjc.** The `specterqa[native]` extra pulls in pyobjc packages (~200MB). Only needed for native macOS and iOS Simulator testing.\n- **Alpha software.** Version 0.4.0. APIs may change. File structure may change. Expect rough edges.\n- **Single-persona per journey (for now).** Multi-persona concurrent testing (e.g., simulating a chat between two users) is on the roadmap but not yet supported.\n- **Deterministic reproduction is hard.** Because the AI makes decisions at runtime, the exact sequence of actions varies between runs. Same journey, same persona, slightly different clicks. This is by design (it catches more issues) but makes exact reproduction tricky.\n\n## Contributing\n\nContributions welcome. The repo is at [github.com/SyncTek-LLC/specterqa](https://github.com/SyncTek-LLC/specterqa).\n\n```bash\ngit clone https://github.com/SyncTek-LLC/specterqa.git\ncd specterqa\npip install -e \".[dev]\"\npytest\n```\n\nOpen an issue before starting large PRs. We'd rather discuss the approach first.\n\n## License\n\nMIT -- see [LICENSE](LICENSE) for details.\n\n---\n\n<!-- mcp-name: io.github.SyncTekLLC/specterqa -->\n\nBuilt by [SyncTek LLC](https://github.com/SyncTek-LLC).\n",
  "bytes": 19743,
  "sha": "fda55b7113d7dfd1e7eb3c632a8c583f6df8ac201d98d8b83eb952ceda774ab3",
  "repo_slug": "synctek-llc/ghostqa",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_synctekllc_ghostqa_4455e1c6/readme"
}