{
  "markdown": "<p align=\"center\">\n  <picture>\n    <source media=\"(prefers-color-scheme: dark)\" srcset=\"assets/logo-dark-horizontal-trans.png\">\n    <source media=\"(prefers-color-scheme: light)\" srcset=\"assets/logo-white-horizontal-trans.png\">\n    <img alt=\"Scrapi AI\" src=\"assets/logo-white-horizontal.png\" width=\"360\">\n  </picture>\n</p>\n\n<h3 align=\"center\">MCP server that converts URLs to clean Markdown/Text for LLM agents</h3>\n\n<p align=\"center\">\n  <a href=\"README-KO.md\">한국어</a> ·\n  <a href=\"https://scrapi.ai\">Website</a> ·\n  <a href=\"https://scrapi.ai/dashboard\">Dashboard</a>\n</p>\n\n<p align=\"center\">\n  <a href=\"https://www.npmjs.com/package/@scrapi.ai/mcp-server\"><img src=\"https://img.shields.io/npm/v/@scrapi.ai/mcp-server.svg\" alt=\"npm version\"></a>\n  <a href=\"https://opensource.org/licenses/MIT\"><img src=\"https://img.shields.io/badge/License-MIT-yellow.svg\" alt=\"License: MIT\"></a>\n</p>\n\n**⚡ Fast & Reliable** — Built on 8+ years of web scraping expertise, 1,900+ production crawlers, and battle-tested anti-bot handling.\n\n## What is this?\n\nAn [MCP (Model Context Protocol)](https://modelcontextprotocol.io/) server that lets AI agents fetch and read web pages. Simply give it a URL, and it returns clean, LLM-ready content — fast.\n\n**Before:** AI can't read web pages directly  \n**After:** \"Summarize this article\" just works ✨\n\n---\n\n## Features\n\n- 🌐 **URL → Markdown**: Preserves headings, lists, links\n- 📄 **URL → Text**: Plain text extraction\n- 🏷️ **Metadata**: Title, author, date, images\n- 🧹 **Clean Output**: No ads, no navigation, no scripts\n- ⚡ **JavaScript Rendering**: Works with SPAs\n- 💳 **Built-in Billing**: Credit tracking, subscription management, usage analytics (MCP keys)\n- 🔄 **Auto-Retry**: 429 rate limit responses automatically retried with Retry-After\n- 🌍 **Dual Transport**: Stdio (npx) + Streamable HTTP for flexible deployment\n\n---\n\n## Transport Modes\n\nScrapi MCP Server supports two transport modes:\n\n| Mode | Best For | Node.js Required |\n|------|----------|-----------------|\n| **Stdio** | Claude Desktop, Cursor, Cline, Claude Code | Yes (auto via npx) |\n| **Streamable HTTP** | All clients, Node.js-free environments | No |\n\n---\n\n## Prerequisites\n\n- [Scrapi MCP](https://scrapi.ai) account (separate from the main Scrapi account)\n- Claude Desktop, Cline, or Cursor installed\n- Node.js 20+\n\n---\n\n## Installation\n\n### Option A: npx (Recommended)\n\nNo installation needed. Just configure your MCP client to use `npx`.\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@scrapi.ai/mcp-server\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n> **Tip:** You can also pass the API key via CLI argument instead of env var:\n> ```json\n> \"args\": [\"-y\", \"@scrapi.ai/mcp-server\", \"--api-key\", \"your-api-key\"]\n> ```\n\n> See [Step 2](#step-2-configure-mcp-server) for where to put this configuration.\n\n### Option B: Install from Source\n\n```bash\n# Clone the repository\ngit clone https://github.com/bamchi/scrapi-mcp-server.git\ncd scrapi-mcp-server\n\n# Install dependencies and build\nnpm install && npm run build\n```\n\n---\n\n## Step 1: Get Your API Key\n\n1. Go to [https://scrapi.ai](https://scrapi.ai)\n2. Sign up or log in\n3. Visit the [MCP Dashboard](https://scrapi.ai/dashboard) — your Free plan (500 credits/month) and API key are created automatically\n4. Copy your `hsmcp_` API key\n\n---\n\n## Step 2: Configure MCP Server\n\n### Claude Desktop\n\n**Option A: Via Settings (Recommended)**\n\n1. Open Claude Desktop\n2. Click Settings (gear icon, bottom left)\n3. Select Developer tab\n4. Click \"Edit Config\" button\n5. Add the mcpServers configuration (see below)\n6. Save and restart Claude Desktop (Cmd+Q, then reopen)\n\n**Option B: Edit config file directly**\n\n- macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`\n- Windows: `%APPDATA%\\Claude\\claude_desktop_config.json`\n\n**Configuration (npx):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@scrapi.ai/mcp-server\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n**Configuration (from source):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"node\",\n      \"args\": [\"/absolute/path/to/scrapi-mcp-server/dist/index.js\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n> Note: Replace `/absolute/path/to/` with the actual path where you cloned the repository.\n\n### Cline\n\nConfig file location:\n- macOS: `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json`\n- Windows: `%APPDATA%\\Code\\User\\globalStorage\\saoudrizwan.claude-dev\\settings\\cline_mcp_settings.json`\n\n**Configuration (npx):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@scrapi.ai/mcp-server\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n**Configuration (from source):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"node\",\n      \"args\": [\"/absolute/path/to/scrapi-mcp-server/dist/index.js\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n### Cursor\n\nCreate or edit `.cursor/mcp.json` in your project root:\n\n**Configuration (npx):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@scrapi.ai/mcp-server\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n**Configuration (from source):**\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"node\",\n      \"args\": [\"/absolute/path/to/scrapi-mcp-server/dist/index.js\"],\n      \"env\": {\n        \"SCRAPI_API_KEY\": \"your-api-key\"\n      }\n    }\n  }\n}\n```\n\n### Claude Code\n\n**Option 1: CLI command (Recommended)**\n\n```bash\nclaude mcp add scrapi-ai -s user -e SCRAPI_API_KEY=your-api-key -- npx -y @scrapi.ai/mcp-server\n```\n\nOr with `--api-key`:\n\n```bash\nclaude mcp add scrapi-ai -s user -- npx -y @scrapi.ai/mcp-server --api-key your-api-key\n```\n\n**Option 2: Edit config file**\n\nEdit `~/.claude.json` or project `.mcp.json`:\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"@scrapi.ai/mcp-server\", \"--api-key\", \"your-api-key\"]\n    }\n  }\n}\n```\n\n### Streamable HTTP\n\nConnect via Streamable HTTP — no Node.js installation needed on the client side.\n\nEndpoint: `https://scrapi.ai/mcp`\n\n**Cursor** (`.cursor/mcp.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"url\": \"https://scrapi.ai/mcp\",\n      \"headers\": {\n        \"Authorization\": \"Bearer your-api-key\"\n      }\n    }\n  }\n}\n```\n\n**Claude Code** (CLI):\n\n```bash\nclaude mcp add --transport http scrapi https://scrapi.ai/mcp \\\n  --header \"Authorization: Bearer your-api-key\"\n```\n\n**Cline** (`cline_mcp_settings.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"type\": \"streamableHttp\",\n      \"url\": \"https://scrapi.ai/mcp\",\n      \"headers\": {\n        \"Authorization\": \"Bearer your-api-key\"\n      }\n    }\n  }\n}\n```\n\n**Claude Desktop** (`claude_desktop_config.json`):\n\n```json\n{\n  \"mcpServers\": {\n    \"scrapi\": {\n      \"command\": \"npx\",\n      \"args\": [\n        \"mcp-remote\",\n        \"https://scrapi.ai/mcp\",\n        \"--header\",\n        \"Authorization: Bearer your-api-key\"\n      ]\n    }\n  }\n}\n```\n\n> Note: Claude Desktop requires the [mcp-remote](https://www.npmjs.com/package/mcp-remote) proxy for HTTP connections.\n\n<details>\n<summary>Self-host the HTTP server (advanced)</summary>\n\nRun your own instance instead of using the hosted endpoint:\n\n```bash\nSCRAPI_API_KEY=your-api-key npx -y -p @scrapi.ai/mcp-server scrapi-http\n# or from source:\nSCRAPI_API_KEY=your-api-key node dist/http.js\n```\n\nThe server starts at `http://localhost:3000` with the MCP endpoint at `/mcp`. Configure with `PORT` and `HOST` environment variables. Replace the URL in the client configurations above with your self-hosted URL (e.g. `http://localhost:3000/mcp`).\n\n**Health check:** `GET http://localhost:3000/health`\n\n</details>\n\n---\n\n## Step 3: Restart Your AI Client\n\n- **Claude Desktop**: Fully quit (Cmd+Q on macOS, Alt+F4 on Windows) and reopen\n- **Claude Code**: Restart the session\n- **Cline**: Restart VS Code\n- **Cursor**: Restart the editor\n\nYou should see the MCP server connection indicator.\n\n---\n\n## Available Tools\n\n### `scrape_url`\n\nScrapes a webpage and returns AI-readable content.\n\n**Parameters:**\n\n| Name     | Type   | Required | Description                              |\n| -------- | ------ | -------- | ---------------------------------------- |\n| `url`    | string | ✅        | URL to scrape                            |\n| `format` | string |          | `markdown` (default) or `text`           |\n\n**Example:**\n\n```json\n{\n  \"url\": \"https://example.com/article\",\n  \"format\": \"markdown\"\n}\n```\n\n**Markdown Output:**\n\n```markdown\n# Article Title\n\n> Author: John Doe | Published: 2024-01-15\n\n## Introduction\n\nThis is the main content of the article, converted to clean markdown...\n\n## Key Points\n\n- Point 1: Important detail\n- Point 2: Another insight\n- [Related Link](https://example.com/related)\n```\n\n**Text Output:**\n\n```text\nArticle Title\n\nAuthor: John Doe | Published: 2024-01-15\n\nIntroduction\n\nThis is the main content of the article, converted to plain text...\n\nKey Points\n\n- Point 1: Important detail\n- Point 2: Another insight\n```\n\n### `scrape_urls`\n\nScrapes multiple webpages in parallel and returns AI-readable content.\n\n**Parameters:**\n\n| Name     | Type     | Required | Description                              |\n| -------- | -------- | -------- | ---------------------------------------- |\n| `urls`   | string[] | ✅        | URLs to scrape (max 10)                  |\n| `format` | string   |          | `markdown` (default) or `text`           |\n\n**Example:**\n\n```json\n{\n  \"urls\": [\"https://example.com/page1\", \"https://example.com/page2\"],\n  \"format\": \"text\"\n}\n```\n\n**Output:**\n\n```json\n[\n  {\n    \"url\": \"https://example.com/page1\",\n    \"content\": \"Page 1 Title\\n\\nThis is the content of page 1...\"\n  },\n  {\n    \"url\": \"https://example.com/page2\",\n    \"content\": \"Page 2 Title\\n\\nThis is the content of page 2...\"\n  }\n]\n```\n\n### `scraper_server_status`\n\nCheck the status of all ScraperServer instances. Shows server health, circuit breaker state, failure counts, and timing info.\n\n**Parameters:** None\n\n**Example:**\n\n```json\n{}\n```\n\n**Output:**\n\n```markdown\n## ScraperServer Status\n\nTotal: 3 | Available: 2\n\n| Name | OS | Status | Failures | Last Success | Last Failure |\n|------|----|--------|----------|--------------|--------------|\n| pluto | linux | OK | 0 | 01/30 14:23:05 | - |\n| mars | mac | FAIL | 2 | 01/29 10:00:00 | 01/30 13:55:12 |\n| venus | linux | OPEN | 3 | 01/28 09:00:00 | 01/30 12:00:00 |\n\n### Issues\n- **mars**: Connection refused - connect(2)\n- **venus**: Circuit breaker open until 01/30 12:30:00\n- **venus**: Net::ReadTimeout\n```\n\n**Status values:**\n\n| Status | Description |\n|--------|-------------|\n| `OK` | Server is healthy |\n| `FAIL` | Server is unhealthy |\n| `OPEN` | Circuit breaker open (isolated for 30 min) |\n| `N/A` | Not yet checked |\n\n### `get_usage`\n\nCheck your API usage and remaining credits.\n\n**Parameters:** None\n\n**Example:**\n\n```json\n{}\n```\n\n**Output:**\n\n```markdown\n## MCP Credits\n\n| Item | Value |\n|------|-------|\n| Plan | starter |\n| Subscription Credits | 1,500 |\n| Purchased Credits | 200 |\n| Total Remaining | 1,700 |\n| Period End | 2026-03-01 |\n```\n\n### `get_billing`\n\nRetrieve detailed billing information including subscription, plans, daily usage, and spending limits.\n\n**Parameters:**\n\n| Name | Type | Required | Description |\n|------|------|----------|-------------|\n| `action` | string | Yes | `subscription`, `plans`, `daily_usage`, or `spending_limits` |\n| `start_date` | string | | Start date for `daily_usage` (YYYY-MM-DD, default: 30 days ago) |\n| `end_date` | string | | End date for `daily_usage` (YYYY-MM-DD, default: today) |\n\n**Example — Current subscription:**\n\n```json\n{ \"action\": \"subscription\" }\n```\n\n```markdown\n## MCP Subscription\n\n| Item | Value |\n|------|-------|\n| Plan | starter (Starter) |\n| Status | active |\n| Monthly Credits | 2,000 |\n| Price | $19.00/mo |\n| Rate Limit | 30 RPM |\n| Burst Limit | 5 concurrent |\n| Period End | 2026-03-01 |\n```\n\n**Example — Available plans:**\n\n```json\n{ \"action\": \"plans\" }\n```\n\n```markdown\n## Available MCP Plans\n\n| Plan | Credits/mo | Price | RPM | Burst |\n|------|-----------|-------|-----|-------|\n| Free (free) | 500 | Free | 10 | 2 |\n| Starter (starter) | 2,000 | $19.00/mo | 30 | 5 |\n| Pro (pro) | 10,000 | $49.00/mo | 60 | 10 |\n| Business (business) | 50,000 | $149.00/mo | 120 | 20 |\n```\n\n**Example — Daily usage history:**\n\n```json\n{ \"action\": \"daily_usage\", \"start_date\": \"2026-02-01\", \"end_date\": \"2026-02-07\" }\n```\n\n```markdown\n## Daily Usage (2026-02-01 ~ 2026-02-07)\n\n| Date | Requests | Credits | Top Tool |\n|------|----------|---------|----------|\n| 2026-02-07 | 45 | 45 | scrape#scrape (45) |\n| 2026-02-06 | 120 | 120 | scrape#scrape (100) |\n\n**Total**: 165 requests, 165 credits\n```\n\n**Example — Spending limits:**\n\n```json\n{ \"action\": \"spending_limits\" }\n```\n\n```markdown\n## Spending Limits\n\n| Item | Value |\n|------|-------|\n| Daily Limit | 500 credits |\n| Today's Usage | 120 credits |\n| Usage % | 24.0% |\n```\n\n---\n\n## Usage Examples\n\n### Example 1: Summarize a News Article\n\n```\nUser: Summarize this article: https://news.example.com/article/12345\n\nClaude: [calls scrape_url]\n\nHere's a summary of the article:\n\n## Key Points\n- Point 1: ...\n- Point 2: ...\n- Point 3: ...\n```\n\n### Example 2: Fetch Page Content\n\n```\nUser: Get the content from https://example.com/data\n\nClaude: [calls scrape_url]\n\n# Page Title\n> Source: https://example.com/data\n\nThe page content is returned in clean Markdown format...\n```\n\n### Example 3: Research Competitor Pricing\n\n```\nUser: What's the pricing on https://competitor.com/product/abc\n\nClaude: [calls scrape_url]\n\nHere's the pricing information:\n- **Product**: ABC Premium\n- **Regular Price**: $99.00\n- **Sale Price**: $79.00 (20% off)\n```\n\n### Example 4: Read API Documentation\n\n```\nUser: Read https://docs.example.com/api/v2 and write integration code\n\nClaude: [calls scrape_url]\n\nI've analyzed the API documentation. Here's the integration code:\n\n// api-client.ts\nexport class ExampleApiClient {\n  private baseUrl = 'https://api.example.com/v2';\n  \n  async getData(): Promise<Response> {\n    // ...\n  }\n}\n```\n\n---\n\n## How It Works\n\n```\n┌─────────────────┐\n│     User        │\n│ \"Summarize this │\n│   URL for me\"   │\n└────────┬────────┘\n         │\n         ▼\n┌─────────────────┐\n│  Claude Desktop │\n│    / Cursor     │\n└────────┬────────┘\n         │\n         ▼\n┌─────────────────┐     ┌─────────────────┐\n│   MCP Server    │────►│   Scrapi API    │\n│  (scrape_url)   │     │ (format param)  │\n└────────┬────────┘     └────────┬────────┘\n         │                       │\n         │◄──────────────────────┘\n         │   Markdown/Text Response\n         ▼\n┌─────────────────┐\n│   AI Response   │\n│ (Summary, etc.) │\n└─────────────────┘\n```\n\n---\n\n## Why Scrapi?\n\nBuilt by the team behind [Scrapi](https://scrapi.ai), with 8+ years of web scraping experience:\n\n- ✅ 1,900+ production crawlers\n- ✅ JavaScript rendering support\n- ✅ Anti-bot handling\n- ✅ 99.9% uptime\n\n---\n\n## Troubleshooting\n\n### \"API key is required\"\n\nMake sure your API key is provided via one of these methods:\n- **Environment variable**: Set `SCRAPI_API_KEY` in your configuration\n- **CLI argument**: Pass `--api-key your-key` in the args\n\n### \"Invalid API key\"\n\nVerify that your API key is correct and active in your Scrapi dashboard.\n\n### npx using an old cached version\n\nIf you upgraded but still see old behavior, clear the npx cache:\n```bash\nnpx clear-npx-cache\n```\n\n### MCP Server not connecting\n\n1. Ensure Node.js 20+ is installed\n2. Try running `node /absolute/path/to/scrapi-mcp-server/dist/index.js` manually to check for errors\n3. Fully quit Claude Desktop (Cmd+Q on macOS, Alt+F4 on Windows) and restart\n4. Check Settings > Developer to verify the server is listed\n\n### Developer tab not visible\n\nUpdate Claude Desktop to the latest version: Claude menu → \"Check for Updates...\"\n\n---\n\n## Support\n\n- Email: support@scrapi.ai\n- Issues: [GitHub Issues](https://github.com/bamchi/scrapi-mcp-server/issues)\n\n---\n\n## License\n\nMIT © [Scrapi](https://scrapi.ai)\n",
  "bytes": 16284,
  "sha": "6928b16e81319365640dafa1107a317da2fb62b787f41b9d5760370d620e15ae",
  "repo_slug": "bamchi/scrapi-mcp-server",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_bamchi_scrapi_31b772f5/readme"
}