{
  "markdown": "# WebScrape MCP Server\n\nEnglish · [Español](#español)\n\n---\n\n## English\n\nMCP server that lets AI agents search the web and extract clean Markdown content — no ads, no clutter, just the text your LLM needs.\n\n### What's New in v2.0.0\n\n**New features:**\n- **JS rendering** — Render JavaScript-heavy sites with Playwright (Chromium headless). Auto-fallback when httpx gets 403 or empty content\n- **Structured data extraction** — Extract JSON-LD, Open Graph, Twitter Cards, meta tags, canonical URLs, and hreflang links with `extract_schema=True`\n- **Screenshots** — New `webscrape_screenshot` tool captures page screenshots with configurable viewport, full-page mode, and PNG/JPEG format\n- **Multi-engine search** — DuckDuckGo primary, automatic fallback to Google and Bing if DDGS is unavailable\n- **Smart truncation** — Content is truncated at paragraph/sentence boundaries instead of mid-word\n\n**Improvements:**\n- Enhanced cache: 500 entries with 15-minute TTL (was 200, no expiry)\n- Better error handling with specific messages for 403, 404, 429, timeouts\n- Updated Dockerfile with Chromium dependencies for Playwright\n\n### Tools\n\n| Tool | Description |\n|------|-------------|\n| `webscrape_fetch_url` | Fetch a single URL and return clean Markdown. Supports `use_readability`, `js_render`, `extract_schema`, and auto-detects PDFs |\n| `webscrape_batch_fetch` | Fetch up to 5 URLs in parallel. Supports PDF auto-detection, JS rendering, and structured data |\n| `webscrape_search` | Search the web (DuckDuckGo → Google → Bing fallback) and scrape results into Markdown |\n| `webscrape_screenshot` | Capture a screenshot of any web page with headless Chromium. Supports PNG/JPEG, viewport sizing, and full-page capture |\n\n### Features\n\n- **PDF support**: URLs ending in `.pdf` or with `application/pdf` content-type are auto-detected and text is extracted page by page\n- **Readability mode**: Pass `use_readability=True` to `webscrape_fetch_url` for cleaner article extraction using Mozilla Readability (removes nav, sidebars, ads, comments)\n- **JS rendering**: Pass `js_render=True` to render JavaScript-heavy sites with Playwright (headless Chromium). Auto-fallback when httpx gets 403 or empty content\n- **Structured data extraction**: Pass `extract_schema=True` to extract JSON-LD, Open Graph, Twitter Cards, meta tags, canonical URLs, and hreflang links\n- **Multi-engine search**: DuckDuckGo primary, automatic fallback to Google and Bing if DDGS is unavailable\n- **Screenshots**: Capture page screenshots with configurable viewport, full-page mode, and PNG/JPEG format\n- **Built-in cache**: 500-entry cache with TTL-based eviction (15 min) for repeated URLs\n- **Batch fetching**: Up to 5 URLs in parallel\n- **Smart truncation**: Content is truncated at paragraph/sentence boundaries instead of mid-word\n\n### How to use\n\n#### Option 1 — MCPize (recommended)\n\n1. Go to https://mcpize.com/marketplace\n2. Search **Web Scrape** and click **Start Free**\n3. You'll get an API key\n4. Configure in your AI client:\n\n```json\n{\n  \"mcpServers\": {\n    \"webscrape\": {\n      \"url\": \"https://webscrape.mcpize.run\",\n      \"headers\": {\n        \"Authorization\": \"Bearer your-api-key\"\n      }\n    }\n  }\n}\n```\n\n#### Option 2 — Render (dev)\n\n```json\n{\n  \"mcpServers\": {\n    \"webscrape\": {\n      \"url\": \"https://webscrape-mcp.onrender.com\"\n    }\n  }\n}\n```\n\n#### Option 3 — Local\n\n```bash\ngit clone https://github.com/carrasquelalex1/webscrape-mcp.git\ncd webscrape-mcp\npip install -r requirements.txt\nplaywright install chromium\npython webscrape_mcp.py\n```\n\n### Official Registry\n\n`io.github.carrasquelalex1/webscrape-mcp`\n\n### Dependencies\n\n`mcp`, `httpx`, `beautifulsoup4`, `markdownify`, `pydantic`, `ddgs`, `readability-lxml`, `PyMuPDF`, `playwright`\n\n### License\n\nMIT\n\n---\n\n## Español\n\nServidor MCP que permite a agentes de IA buscar en la web y extraer contenido limpio en Markdown — sin anuncios, sin navegación, solo el texto que tu LLM necesita.\n\n### Novedades en v2.0.0\n\n**Nuevas características:**\n- **Rendering JS** — Renderiza sitios con JavaScript usando Playwright (Chromium headless). Fallback automático cuando httpx recibe 403 o contenido vacío\n- **Extracción de datos estructurados** — Extrae JSON-LD, Open Graph, Twitter Cards, meta tags, URLs canónicas y links hreflang con `extract_schema=True`\n- **Capturas de pantalla** — Nueva herramienta `webscrape_screenshot` que captura screenshots con viewport configurable, modo full-page y formato PNG/JPEG\n- **Búsqueda multi-motor** — DuckDuckGo primario, fallback automático a Google y Bing si DDGS no está disponible\n- **Truncado inteligente** — El contenido se trunca en límites de párrafo/oración en vez de cortar palabras a la mitad\n\n**Mejoras:**\n- Caché mejorada: 500 entradas con TTL de 15 minutos (antes 200, sin expiración)\n- Mejor manejo de errores con mensajes específicos para 403, 404, 429, timeouts\n- Dockerfile actualizado con dependencias de Chromium para Playwright\n\n### Tools\n\n| Tool | Descripción |\n|------|-------------|\n| `webscrape_fetch_url` | Obtiene una URL y la convierte a Markdown limpio. Soporta `use_readability`, `js_render`, `extract_schema`, y detecta PDFs automáticamente |\n| `webscrape_batch_fetch` | Obtiene hasta 5 URLs en paralelo. Soporta detección de PDFs, rendering JS, y datos estructurados |\n| `webscrape_search` | Busca en la web (DuckDuckGo → Google → Bing como fallback) y extrae los resultados a Markdown |\n| `webscrape_screenshot` | Captura una captura de pantalla de cualquier página web con Chromium headless. Soporta PNG/JPEG, tamaño de viewport, y captura completa |\n\n### Características\n\n- **Soporte PDF**: URLs que terminan en `.pdf` o con content-type `application/pdf` se detectan automáticamente y se extrae el texto página por página\n- **Modo Readability**: Usá `use_readability=True` en `webscrape_fetch_url` para extraer artículos de forma más limpia (elimina navegación, barras laterales, anuncios, comentarios)\n- **Rendering JS**: Usá `js_render=True` para renderizar sitios con JavaScript usando Playwright (Chromium headless). Fallback automático cuando httpx recibe 403 o contenido vacío\n- **Extracción de datos estructurados**: Usá `extract_schema=True` para extraer JSON-LD, Open Graph, Twitter Cards, meta tags, URLs canónicas, y links hreflang\n- **Búsqueda multi-motor**: DuckDuckGo primario, fallback automático a Google y Bing si DDGS no está disponible\n- **Capturas de pantalla**: Captura de páginas con viewport configurable, modo full-page, y formato PNG/JPEG\n- **Caché integrada**: 500 entradas con evicción por TTL (15 min) para URLs repetidas\n- **Batch fetching**: Hasta 5 URLs en paralelo\n- **Truncado inteligente**: El contenido se trunca en límites de párrafo/oración en vez de cortar palabras a la mitad\n\n### Cómo usarlo\n\n#### Opción 1 — MCPize (recomendada)\n\n1. Ve a https://mcpize.com/marketplace\n2. Busca **Web Scrape** y haz clic en **Start Free**\n3. Obtendrás una API key\n4. Configura en tu cliente de IA:\n\n```json\n{\n  \"mcpServers\": {\n    \"webscrape\": {\n      \"url\": \"https://webscrape.mcpize.run\",\n      \"headers\": {\n        \"Authorization\": \"Bearer tu-api-key\"\n      }\n    }\n  }\n}\n```\n\n#### Opción 2 — Render (desarrollo)\n\n```json\n{\n  \"mcpServers\": {\n    \"webscrape\": {\n      \"url\": \"https://webscrape-mcp.onrender.com\"\n    }\n  }\n}\n```\n\n#### Opción 3 — Local\n\n```bash\ngit clone https://github.com/carrasquelalex1/webscrape-mcp.git\ncd webscrape-mcp\npip install -r requirements.txt\nplaywright install chromium\npython webscrape_mcp.py\n```\n\n### Registro Oficial\n\n`io.github.carrasquelalex1/webscrape-mcp`\n\n### Dependencias\n\n`mcp`, `httpx`, `beautifulsoup4`, `markdownify`, `pydantic`, `ddgs`, `readability-lxml`, `PyMuPDF`, `playwright`\n\n### Licencia\n\nMIT\n",
  "bytes": 7681,
  "sha": "58a1d7bfe9366fa761ac64f408f09c724942b3d6993c281ee1c6709b861e803e",
  "repo_slug": "carrasquelalex1/webscrape-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_carrasquelalex1_webscrape_mcp_25e551d4/readme"
}