{
"markdown": "# Fetch URL MCP Server\n\n[](https://www.npmjs.com/package/%40j0hanz%2Ffetch-url-mcp) [](#contributing-and-license)\n\n[](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D) [](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D&quality=insiders) [](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)\n\n[](https://lmstudio.ai/install-mcp?name=fetch-url&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D) [](https://cursor.com/en/install-mcp?name=fetch-url&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)\n\nAn MCP server that fetches web pages and converts them to clean, readable Markdown.\n\n## Overview\n\nThis server takes a URL, fetches the page, and strips away everything you don't need — navigation, sidebars, banners, scripts — leaving just the main content as Markdown. It's perfect for feeding into LLMs, giving them the distilled essence of a page without the noise. It also recognizes GitHub, GitLab, Bitbucket, and Gist URLs and rewrites them to fetch the raw content directly.\n\nBy default it runs over stdio. Pass `--http` if you need a proper HTTP endpoint with auth, rate limiting, TLS, and session support.\n\n## Key Features\n\n- **HTML to Markdown** — Turns any public web page into clean, readable Markdown with metadata like `title`, `url`, `contentSize`, and `truncated`.\n- **Smart URL handling** — Recognizes GitHub, GitLab, Bitbucket, and Gist page URLs and rewrites them to raw-content endpoints before fetching.\n- **Task mode** — Big or slow pages can run as async MCP tasks with progress updates, instead of blocking. In HTTP mode, tasks are bound to the authenticated caller rather than a single MCP session, so they can be resumed after reconnecting as the same authenticated subject. Polling task state exposes task summaries; numeric progress remains out-of-band via `notifications/progress`.\n- **Self-documenting** — Includes an `internal://instructions` resource and a `get-help` prompt so clients know how to use it.\n- **HTTP mode** — Optionally serves over Streamable HTTP with host/origin validation, bearer or OAuth auth, rate limiting, health checks, and TLS.\n\n## Web Client\n\nA browser-based client is available if you want to use the server without any MCP setup.\n\n**[Live app](https://fetch-url-client.vercel.app)** · [Source code](https://github.com/j0hanz/fetch-url)\n\n## Requirements\n\n- **Node.js** >= 24\n- **Docker** (optional) — only needed if you want to run the container image\n\n## Quick Start\n\nAdd this to your MCP client config:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\n## Client Configuration\n\n\nInstall in VS Code
\n\n[](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D)\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n \"servers\": {\n \"fetch-url-mcp\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nOr install via CLI:\n\n```sh\ncode --add-mcp '{\"name\":\"fetch-url-mcp\",\"command\":\"npx\",\"args\":[\"-y\",\"@j0hanz/fetch-url-mcp@latest\"]}'\n```\n\nFor more info, see [VS Code MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n \n\n\nInstall in VS Code Insiders
\n\n[](https://insiders.vscode.dev/redirect/mcp/install?name=fetch-url&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D&quality=insiders)\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n \"servers\": {\n \"fetch-url-mcp\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nOr install via CLI:\n\n```sh\ncode-insiders --add-mcp '{\"name\":\"fetch-url-mcp\",\"command\":\"npx\",\"args\":[\"-y\",\"@j0hanz/fetch-url-mcp@latest\"]}'\n```\n\nFor more info, see [VS Code Insiders MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n \n\n\nInstall in Cursor
\n\n[](https://cursor.com/en/install-mcp?name=fetch-url&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)\n\nAdd to `~/.cursor/mcp.json`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Cursor MCP docs](https://docs.cursor.com/context/model-context-protocol).\n\n \n\n\nInstall in Visual Studio
\n\n[](https://vs-open.link/mcp-install?%7B%22fetch-url-mcp%22%3A%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40j0hanz%2Ffetch-url-mcp%40latest%22%5D%7D%7D)\n\nFor solution-scoped setup, add this to `.mcp.json` at the solution root:\n\n```json\n{\n \"servers\": {\n \"fetch-url-mcp\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Visual Studio MCP docs](https://learn.microsoft.com/en-us/visualstudio/ide/mcp-servers).\n\n \n\n\nInstall in Goose
\n\nAdd to `~/.config/goose/config.yaml` on macOS/Linux or `%APPDATA%\\Block\\goose\\config\\config.yaml` on Windows:\n\n```yaml\nextensions:\n fetch-url-mcp:\n name: fetch-url-mcp\n cmd: npx\n args: ['-y', '@j0hanz/fetch-url-mcp@latest']\n enabled: true\n type: stdio\n timeout: 300\n```\n\nFor more info, see [Goose extension docs](https://block.github.io/goose/docs/getting-started/using-extensions/).\n\n \n\n\nInstall in LM Studio
\n\n[](https://lmstudio.ai/install-mcp?name=fetch-url&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBqMGhhbnovZmV0Y2gtdXJsLW1jcEBsYXRlc3QiXX0%3D)\n\nAdd to `~/.lmstudio/mcp.json` on macOS/Linux or `%USERPROFILE%/.lmstudio/mcp.json` on Windows:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [LM Studio MCP docs](https://lmstudio.ai/docs/basics/mcp).\n\n \n\n\nInstall in Claude Desktop
\n\nAdd to `claude_desktop_config.json`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Claude Desktop MCP docs](https://modelcontextprotocol.io/quickstart/user).\n\n \n\n\nInstall in Claude Code
\n\nUse the CLI:\n\n```sh\nclaude mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nFor project-scoped config, Claude Code writes `.mcp.json` with:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"],\n \"env\": {}\n }\n }\n}\n```\n\nFor more info, see [Claude Code MCP docs](https://docs.anthropic.com/en/docs/claude-code/mcp).\n\n \n\n\nInstall in Windsurf
\n\nAdd to `~/.codeium/windsurf/mcp_config.json`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Windsurf MCP docs](https://docs.windsurf.com/windsurf/cascade/mcp).\n\n \n\n\nInstall in Amp
\n\nAdd to `~/.config/amp/settings.json` on macOS/Linux, `%USERPROFILE%\\.config\\amp\\settings.json` on Windows, or `.amp/settings.json` for workspace-scoped config:\n\n```json\n{\n \"amp.mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nOr install via CLI:\n\n```sh\namp mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nFor more info, see [Amp docs](https://ampcode.com/manual).\n\n \n\n\nInstall in Cline
\n\nOpen the MCP Servers panel, choose `Configure MCP Servers`, and add this to `cline_mcp_settings.json`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Cline MCP docs](https://docs.cline.bot/mcp/configuring-mcp-servers).\n\n \n\n\nInstall in Codex CLI
\n\nUse the CLI:\n\n```sh\ncodex mcp add fetch-url-mcp -- npx -y @j0hanz/fetch-url-mcp@latest\n```\n\nOr add this to `~/.codex/config.toml` or project-scoped `.codex/config.toml`:\n\n```toml\n[mcp_servers.fetch-url-mcp]\ncommand = \"npx\"\nargs = [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n```\n\nFor more info, see [Codex MCP docs](https://developers.openai.com/codex/mcp/).\n\n \n\n\nInstall in GitHub Copilot
\n\nAdd to `.vscode/mcp.json`:\n\n```json\n{\n \"servers\": {\n \"fetch-url-mcp\": {\n \"type\": \"stdio\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [GitHub Copilot MCP docs](https://code.visualstudio.com/docs/copilot/chat/mcp-servers).\n\n \n\n\nInstall in Warp
\n\nOpen `Personal > MCP Servers` in Warp, choose `+ Add`, and either add a CLI server with:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nOr paste this JSON snippet when using Warp's multi-server import flow:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Warp MCP docs](https://docs.warp.dev/features/warp-ai/mcp).\n\n \n\n\nInstall in Kiro
\n\nUse Kiro's MCP Servers panel or the `Add to Kiro` install flow. Kiro stores workspace-scoped MCP config in `.kiro/settings/mcp.json` and user-scoped config in `~/.kiro/settings/mcp.json`.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Kiro MCP docs](https://kiro.dev/blog/unlock-your-development-productivity-with-kiro-and-mcp/).\n\n \n\n\nInstall in Gemini CLI
\n\nAdd to `~/.gemini/settings.json`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Gemini CLI MCP docs](https://google-gemini.github.io/gemini-cli/docs/tools/mcp-server.html).\n\n \n\n\nInstall in Zed
\n\nAdd to `~/.config/zed/settings.json`:\n\n```json\n{\n \"context_servers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"],\n \"env\": {}\n }\n }\n}\n```\n\nFor more info, see [Zed MCP docs](https://zed.dev/docs/ai/mcp).\n\n \n\n\nInstall in Augment
\n\nUse the Augment Settings panel and either add the server manually or choose `Import from JSON`:\n\n```json\n{\n \"mcpServers\": {\n \"fetch-url-mcp\": {\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]\n }\n }\n}\n```\n\nFor more info, see [Augment MCP docs](https://docs.augmentcode.com/setup-augment/mcp).\n\n \n\n\nInstall in Roo Code
\n\nUse Roo Code's MCP Servers UI or marketplace flow.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Roo Code docs](https://docs.roocode.com/).\n\n \n\n\nInstall in Kilo Code
\n\nUse Kilo Code's MCP Servers UI or marketplace flow.\n\nFor this server, use:\n\n- `command`: `npx`\n- `args`: `[\"-y\", \"@j0hanz/fetch-url-mcp@latest\"]`\n\nFor more info, see [Kilo Code docs](https://kilocode.ai/docs).\n\n \n\n## Use Cases\n\n- **Documentation for LLMs** — Grab a docs page, blog post, or reference article as Markdown and pass it straight into a context window.\n- **Repository content** — Hand it a GitHub, GitLab, or Bitbucket URL and it resolves the raw content endpoint. Works with Gists too.\n- **Slow or large pages** — Task mode lets big fetches run in the background while sending monotonic progress updates back to the client, while `tasks/get` exposes the latest task summary fields such as `statusMessage`, `createdAt`, `lastUpdatedAt`, `ttl`, and `pollInterval`.\n\n## Architecture\n\n```text\n[MCP Client]\n ├─ stdio -> `src/index.ts` -> `startStdioServer()` -> `createMcpServer()`\n └─ HTTP (`--http`) -> `src/index.ts` -> `startHttpServer()` -> HTTP dispatcher\n ├─ `GET /health`\n ├─ `GET /.well-known/oauth-protected-resource`\n ├─ `GET /.well-known/oauth-protected-resource/mcp`\n └─ `POST|GET|DELETE /mcp`\n\n`createMcpServer()`\n ├─ registers tool: `fetch-url`\n ├─ registers prompt: `get-help`\n ├─ registers resources:\n │ - `internal://instructions`\n ├─ enables capabilities: logging, resources, prompts, tasks\n └─ installs task handlers, log-level handling, and shutdown cleanup\n\n`fetch-url` execution\n ├─ validate input with `fetchUrlInputSchema`\n ├─ normalize URL and block local/private targets unless allowed\n ├─ rewrite supported code-host URLs to raw endpoints when possible\n ├─ fetch content via the shared pipeline\n ├─ transform HTML into Markdown in the transform worker path\n └─ validate `structuredContent` with `fetchUrlOutputSchema`\n```\n\n### Request Lifecycle\n\n```text\n[Client] -- initialize {protocolVersion, capabilities} --> [Server]\n[Server] -- {protocolVersion, capabilities, serverInfo} --> [Client]\n[Client] -- notifications/initialized --> [Server]\n[Client] -- tools/call {name, arguments} --> [Server]\n[Server] -- {content: [{type, text}], structuredContent?, isError?} --> [Client]\n```\n\n## MCP Surface\n\n### Tools\n\n#### `fetch-url`\n\nTakes a URL and returns Markdown. Read-only, with no JavaScript execution. Supports running as a background MCP task for large or slow pages. In task mode, `tasks/get` and `tasks/list` expose task summaries including `status`, `statusMessage`, `createdAt`, `lastUpdatedAt`, `ttl`, and `pollInterval`; numeric progress remains out-of-band via `notifications/progress`.\n\n| Parameter | Type | Required | Description |\n| --------- | -------- | -------- | --------------------------- |\n| `url` | `string` | yes | Target URL. Max 2048 chars. |\n\nYou get text content back by default. If output validation passes, the response also includes `structuredContent` with typed fields: `url`, `resolvedUrl`, `finalUrl`, `title`, `metadata`, `markdown`, `fetchedAt`, `contentSize`, and `truncated`. A `true` value for `truncated` means the content hit a server-side size limit.\n\nTo opt into progress updates, include `_meta.progressToken` in the tool call. The token may be a string or number, and the server may emit monotonic `notifications/progress` updates while the fetch runs.\n\nTo run the tool in task mode, include `params.task = { ttl?: }`. `tasks/result` waits until the task reaches a terminal status and then returns the stored output or a terminal error payload for cancelled or failed tasks. Task summaries and final results include `_meta[\"io.modelcontextprotocol/related-task\"] = { \"taskId\": \"\" }`.\n\n```json\n{\n \"method\": \"tools/call\",\n \"params\": {\n \"name\": \"fetch-url\",\n \"arguments\": {\n \"url\": \"https://example.com/docs\"\n },\n \"task\": {\n \"ttl\": 300000,\n \"pollInterval\": 1000\n },\n \"_meta\": {\n \"progressToken\": 7\n }\n }\n}\n```\n\n```text\n1. [Client] -- tools/call {name: \"fetch-url\", arguments} --> [Server]\n2. [Server] -- dispatch(\"fetch-url\") --> [src/tools/fetch-url.ts]\n3. [Handler] -- validate(fetchUrlInputSchema) --> normalize / fetch / transform\n4. [Handler] -- validate(fetchUrlOutputSchema) --> assemble content + structuredContent\n5. [Server] -- result or tool error --> [Client]\n```\n\n### Resources\n\n| Resource | URI | MIME Type | Description |\n| ---------------------------- | ------------------------- | --------------- | -------------------------------------------- |\n| `fetch-url-mcp-instructions` | `internal://instructions` | `text/markdown` | Guidance for using the Fetch URL MCP server. |\n\n### Prompts\n\n| Prompt | Arguments | Description |\n| ---------- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |\n| `get-help` | `topic?` | Return Fetch URL server instructions: workflows, task mode, and error handling. Optional values: `capabilities`, `workflows`, `constraints`, `errors`. |\n\n## MCP Capabilities\n\n| Capability | Status | Notes |\n| ---------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------- |\n| completions | confirmed | Advertised in `createServerCapabilities()`. |\n| logging | confirmed | Advertised in `createServerCapabilities()`. |\n| resources | confirmed | Static instruction resource is registered during server startup. Subscription and list-changed support are not advertised. |\n| prompts | confirmed | `get-help` is registered during server startup. |\n| tasks | confirmed | Advertised in `createServerCapabilities()` and backed by registered task handlers plus optional tool task support. |\n| progress notifications | confirmed | Opt-in via `_meta.progressToken`. Tool execution reports monotonic `notifications/progress` updates during fetch and transform stages. |\n| task status updates | confirmed | `notifications/tasks/status` is emitted when task status changes and `TASKS_STATUS_NOTIFICATIONS=true`. |\n\n### Tool Annotations\n\n| Annotation | Value |\n| ----------------- | ------- |\n| `readOnlyHint` | `true` |\n| `destructiveHint` | `false` |\n| `idempotentHint` | `true` |\n| `openWorldHint` | `true` |\n\n### Structured Output\n\nThe tool declares an `outputSchema` and includes `structuredContent` in the response when validation passes. Clients that support structured output get typed data directly; the rest use the text fallback.\n\n## Configuration\n\nAll configuration is through environment variables. For basic stdio usage, nothing needs to be set.\n\n### HTTP Server\n\n| Variable | Default | Notes |\n| ------------------------------------- | ----------- | ---------------------------------------------------------------------- |\n| `HOST` | `127.0.0.1` | Bind address. Non-loopback bindings also require `ALLOW_REMOTE=true`. |\n| `PORT` | `3000` | Listening port for `--http`. |\n| `ALLOW_REMOTE` | `false` | Must be enabled to bind to a non-loopback interface. |\n| `ALLOWED_HOSTS` | empty | Additional allowed `Host` and `Origin` values. |\n| `SERVER_MAX_CONNECTIONS` | `0` | Optional connection cap. |\n| `SERVER_TRUST_PROXY` | `false` | Trust `X-Forwarded-For` / `Forwarded` for client IP resolution. |\n| `SERVER_HEADERS_TIMEOUT_MS` | unset | Optional Node server tuning. |\n| `SERVER_REQUEST_TIMEOUT_MS` | unset | Optional Node server tuning. |\n| `SERVER_KEEP_ALIVE_TIMEOUT_MS` | unset | Optional keep-alive tuning. |\n| `SERVER_KEEP_ALIVE_TIMEOUT_BUFFER_MS` | unset | Optional keep-alive tuning buffer. |\n| `SERVER_MAX_HEADERS_COUNT` | unset | Optional header count limit. |\n| `SERVER_BLOCK_PRIVATE_CONNECTIONS` | `false` | Enables inbound private-network protections when not trusting a proxy. |\n\n### Authentication & OAuth\n\n| Variable | Default | Notes |\n| ------------------------- | ------- | ----------------------------------------------------------- |\n| `ACCESS_TOKENS` | unset | Comma- or space-separated static bearer tokens. |\n| `API_KEY` | unset | Alternate static token source for header auth. |\n| `OAUTH_ISSUER_URL` | unset | Enables OAuth mode when combined with the other OAuth URLs. |\n| `OAUTH_AUTHORIZATION_URL` | unset | Optional explicit authorization endpoint. |\n| `OAUTH_TOKEN_URL` | unset | Optional explicit token endpoint. |\n| `OAUTH_REVOCATION_URL` | unset | Optional OAuth revocation endpoint. |\n| `OAUTH_REGISTRATION_URL` | unset | Optional OAuth dynamic client registration endpoint. |\n| `OAUTH_INTROSPECTION_URL` | unset | Required for OAuth token introspection. |\n| `OAUTH_REQUIRED_SCOPES` | empty | Required scopes enforced after auth. |\n| `OAUTH_CLIENT_ID` | unset | Optional introspection client ID. |\n| `OAUTH_CLIENT_SECRET` | unset | Optional introspection client secret. |\n\n### TLS\n\n| Variable | Default | Notes |\n| ---------------------- | ------- | ----------------------------------------------------------- |\n| `SERVER_TLS_KEY_FILE` | unset | Enable HTTPS when set together with `SERVER_TLS_CERT_FILE`. |\n| `SERVER_TLS_CERT_FILE` | unset | TLS certificate path. |\n| `SERVER_TLS_CA_FILE` | unset | Optional custom CA bundle. |\n\n### Fetching\n\n| Variable | Default | Notes |\n| ------------------- | ------------------------- | -------------------------------------------------- |\n| `ALLOW_LOCAL_FETCH` | `false` | Allows loopback and private-network fetch targets. |\n| `FETCH_TIMEOUT_MS` | `15000` | Network fetch timeout in milliseconds. |\n| `USER_AGENT` | `fetch-url-mcp/` | Override the outbound user agent string. |\n\n### Tool Output\n\n| Variable | Default | Notes |\n| -------------------------- | ------- | ---------------------------------------------- |\n| `MAX_INLINE_CONTENT_CHARS` | `0` | `0` means no explicit inline truncation limit. |\n\n### Tasks\n\n| Variable | Default | Notes |\n| ---------------------------- | ------- | ------------------------------------------------------------------------------------ |\n| `TASKS_MAX_TOTAL` | `5000` | Total retained task capacity, including completed/cancelled tasks until they expire. |\n| `TASKS_MAX_PER_OWNER` | `1000` | Per-owner retained task cap, clamped to the total cap. |\n| `TASKS_STATUS_NOTIFICATIONS` | `false` | Enables status notifications for tasks. |\n| `TASKS_REQUIRE_INTERCEPTION` | `true` | Requires interception for task-capable tool execution. |\n\n### Transform Workers\n\n| Variable | Default | Notes |\n| ------------------------------------------ | --------- | ------------------------------------- |\n| `TRANSFORM_CANCEL_ACK_TIMEOUT_MS` | `200` | Cancellation acknowledgement timeout. |\n| `TRANSFORM_WORKER_MODE` | `threads` | Worker execution mode. |\n| `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB` | unset | Optional worker memory limit. |\n| `TRANSFORM_WORKER_MAX_YOUNG_GENERATION_MB` | unset | Optional worker memory limit. |\n| `TRANSFORM_WORKER_CODE_RANGE_MB` | unset | Optional worker memory limit. |\n| `TRANSFORM_WORKER_STACK_MB` | unset | Optional worker stack size. |\n\n### Content Cleanup\n\n| Variable | Default | Notes |\n| ------------------------------------- | -------------- | ------------------------------------------ |\n| `FETCH_URL_MCP_EXTRA_NOISE_TOKENS` | empty | Extra noise-removal tokens. |\n| `FETCH_URL_MCP_EXTRA_NOISE_SELECTORS` | empty | Extra DOM selectors for noise removal. |\n| `FETCH_URL_MCP_LOCALE` | system default | Locale override for extraction heuristics. |\n| `MARKDOWN_HEADING_KEYWORDS` | built-in list | Override heading keywords used by cleanup. |\n\n### Logging\n\n| Variable | Default | Notes |\n| ------------ | ------- | ------------------------------------ |\n| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error`. |\n| `LOG_FORMAT` | `text` | Set to `json` for structured logs. |\n\n## HTTP Endpoints\n\n| Method | Path | Auth | Purpose |\n| -------- | ------------------------------------------- | ------------------------------------------ | ------------------------------------------------------- |\n| `GET` | `/health` | no, unless `?verbose=1` on a remote server | Basic health response, with optional diagnostics. |\n| `GET` | `/.well-known/oauth-protected-resource` | no | OAuth protected-resource metadata. |\n| `GET` | `/.well-known/oauth-protected-resource/mcp` | no | OAuth protected-resource metadata for the MCP endpoint. |\n| `POST` | `/mcp` | yes | Session initialization and JSON-RPC requests. |\n| `GET` | `/mcp` | yes | Session-bound server-to-client stream handling. |\n| `DELETE` | `/mcp` | yes | Session shutdown. |\n\n## Security\n\n| Control | Status | Notes |\n| -------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| Host and origin validation | implemented | HTTP requests are rejected unless `Host` and `Origin` match the allowlist built from loopback, the configured host, and `ALLOWED_HOSTS`. |\n| Authentication | implemented | HTTP mode supports static bearer tokens locally or OAuth token introspection; remote bindings require OAuth. |\n| Protocol version checks | implemented | Session-bound MCP HTTP requests validate `MCP-Protocol-Version` and pin it to the negotiated session version. |\n| Rate limiting | implemented | Requests pass through the HTTP rate limiter before route dispatch. Enable `SERVER_TRUST_PROXY=true` behind a trusted reverse proxy so limits apply to the forwarded client IP instead of the proxy hop. |\n| Outbound SSRF protections | implemented | Local/private IPs, metadata endpoints, and `.local`/`.internal` hosts are blocked unless `ALLOW_LOCAL_FETCH=true`. |\n| TLS | optional | HTTPS is enabled when both TLS key and certificate files are configured. |\n| Stdio logging safety | implemented | Server logs are written to stderr, not stdout, so stdio MCP traffic stays clean. |\n\n## Development\n\n### Essential Commands\n\n| Command | Description |\n| -------------------- | ------------------------------------------------- |\n| `npm run build` | Clean, compile TypeScript, copy assets. |\n| `npm run dev` | Watch mode TypeScript compilation. |\n| `npm run dev:run` | Run the server with `--watch` and `.env` support. |\n| `npm start` | Start the compiled server. |\n| `npm test` | Run the full test suite. |\n| `npm run lint` | Lint with ESLint. |\n| `npm run lint:fix` | Auto-fix lint issues. |\n| `npm run type-check` | Type-check source and tests. |\n| `npm run format` | Format with Prettier. |\n| `npm run inspector` | Build and launch MCP Inspector. |\n\n\nAll npm scripts
\n\n| Script | Command |\n| ------------------------ | ------------------------------------------------------------------------------------------------------------------- |\n| `clean` | `node scripts/tasks.mjs clean` |\n| `build` | `node scripts/tasks.mjs build` |\n| `copy:assets` | `node scripts/tasks.mjs copy:assets` |\n| `prepare` | `npm run build` |\n| `dev` | `tsc --watch --preserveWatchOutput` |\n| `dev:run` | `node --env-file=.env --watch dist/index.js` |\n| `start` | `node dist/index.js` |\n| `format` | `prettier --write .` |\n| `type-check` | `node scripts/tasks.mjs type-check` |\n| `type-check:src` | `node node_modules/typescript/bin/tsc -p tsconfig.json --noEmit` |\n| `type-check:tests` | `node node_modules/typescript/bin/tsc -p tsconfig.test.json --noEmit` |\n| `type-check:diagnostics` | `tsc --noEmit --extendedDiagnostics` |\n| `type-check:trace` | `node -e \"require('fs').rmSync('.ts-trace',{recursive:true,force:true})\" && tsc --noEmit --generateTrace .ts-trace` |\n| `lint` | `eslint .` |\n| `lint:tests` | `eslint src/__tests__` |\n| `lint:fix` | `eslint . --fix` |\n| `test` | `node scripts/tasks.mjs test` |\n| `test:fast` | `node --test --import tsx/esm src/__tests__/**/*.test.ts node-tests/**/*.test.ts` |\n| `test:coverage` | `node scripts/tasks.mjs test --coverage` |\n| `knip` | `knip` |\n| `knip:fix` | `knip --fix` |\n| `inspector` | `npm run build && npx -y @modelcontextprotocol/inspector node dist/index.js --stdio` |\n| `prepublishOnly` | `npm run lint && npm run type-check && npm run build` |\n\n \n\n## Build and Release\n\n- `npm run prepublishOnly` runs lint, type-check, and build as a single release gate.\n- CI workflows are under `.github/workflows/`.\n- `Dockerfile` and `docker-compose.yml` are included for containerized runs.\n- Published on npm as [`@j0hanz/fetch-url-mcp`](https://www.npmjs.com/package/@j0hanz/fetch-url-mcp).\n\n## Troubleshooting\n\n| Symptom | Likely Cause | Fix |\n| --------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------------------- |\n| Server output mixes with MCP traffic on stdio | Logs going to stdout | Ensure all logging writes to stderr; the server does this by default. |\n| HTTP mode returns `403` | Host/origin mismatch | Add the domain to `ALLOWED_HOSTS` or verify loopback bindings. |\n| HTTP mode rate limits every request from your proxy | `SERVER_TRUST_PROXY` not enabled | Enable `SERVER_TRUST_PROXY=true` when the server is behind a trusted reverse proxy. |\n| HTTP mode returns `401` | Missing or invalid token | Set `ACCESS_TOKENS` or configure OAuth env vars for remote bindings. |\n| Fetch returns private-IP error | SSRF protections blocked the target | Set `ALLOW_LOCAL_FETCH=true` if the target is intentionally local. |\n| `truncated: true` in response | Content exceeded inline limits | Increase `MAX_INLINE_CONTENT_CHARS` or accept truncated output. |\n| Transform timeout or worker crash | Large or complex HTML | Tune `TRANSFORM_WORKER_MAX_OLD_GENERATION_MB` or increase `FETCH_TIMEOUT_MS`. |\n| Client config not working | Wrong config format for the client | Check the matching `` block above — config keys vary by client. |\n\n## Credits\n\n| Dependency | Registry |\n| ------------------------------------------------------------------------------------------ | -------- |\n| [@modelcontextprotocol/server](https://www.npmjs.com/package/@modelcontextprotocol/server) | npm |\n| [@modelcontextprotocol/node](https://www.npmjs.com/package/@modelcontextprotocol/node) | npm |\n| [@mozilla/readability](https://www.npmjs.com/package/@mozilla/readability) | npm |\n| [linkedom](https://www.npmjs.com/package/linkedom) | npm |\n| [node-html-markdown](https://www.npmjs.com/package/node-html-markdown) | npm |\n| [undici](https://www.npmjs.com/package/undici) | npm |\n| [zod](https://www.npmjs.com/package/zod) | npm |\n\n## Contributing and License\n\nPull requests welcome. Please make sure these pass before submitting:\n\n1. `npm run lint` and `npm run type-check`\n2. `npm test`\n3. `npm run format`\n\n## License\n\nMIT License. See [LICENSE](LICENSE) for details.\n",
"bytes": 39308,
"sha": "fa1d5890cac02594b648eef0ccb4bc95800be9c7fcd2ec9684ca7685a2891fcc",
"repo_slug": "j0hanz/super-fetch-mcp-server",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_j0hanz_superfetch_d8c08e31/readme"
}