{
  "markdown": "# LionScraper MCP + CLI + HTTP API bridge\n\n[简体中文](README_cn.md)\n\n- **Website**: [lionscraper.com](https://www.lionscraper.com/)\n- **npm**: [package `lionscraper`](https://www.npmjs.com/package/lionscraper)\n- **PyPI**: [project `lionscraper`](https://pypi.org/project/lionscraper/)\n\n## What is this?\n\n**LionScraper** is a browser extension that can collect lists, articles, links, images, and more from web pages. This repository provides the companion **bridge** between your tools and that extension in three ways:\n\n- **MCP** (`lionscraper-mcp`): connect an **AI app** (e.g. Cursor) so the model can call scraping tools over stdio.\n- **CLI** (`lionscraper`): run **daemon**, **scrape**, **ping**, and more from a terminal on the same local HTTP/WebSocket port as the extension.\n- **HTTP API**: when the daemon is running, call the same capabilities over **loopback JSON HTTP** (e.g. `/v1/...`) from scripts or any HTTP client—no MCP or CLI front-end required.\n\n**The real scraping logic runs in the extension**; these packages connect and forward.\n\n## Before you start\n\n1. **Browser**: Chrome or Edge (follow what the extension supports).\n2. **LionScraper extension**: install and enable from the store.\n   - **Chrome**: [Chrome Web Store — LionScraper](https://chromewebstore.google.com/detail/godiccfjpjdapemodajccjjjcdcccimf)\n   - **Microsoft Edge**: [Edge Add-ons — LionScraper](https://microsoftedge.microsoft.com/addons/detail/llfpnjbphhfkgbgljpngbjpjpnljkijk)\n3. **Runtime** (pick one or both implementations):\n   - **Node.js** **18+** for the npm package — [Node.js](https://nodejs.org/)\n   - **Python** **3.10+** for the PyPI package — [Python](https://www.python.org/downloads/)\n4. **For MCP**: an AI app that supports MCP (e.g. Cursor, Trae).\n5. **For the HTTP API**: same browser, extension, and daemon as the CLI; see the package READMEs for paths and examples.\n\n**HTTP fallback without Chrome/Edge**: If neither browser is detected under standard paths and the extension is not connected, MCP still starts; `ping` succeeds with **http_fetch** mode and `scrape*` use a minimal server-side HTTP GET (no JS execution). If a browser is installed but the extension is not connected, you still get the extension connection flow. The Node auto-spawn path fixes Unix installs where `lionscraper.js` was resolved without a leading `/` (e.g. Glama/Docker). The Python package uses **aiohttp** for outbound HTTP/WebSocket to the daemon.\n\n## Two implementations\n\n| | **Node.js (npm)** | **Python (pip)** |\n|--|-------------------|------------------|\n| **Registry** | `io.github.dowant/lionscraper-node` | `io.github.dowant/lionscraper-python` |\n| **Docs (EN)** | [packages/node/README.md](packages/node/README.md) | [packages/python/README.md](packages/python/README.md) |\n| **Docs (ZH)** | [packages/node/README_cn.md](packages/node/README_cn.md) | [packages/python/README_cn.md](packages/python/README_cn.md) |\n\nInstall one or both; they are separate packages with the same CLI command names.\n\n### Install (npm)\n\nPublished as **[lionscraper](https://www.npmjs.com/package/lionscraper)** on npm.\n\n```bash\nnpm install -g lionscraper\n```\n\nWithout a global install, MCP can use **`npx`**; see the **npx** JSON examples under [Add MCP in your AI app](#add-mcp-in-your-ai-app).\n\n### Install (pip)\n\nPublished as **[lionscraper](https://pypi.org/project/lionscraper/)** on PyPI.\n\n```bash\npip install -U lionscraper\n```\n\nA **virtual environment** is recommended, or `pip install -U --user lionscraper` if you prefer not to install into the system interpreter.\n\n### Commands (both packages)\n\n| Command | Role |\n|--------|------|\n| **`lionscraper-mcp`** | Thin MCP server (stdio) for AI apps |\n| **`lionscraper`** | CLI: `daemon`, `stop`, `scrape`, `ping`, … (also serves the **HTTP API** on the same port) |\n\nAfter **`pip install -U lionscraper`**, if `lionscraper-mcp` is not on your `PATH`, use **`python -m lionscraper`** with **no extra arguments** for MCP stdio (see [packages/python/README.md](packages/python/README.md)).\n\n**`PORT`** (default **13808**) must match the extension **bridge port** in all modes.\n\n## CLI quick start\n\n```bash\nlionscraper daemon\nlionscraper ping\nlionscraper scrape -u https://www.example.com\n```\n\nFull flags, multiple URLs, pagination, and **HTTP API** details: [packages/node/README.md](packages/node/README.md) / [packages/python/README.md](packages/python/README.md).\n\n## Add MCP in your AI app\n\nExamples assume **`lionscraper-mcp`** is on your `PATH` (from npm or pip). In MCP JSON, every **`env` value is a string**.\n\n**Minimal config** (`PORT` defaults to **13808**; must match the extension bridge port):\n\n```json\n{\n  \"mcpServers\": {\n    \"lionscraper\": {\n      \"command\": \"lionscraper-mcp\"\n    }\n  }\n}\n```\n\n**Full `env` example** (omit keys you do not need):\n\n```json\n{\n  \"mcpServers\": {\n    \"lionscraper\": {\n      \"command\": \"lionscraper-mcp\",\n      \"env\": {\n        \"PORT\": \"13808\",\n        \"TIMEOUT\": \"120000\",\n        \"LANG\": \"en-US\",\n        \"TOKEN\": \"\",\n        \"DAEMON\": \"\"\n      }\n    }\n  }\n}\n```\n\n**npx (no global install)** — requires Node.js; the first run may download the package. The npm **package name** is `lionscraper`; the executable is `lionscraper-mcp`. Use `command` **`npx`** and pass **`lionscraper`** then **`lionscraper-mcp`** in `args` (after `-y`).\n\n**Minimal config (npx):**\n\n```json\n{\n  \"mcpServers\": {\n    \"lionscraper\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"lionscraper\", \"lionscraper-mcp\"]\n    }\n  }\n}\n```\n\n**Full `env` example (npx):**\n\n```json\n{\n  \"mcpServers\": {\n    \"lionscraper\": {\n      \"command\": \"npx\",\n      \"args\": [\"-y\", \"lionscraper\", \"lionscraper-mcp\"],\n      \"env\": {\n        \"PORT\": \"13808\",\n        \"TIMEOUT\": \"120000\",\n        \"LANG\": \"en-US\",\n        \"TOKEN\": \"\",\n        \"DAEMON\": \"\"\n      }\n    }\n  }\n}\n```\n\nTo pin a version, use e.g. `\"lionscraper@1.0.1\"` in place of `\"lionscraper\"` inside `args`.\n\n- **`PORT`**: HTTP + WebSocket listen port; default **13808**; must match the extension **bridge port**.\n- **`TIMEOUT`**: ms to wait for a previous instance to release the port; default **120000**; **`0`** forces takeover quickly.\n- **`LANG`**: tool descriptions and stderr language (**`en-US`**, **`zh-CN`**, or POSIX forms).\n- **`TOKEN`**: Bearer token shared with the daemon; empty means no auth.\n- **`DAEMON`**: only **`0`** disables auto-starting `lionscraper daemon` from thin MCP.\n\nRestart MCP or the host app after changing config.\n\n### Python: MCP via `python -m`\n\n```json\n{\n  \"mcpServers\": {\n    \"lionscraper\": {\n      \"command\": \"python\",\n      \"args\": [\"-m\", \"lionscraper\"]\n    }\n  }\n}\n```\n\nUse the same **`python`** you used to install the package (or `python3` on some systems).\n\n## Match the port in the browser extension\n\n1. Open LionScraper **settings / options**.\n2. Set **bridge port** to the same value as **`PORT`** (e.g. `13808`).\n3. If needed, use **Reconnect**, reload the extension, or restart the browser.\n\n## Day-to-day use\n\n1. Keep the extension **enabled** and target pages **open** as required.\n2. Ask in natural language (e.g. check connection, scrape lists / article / emails / phones / links / images).\n3. If you see “not connected” or timeouts, retry a connection check and confirm **PORT** matches.\n\n## FAQ\n\n**Extension not connected or scrape fails?**\n\n- Is the extension enabled?\n- Does **PORT** in the AI app match the extension **bridge port** exactly?\n- One bridge per machine is usually enough; duplicate MCP configs can conflict.\n\n**Seeing MCP tools in the client means everything works?**\n\nNot necessarily. Tools only prove **AI → bridge**; the extension must also register on the same port.\n\n## MCP Registry and directories\n\nOfficial MCP Registry entries (both use `server.json`):\n\n| Path | Registry name | Package |\n|------|----------------|---------|\n| [packages/node/server.json](packages/node/server.json) | `io.github.dowant/lionscraper-node` | [npm: lionscraper](https://www.npmjs.com/package/lionscraper) (`mcpName` in `package.json`) |\n| [packages/python/server.json](packages/python/server.json) | `io.github.dowant/lionscraper-python` | [PyPI: lionscraper](https://pypi.org/project/lionscraper/) (`mcp-name` comment in English `README.md`) |\n\n**Publish outline** (install the official CLI, see [Quickstart](https://modelcontextprotocol.io/registry/quickstart)):\n\n1. Publish **npm / PyPI** at the version in each `server.json`.\n2. In **`packages/node`**: `mcp-publisher login github`, then `mcp-publisher publish`.\n3. In **`packages/python`**: `mcp-publisher publish` (login reused).\n\nThird-party listings (e.g. [Glama](https://glama.ai/mcp/servers)) have their own rules; [Smithery](https://smithery.ai/docs/build/publish) targets public HTTPS/streaming setups rather than local stdio + npm/pip by default.\n\n## Third-party directory (Glama)\n\nThis project is listed on Glama (e.g. [LionScraper on Glama](https://glama.ai/mcp/servers/dowant/lionscraper-mcp)). If the page shows **cannot be installed** or **license not found**, typical fixes are: add a root **`LICENSE`** (this repo includes [LICENSE](LICENSE)), add **`glama.json`** with maintainer **GitHub usernames** for org-owned repos ([glama.json](glama.json)—edit `maintainers` if claim fails), **claim** the server on Glama, and optionally complete Glama’s **Docker / release** flow if you need their install and security/quality checks—official install remains **`npm install -g lionscraper`** and **`pip install -U lionscraper`**. See also the [score / checklist page](https://glama.ai/mcp/servers/dowant/lionscraper-mcp/score).\n\n## License\n\n[MIT](LICENSE) (same as the npm and PyPI packages).\n",
  "bytes": 9607,
  "sha": "4d01c7a422db769cb62dd3a255f0cb0b5f1b4e5faa146e5fa99f539994aa9980",
  "repo_slug": "dowant/lionscraper-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_dowant_lionscraper_python_b8b62e1b/readme"
}