AI-First Scraper
Three MCP tools: fetch_page, fetch_pages_batch, search_web. Ad-free Markdown for AI agents.
Open source Open in the app JSON README (API)
About
Three MCP tools: fetch_page, fetch_pages_batch, search_web. Ad-free Markdown for AI agents.
Details
- Kind
- MCP servers
- Topic
- Web search, scraping & browser
- Publisher
- yubinkim444
- Origin
- official
- Category
- ferramentas
- Transport
- local
- Version
- 1.0.2
- Stars
- 2
- Last push
- 2026-08-26T20:15:12Z
- Repository state
- ativo
- Language
- Python
- License
- MIT
- Added
- 2026-08-29 04:01:42
- Updated
- 2026-08-29 04:01:42
- Origin id
io.github.yubinkim444/ai-first-scraper-mcp
README
<!--
mcp-name: io.github.yubinkim444/ai-first-scraper-mcp
-->
# ai-first-scraper-mcp
> **Plug Claude Desktop, Cursor, or Cline straight into an ad-free web
> scraper + search engine.** Three tools, one line of config.
[](https://pypi.org/project/ai-first-scraper-mcp/)
[]()
[]()
[]()
---
## What it does
Adds three tools to any [MCP](https://modelcontextprotocol.io)-compatible
agent:
| Tool | What it does |
|------|--------------|
| `fetch_page` | Fetch one URL → return clean Markdown (HTML or PDF). |
| `fetch_pages_batch` | Fetch up to 25 URLs in parallel → return Markdown for each. |
| `search_web` | Run a web search and return the top-k result pages already converted to Markdown. |
No more "the model called curl and then tried to parse 80kB of ad HTML."
Your agent receives clean Markdown ready to reason about.
Backed by the [ai-first-scraper](https://github.com/yubinkim444/ai-first-scraper)
and [ai-first-search](https://github.com/yubinkim444/ai-first-search) APIs.
---
## Install
### Fastest — `uvx` (no install, runs from PyPI on demand)
```jsonc
// claude_desktop_config.json / cline_mcp_settings.json / ~/.cursor/mcp.json
{
"mcpServers": {
"ai-first-scraper": {
"command": "uvx",
"args": ["ai-first-scraper-mcp"]
}
}
}
```
Restart your client (Claude Desktop / Cursor / Cline). The three tools above
will appear automatically.
### Alternative — pip install
```bash
pip install ai-first-scraper-mcp
```
```jsonc
{
"mcpServers": {
"ai-first-scraper": {
"command": "ai-first-scraper-mcp"
}
}
}
```
---
## Where the config file lives
| Client | Config path |
|--------|-------------|
| **Claude Desktop (macOS)** | `~/Library/Application Support/Claude/claude_desktop_config.json` |
| **Claude Desktop (Windows)** | `%APPDATA%\Claude\claude_desktop_config.json` |
| **Cursor** | `~/.cursor/mcp.json` |
| **Cline (VS Code)** | `~/Library/Application Support/Code/User/globalStorage/saoudrizwan.claude-dev/settings/cline_mcp_settings.json` |
---
## Point at your own backend (optional)
By default this server calls the public `ai-first-scraper.onrender.com` and
`ai-first-search.onrender.com` instances. If you want to self-host, set env
vars in your MCP config:
```jsonc
{
"mcpServers": {
"ai-first-scraper": {
"command": "uvx",
"args": ["ai-first-scraper-mcp"],
"env": {
"SCRAPER_URL": "https://your-scraper.example.com",
"SEARCH_URL": "https://your-search.example.com",
"AFS_TIMEOUT": "60"
}
}
}
}
```
---
## Verify it works
Open your MCP client and ask the agent:
> "Use the search_web tool to find the top 3 recent articles about MCP and
> summarize them in 5 bullets each."
You should see the agent call `search_web`, get back Markdown for each result,
and produce the summary without ever touching raw HTML.
---
## Companion projects
- **[ai-first-scraper](https://github.com/yubinkim444/ai-first-scraper)** — the
per-URL Markdown cleaner this MCP server fans out to.
- **[ai-first-search](https://github.com/yubinkim444/ai-first-search)** —
search → scrape → markdown pipeline.
- **[mcp-rec](https://github.com/yubinkim444/mcp-rec)** — record & replay any MCP server's traffic for tests and bug reports.
- **[llm-cache-proxy](https://github.com/yubinkim444/llm-cache-proxy)** — local cache for OpenAI/Anthropic API calls.
- **[promptlocker](https://github.com/yubinkim444/promptlock)** — lockfile for prompts.
- **[context-diff](https://github.com/yubinkim444/context-diff)** — see what blew up your Claude Code context window.
- **[agentwatch](https://github.com/yubinkim444/agentwatch)** — overlay for browser AI agents.
---
## Develop locally
```bash
git clone https://github.com/yubinkim444/ai-first-scraper-mcp.git
cd ai-first-scraper-mcp
uv sync # or: pip install -e .
ai-first-scraper-mcp # speaks MCP over stdio
```
To test against a local client, point its MCP config at the same command.
---
## About the author
Built by [yubinkim444](https://github.com/yubinkim444), who also makes
**[Kay's Records](https://kay-s-record.web.app/get.html)** — an app for iOS
and Android.
If this project saved you time, giving the app a try is the nicest way to say
thanks.
## License
MIT © yubinkim444