{
  "markdown": "# freellmpool\n\n<!-- mcp-name: io.github.0xzr/freellmpool -->\n\n![freellmpool tokenmax terminal demo](assets/demo.svg)\n\n![178 enabled chat routes, 22 LLM providers cataloged, keyless start when available](assets/tokenmax-results.svg)\n\nfreellmpool catalogs 22 LLM providers as distinct groups spanning recurring\nfree tiers, keyless endpoints, finite trials, pin-only routes, and disabled\ncandidates. It exposes 178 enabled chat routes and 431 cataloged chat models,\nand automatically pools only\nenabled routes you can access behind one OpenAI-compatible endpoint — as a CLI,\na Python library, or a local proxy. It can start without credentials when an\nenabled keyless route is available.\n\n[![PyPI](https://img.shields.io/pypi/v/freellmpool.svg)](https://pypi.org/project/freellmpool/)\n[![CI](https://github.com/0xzr/freellmpool/actions/workflows/ci.yml/badge.svg)](https://github.com/0xzr/freellmpool/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Website](https://img.shields.io/badge/docs-0xzr.github.io%2Ffreellmpool-6ea8ff)](https://0xzr.github.io/freellmpool/)\n\n[FAQ](FAQ.md): where prompts go, ToS posture, failover, bans, and comparisons.\n\n## Release and distribution status\n\n- **Latest release: 0.13.0.** The GitHub release and PyPI package are both\n  0.13.0; `pip install freellmpool` and `uvx freellmpool` install the audited\n  provider catalog, bounded streaming and sentinel hardening, Hermes profile,\n  proxy readiness/provider APIs, `spread` routing, and OpenCode\n  registry-readiness hardening.\n\n- **Registry publication status: pending.** `opencode-freellmpool` and\n  `opencode-freellmpool-tui` are tested but not published on npm as of\n  2026-08-29. Use their repository-local installation instructions for now.\n\n## 30-second quickstart\n\nFresh install to first free-model reply is measured at about 19 seconds under\nthe 30-second target on a clean Linux/Python 3.12 environment, with no API keys\nwhen a keyless provider is up:\n\n```bash\n# One command when uv is installed; no checkout or manually managed virtualenv:\nuvx freellmpool ask --max-tokens 32 \"Reply with one short sentence: freellmpool is ready.\"\n```\n\nPortable virtual-environment path:\n\n```bash\npython3 -m venv .venv\n. .venv/bin/activate\npython -m pip install --upgrade pip\npython -m pip install freellmpool\nfreellmpool ask --max-tokens 32 \"Reply with one short sentence: freellmpool is ready.\"\n```\n\nCI runs the same path from this checkout with\n`FREELLMPOOL_QUICKSTART_PACKAGE=. scripts/quickstart-test.sh`.\n\nThe catalog covers provider-operated free tiers, keyless routes, and a few\nexplicit finite-trial or disabled candidates. Eligibility and limits differ by\nprovider. freellmpool automatically uses only enabled routes you can access,\nfails over when one is rate limited or down, and tracks local per-day usage.\n\nSeveral providers (Pollinations, OVHcloud, and Kilo Gateway) need no API key,\nand LLM7 works without one, so the quickstart can answer without signup when a\nkeyless provider is available.\n\nTo inspect your local provider keys, agent CLIs, proxy config, and Tailscale\nstate before wiring tools, run the print-only init wizard:\n\n```bash\nfreellmpool init --yes\nfreellmpool init --yes --agent opencode\nfreellmpool init --yes --agent metaswarm --tailnet\n```\n\nAdd keys for the other providers to unlock more models and higher limits.\n\n### Local runtimes (explicit, pin-only)\n\nLM Studio, Ollama, and llama.cpp can be previewed from a fixed list of literal\nloopback endpoints without scanning your LAN or processes:\n\n```bash\nfreellmpool local discover\nfreellmpool local discover --name lm_studio\nfreellmpool local import --name lm_studio --yes\nfreellmpool ask --providers local_lm_studio --model \"<model-from-discover>\" \"hello\"\n```\n\nDiscovery is read-only. Import is a separate affirmative step, writes no\ncredential, and keeps every imported model out of automatic routing\n(`auto = false`). `freellmpool local remove local_lm_studio --yes` reverses only\nthe block managed by the importer. Custom endpoints must be canonical literal\nloopback URLs such as `http://127.0.0.1:1234/v1`; hostnames, LAN addresses,\nredirects, and broad network scans are rejected.\n\n## First-run setup with `freellmpool init`\n\n`freellmpool init` inspects provider keys, installed agent CLIs, Tailscale\nstate, and proxy config, then prints one copy-pastable next step without editing\nfiles. Run it detect-only first:\n\n```bash\nfreellmpool init --yes\n```\n\n`--json` emits the same detection as versioned JSON for scripts and agents.\n\n### Tailnet / remote agent gateway\n\nServe the proxy on your Tailscale 100.x address with a generated API key:\n\n```bash\nfreellmpool tailnet serve --port 8080\n```\n\nFrom a remote machine:\n\n```bash\nfreellmpool tailnet connect <tailnet-ip> --port 8080\n```\n\nBoth sides support `--api-key <shared-secret>` if you want to pin a key instead\nof using a generated token. Tailnet serving requires auth by default; do not\nrun unauthenticated over non-loopback interfaces.\n\n### Metaswarm agent lanes\n\nThis project uses one Umans/Kimi K2.7 worker lane, one MiniMax M3 lane, Codex as\nescalation, and Claude Opus only for final pre-ship review. The installable\nMetaswarm profile mirrors that posture: one free/cheap worker lane through the\nlocal proxy, one larger freellmpool reviewer lane, and Codex/Opus as explicit\nuser-owned paid escalation/final-review lanes only (never silent).\n\n```bash\nfreellmpool init --yes --agent metaswarm --tailnet\nfreellmpool profile install metaswarm\nfreellmpool tailnet serve --port 8080\nfreellmpool profile doctor metaswarm --dry-run\n```\n\n## Run a coding agent on free models\n\nfreellmpool's proxy speaks the OpenAI API and includes an experimental\nAnthropic-compatible path, so coding agents can run against pooled free tiers —\njust point them at the proxy:\n\n```bash\nfreellmpool proxy                       # starts http://localhost:8080\nfreellmpool code claude                 # prints the one-line setup for Claude Code\nfreellmpool profile list                # richer installable profiles\nfreellmpool profile show metaswarm      # Tailnet-aware Metaswarm profile\nfreellmpool profile install hermes       # Hermes custom-endpoint config\n# (also: codex, aider, cline, continue, cursor, hermes, opencode, metaswarm)\n```\n\nThe Hermes profile prints (and never writes) this supported custom-endpoint\nblock; `hermes model` provides the interactive equivalent:\n\n```yaml\nmodel:\n  provider: custom\n  default: quality\n  base_url: http://localhost:8080/v1\n  api_key: anything\n```\n\nClaude Code gateway mode can also be launched directly:\n\n```bash\nANTHROPIC_BASE_URL=http://localhost:8080 \\\nANTHROPIC_AUTH_TOKEN=dummy \\\nANTHROPIC_API_KEY=dummy \\\nANTHROPIC_MODEL=auto \\\nANTHROPIC_SMALL_FAST_MODEL=auto \\\nCLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1 \\\nclaude\n```\n\nExisting OpenAI-compatible apps work the same way: set\n`OPENAI_BASE_URL=http://localhost:8080/v1` and keep your code unchanged.\nAnthropic-compatible tools can use the experimental bridge with\n`ANTHROPIC_BASE_URL=http://localhost:8080`.\n\nText-only `/v1/responses` and `/v1/messages` requests stream incrementally from\nthe selected provider. Tool calls and richer content intentionally stay on the\nbuffered compatibility path. The browser dashboard and playground share one\npublic, data-free shell; when proxy auth is enabled it prompts for the bearer\ntoken and keeps it only in page memory while protected status, inventory,\nmodel, and battle calls continue to require the `Authorization` header.\n\n**OpenCode** gets a deeper integration in 0.12.0: a live in-editor **dashboard** (routing mode,\nestimated savings, tokens served free, provider race, latency), per-request\n**agent routing** via the model picker (`freellmpool/agent|spread|auto|fast|quality|fair`), and `freellmpool_status`\n/ `freellmpool_models` tools — see [integrations/opencode-tui](integrations/opencode-tui)\nand the [guide](https://0xzr.github.io/freellmpool/run-opencode-on-free-models.html).\nThe plugin registers its routing aliases automatically on supported OpenCode\nversions without rewriting user configuration. Restart OpenCode and check\n`opencode models freellmpool`.\nThe package tarballs are validated in CI, but npm publication remains pending;\nthe linked local-file instructions remain the working install path.\n\n**New in 0.11:** capacity tools — `freellmpool capacity status` shows which free\ntiers are usable right now, `freellmpool providers health` live-probes them, and\n`freellmpool keys add` walks you through configuring more (see\n[Capacity & provider health](#capacity--provider-health) and\n[docs/CAPACITY.md](docs/CAPACITY.md)).\n\n**New in 0.10:** an async API (`AsyncPool`), an MCP server (`freellmpool mcp`),\nlatency-aware routing with `freellmpool benchmark`, observability hooks, and a\nplugin system for custom providers. See the [changelog](CHANGELOG.md).\n\n## Install\n\n```bash\npip install freellmpool      # or: pipx install freellmpool\n```\n\nOnly dependency is `httpx`. Python 3.11+.\n\n### Docker and Open WebUI\n\nThe published container runs the local proxy as an unprivileged user. Persist\nquota, stats, route-health, cache, and configuration state in a named volume:\n\n```bash\ndocker volume create freellmpool-data\ndocker run --rm -p 127.0.0.1:8080:8080 \\\n  --volume freellmpool-data:/home/freellmpool/.config/freellmpool \\\n  --env GROQ_API_KEY \\\n  ghcr.io/0xzr/freellmpool:0.13.0\n```\n\nOmit `--env GROQ_API_KEY` if you want a credential-free start; the proxy can\nanswer only while an enabled keyless route is available. Keep the published\nport on loopback. If you deliberately expose it, set\n`FREELLMPOOL_PROXY_KEY` and require that Bearer token from clients.\n\nFor the bundled Open WebUI stack, place any provider credentials in `.env` and\nrun `docker compose up -d`. Compose waits for the proxy health check and keeps\nboth freellmpool and Open WebUI state in named volumes; open\n`http://localhost:3000` after both services are healthy.\n\n## Command line\n\n```bash\nfreellmpool ask \"Write a haiku about sqlite\"\ngit diff | freellmpool ask \"Write a commit message for this\"\nfreellmpool tokenmax \"Hardest question you've got\"  # 🌈 blast models, print answers, optional synthesis\nfreellmpool providers        # which providers are configured\nfreellmpool models           # cataloged provider/model ids\nfreellmpool stats            # lifetime tokens served free + estimated cost avoided\nfreellmpool badge -o badge.svg   # a shareable SVG badge of that total\nfreellmpool badge --summary -o summary.svg   # a larger usage summary card\n```\n\n`freellmpool tokenmax` is the tongue-in-cheek maximum-effort mode: it fans your\nprompt out to many available models at once and prints each answer. The CLI adds\na synthesized verdict by default unless you pass `--no-synthesize`; the MCP tool\nreturns the model answers for the calling agent to synthesize. (See\n[docs/MCP.md](docs/MCP.md).)\n\n`freellmpool stats` is a running, **persistent** lifetime total (it survives restarts\nand upgrades). Embed `freellmpool badge` in a README, or use\n`freellmpool badge --summary -o summary.svg` for a larger card with tokens,\nrequests, estimated savings, and provider mix. The proxy can serve both\n`/badge.svg` and `/summary.svg` live when `FREELLMPOOL_PUBLIC_BADGE=1` makes\npublic badges embeddable.\n\nPin a provider or model; common OpenAI/Anthropic model names are mapped to a free\nequivalent so existing scripts keep working:\n\n```bash\nfreellmpool ask -m groq/openai/gpt-oss-20b \"hi\"\nfreellmpool ask -p cerebras,groq \"hi\"\nfreellmpool ask -m gpt-4o-mini \"hi\"      # routed to a free model\n```\n\n### Roles\n\n`freellmpool roles` lists ask-role presets (`coder`, `critic`, `summarizer`,\n`grounded-reader`, `long-context`, `cheap`, `fast`, `second-opinion`, ...). Each\nrole sets routing, token budget, temperature, task intent, and system-prompt hints\nwithout inventing a second routing engine. Explicit flags (`--model`, `--providers`,\n`--routing`, `--task`, `--max-tokens`) win over role defaults.\n\n```bash\nfreellmpool ask --role coder \"write a pytest for this function\"\nfreellmpool ask --role grounded-reader \"read this Markdown file\"\nfreellmpool ask --routing quality --task general \"ignore automatic task classification\"\nFREELLMPOOL_MODE=wise freellmpool ask --role cheap \"summarize this patch\"\n```\n\n## As a proxy\n\nRun a local server that speaks the OpenAI API, then point any OpenAI-compatible\ntool at it. On loopback, any placeholder API key works unless you configured\n`FREELLMPOOL_PROXY_KEY` or passed `--api-key`; Tailnet/LAN serving requires a\nreal proxy bearer token by default.\n\n```bash\nfreellmpool proxy\nexport OPENAI_BASE_URL=http://localhost:8080/v1\nexport OPENAI_API_KEY=unused\n```\n\n```python\nfrom openai import OpenAI\nclient = OpenAI()\nprint(client.chat.completions.create(\n    model=\"auto\",\n    messages=[{\"role\": \"user\", \"content\": \"hi\"}],\n).choices[0].message.content)\n\n# audio → text (Whisper), same client:\nprint(client.audio.transcriptions.create(\n    model=\"auto\", file=open(\"audio.mp3\", \"rb\"),\n).text)\n```\n\nOr with `curl` (multipart upload):\n\n```bash\ncurl -s http://localhost:8080/v1/audio/transcriptions \\\n  -F file=@audio.mp3 -F model=auto\n```\n\nThe proxy also implements the OpenAI Responses API (for the Codex CLI) and an\nexperimental Anthropic Messages API path (for Claude Code), so coding agents can\nrun on free models too. `freellmpool code <agent>` prints the exact setup, while\n`freellmpool profile install <agent>` prints the fuller copy-pastable profile\nwithout mutating third-party config:\n\n```bash\nfreellmpool code aider       # also: claude, codex, cline, continue, cursor, hermes, opencode\nfreellmpool profile show opencode\nfreellmpool profile doctor opencode --dry-run\n```\n\nRelease 0.12.0 proxy surfaces:\n\n- `/v1/chat/completions` — OpenAI-compatible chat, token streaming, tool calling.\n- `/v1/responses` — Responses API shim with incremental text streaming; tools\n  and rich content use the buffered compatibility path.\n- `/v1/messages` — experimental Anthropic-compatible Messages path with the\n  same incremental-text / buffered-tool split.\n- `/v1/embeddings` and `/v1/audio/transcriptions` — OpenAI-compatible embedding\n  and Whisper-style multipart transcription.\n- `/v1/models` — routing aliases plus concrete `provider/model` ids.\n- `/v1/models?ready=true` — the same shape, limited to locally ready targets.\n- `/v1/providers` — authenticated, secret-free provider/model readiness details.\n- `/freellmpool/battle` and `/playground` — bounded browser/JSON model comparisons.\n- `/healthz` and `/livez` — public process liveness aliases.\n- `/readyz` — public advisory local-capacity readiness (200 when at least one\n  provider is ready, otherwise 503); it never live-probes an upstream.\n- `/dashboard`, `/status`, `/badge.svg`, and `/summary.svg` — local operations surfaces.\n\n`/`, `/dashboard`, and `/playground` serve one public, data-free shell. When a\nproxy key is set, the shell prompts for it and keeps it only in page memory;\nstatus, provider/model inventory, battle, and model API calls remain header-auth\nprotected. Liveness/readiness stay public for orchestrator probes; readiness is\na local quota/cooldown snapshot, not an upstream health guarantee.\nSetup snippets for specific tools are in [docs/INTEGRATIONS.md](docs/INTEGRATIONS.md)\nand [docs/AGENTS.md](docs/AGENTS.md). The repo also includes an experimental\n[metaswarm review adapter](integrations/metaswarm) for using `freellmpool` as an\nexternal-tools reviewer/second opinion. `freellmpool profile show metaswarm`\ndocuments a free/cheap worker lane, a larger reviewer lane, Tailnet client setup,\nand paid Codex/Opus lanes as explicit user-owned escalation paths only.\n\n## As a library\n\n```python\nfrom freellmpool import Pool\n\npool = Pool.from_default_config()\nreply = pool.ask(\"Summarize the plot of Hamlet in 20 words.\")\nprint(reply.text, \"—\", reply.provider_id)\n\nvectors = pool.embed([\"first document\", \"second document\"]).vectors\n\nwith open(\"audio.mp3\", \"rb\") as f:\n    text = pool.transcribe(f.read(), \"audio.mp3\").text   # Whisper, failover across providers\n```\n\nAsync is the same API with `await`:\n\n```python\nfrom freellmpool import AsyncPool\n\nasync with AsyncPool.from_default_config() as pool:\n    reply = await pool.aask(\"Summarize the plot of Hamlet in 20 words.\")\n```\n\nPass `on_event=...` to either pool to receive structured routing/cache events\n(`attempt`/`success`/`error`/`cooldown`/`cache_hit`/`cache_miss`/`exhausted`) for logging or tracing. Add\nyour own endpoint with `register_provider(...)`, or a new request shape with\n`register_adapter(name, fn)`.\n\n## Benchmark your providers\n\n`freellmpool benchmark` times one call per configured provider and prints\nlatency and success, so you can see which of your free tiers are fastest right\nnow. The router learns the same latency/success signal from real traffic as it\nruns; set `FREELLMPOOL_ROUTING=fast` to prefer the lowest-latency provider\ninstead of the default least-used-first.\n\n```\n$ freellmpool benchmark\n  provider/model            status   latency  note\n  cerebras/gpt-oss-120b     ok        180 ms  6 tok\n  groq/openai/gpt-oss-20b   ok        240 ms  6 tok\n  ovh/Meta-Llama-3_3-70B-Instruct  FAIL    -  HTTP 429\n```\n\n## Capacity & provider health\n\nFree tiers drift through the day — keys expire, providers go down, daily caps\nfill. These commands tell you what's usable right now and what to set up next:\n\n```bash\nfreellmpool capacity status --target 5   # who's healthy / near quota / missing a key\nfreellmpool capacity status --refresh    # explicitly refresh advisory provider metadata\nfreellmpool quota-wise status            # local headroom + recommended mode\nfreellmpool providers health             # send one tiny request to each, time it\nfreellmpool keys checklist --target 5    # which keys to add to reach N healthy providers\nfreellmpool keys add groq                # configure a key (and record metadata)\n```\n\nProtocol support is verified separately from basic provider health. Run\n`freellmpool conformance run` to check chat, streaming, tools, JSON object/schema,\nvision, Responses, and Anthropic Messages with bounded synthetic requests, then inspect\nthe sanitized evidence with `freellmpool conformance status --json`.\nFeature-specific auto-routing uses only verified targets; an exact\nprovider/model pin is the explicit override. Full operator and privacy contract:\n[docs/PROTOCOL_CONFORMANCE.md](docs/PROTOCOL_CONFORMANCE.md).\n\n`capacity status` is offline and cache-first: it reads your catalog, environment, and\nper-day quota counters and labels each provider `healthy`, `low_quota`,\n`exhausted`, `invalid_key`, or `missing`. Cached metadata from the advisory\nexternal catalog ([mnfst/awesome-free-llm-apis](https://github.com/mnfst/awesome-free-llm-apis))\ncan suggest free providers you could add; only an explicit\n`freellmpool capacity status --refresh` or `freellmpool catalog sync` uses the\nnetwork for that metadata. It remains advisory only; your `providers.toml`\nstays the source of truth for routing. `keys add <name>` can import a\nsuggested provider from that catalog or create an OpenAI-compatible stub and\nautodiscover its models. The proxy `/dashboard` shows the same capacity at a\nglance. Full reference: [docs/CAPACITY.md](docs/CAPACITY.md).\n\n`FREELLMPOOL_MODE=wise` is the conservative quota mode: `ask` defaults to a\nsmaller output budget and spread routing, `tokenmax` narrows its default fan-out,\nand broad multi-model calls require confirmation unless you pass `--yes`.\nPer-command `--mode normal|wise` overrides the environment, and\n`[settings] mode = \"wise\"` works from `config.toml`. The `conserve` role is a\nquota-conscious shorthand for small, spread-routed answers.\n\nFor a bounded second opinion instead of a full `tokenmax` blast:\n\n```bash\nfreellmpool ask --second-opinion --opinions 3 \"is this implementation plan sound?\"\nfreellmpool ask --role second-opinion --synthesize \"which release note is clearer?\"\n```\n\nThe shared panel asks a few diverse providers, keeps individual failures visible,\nand can append a non-fatal synthesis when you pass `--synthesize`.\n\nFor a side-by-side comparison you can inspect in the terminal or local browser:\n\n```bash\nfreellmpool battle \"which changelog entry is clearer?\" --synthesize\nfreellmpool proxy --port 8080\nfreellmpool playground --port 8080\n```\n\nBundled recipes wrap common workflows in JSON files you can inspect and run:\n\n```bash\nfreellmpool recipe list\nfreellmpool recipe run second-opinion \"is this launch plan clear?\" --synthesize\nfreellmpool recipe run pr-review --input patch.diff\nfreellmpool recipe run repo-summary --path 'src/freellmpool/*.py'\nfreellmpool recipe run metaswarm-worker-review --input worker.md --validation-output-file validation.txt\n```\n\nRecipes use the same role presets and shared panel helper as `ask` and `battle`;\nthere is no separate routing engine.\n\n### Local foreground job queue\n\nFor slow, quota-aware work that should not block a live session, queue jobs to\nan append-only JSONL log under your config dir (override with\n`FREELLMPOOL_JOBS_PATH`). The queue is foreground-only: `jobs run` processes\none job at a time and records started/completed/failed/cancelled events.\nCompleted ask jobs keep their output in the job log; completed recipe jobs also\nwrite run records and Markdown reports via the same report helpers used by\n`freellmpool report`.\n\n```bash\n# queue a recipe job\nfreellmpool jobs add --recipe pr-review --input patch.diff\n\n# queue an ask job with a role preset\nfreellmpool jobs add --role summarizer \"summarize the latest changelog\"\n\nfreellmpool jobs list            # replayed state (idempotent across restarts)\nfreellmpool jobs watch           # one-shot refresh render, no daemon\n\nfreellmpool jobs run --dry-run   # print execution order, mutate nothing\nfreellmpool jobs run --max-failures 2   # halt after N consecutive failures\nfreellmpool jobs cancel <job-id> # append a cancel tombstone, not a mutation\n\nfreellmpool report list\nfreellmpool report last --markdown\nfreellmpool report last --html --path\nfreellmpool cost show <run-id>\n```\n\nCancellation is a new tombstone event, not a re-write of the earlier queued\nrecord — a crash before `jobs run` finishes still leaves the queue\nreplayable, and cancelled jobs stay cancelled after restart. Duplicate\nsubmissions create distinct jobs; pass `--dedupe` to reject re-submission of\nthe same recipe or role while a job is still pending.\n\n## As an MCP server\n\n`freellmpool mcp` runs a Model Context Protocol server over newline-delimited\nJSON-RPC stdio, so Claude Desktop, Claude Code, or Cursor can hand subtasks to\nfree models. It exposes ask, panel/second-opinion, battle, recipe, roles,\nTailnet-info, quota-wise, route-preview, models, quota, stats, and `tokenmax`\ntools. See [docs/MCP.md](docs/MCP.md). A [`server.json`](server.json) is included\nfor the [MCP registry](https://registry.modelcontextprotocol.io/).\n\n## In Simon Willison's `llm` CLI\n\nThere's a plugin: `llm install llm-freellmpool` → `llm -m freellmpool \"...\"`.\nIt can answer without an API key while an enabled keyless route is available.\nSource: [plugins/llm-freellmpool](plugins/llm-freellmpool).\n\n## Provider keys\n\nfreellmpool reads keys from the environment and uses whatever is set. No key is\nrequired while an enabled keyless route is available; credentials expand the\nset of accessible routes. Step-by-step signup links, including verification and\npayment-method caveats, are in\n[docs/ACCOUNTS.md](docs/ACCOUNTS.md).\n\n| Provider | Env var | Notes |\n|---|---|---|\n| Pollinations | — | no key needed |\n| OVHcloud | — | no key needed (anonymous tier) |\n| Kilo Gateway | — | no key needed |\n| LLM7 | `LLM7_API_KEY` | optional |\n| Groq | `GROQ_API_KEY` | current free-plan routes; exact limits vary by model |\n| Cerebras | `CEREBRAS_API_KEY` | finite $5 trial; current routes are explicit pins, not automatic recurring capacity |\n| NVIDIA NIM | `NVIDIA_API_KEY` | |\n| OpenRouter | `OPENROUTER_API_KEY` | free models |\n| Google Gemini | `GEMINI_API_KEY` | |\n| Cloudflare | `CLOUDFLARE_API_TOKEN` + `CLOUDFLARE_ACCOUNT_ID` | |\n| Hugging Face router | `HF_TOKEN` | router free tier |\n| OpenCode Zen | — | cataloged, disabled by default pending opt-in |\n| Aion Labs | `AION_API_KEY` | 20K free tokens/day, no card |\n| ModelScope API Inference | `MODELSCOPE_API_KEY` | 2,000 free calls/day |\n| Morph | `MORPH_API_KEY` | current priced aliases are retained disabled for explicit future verification |\n| Vercel AI Gateway | `AI_GATEWAY_API_KEY` | automatic routing is limited to the publicly price-verified zero-price Poolside route; other routes are pin-only or disabled; set a gateway-side budget |\n| SiliconFlow | `SILICONFLOW_API_KEY` | free models; identity verification required |\n| Mistral, Cohere, SambaNova, Z.ai, Ollama Cloud | see `.env.example` | |\n\nA `config.toml` (see [config.toml.example](config.toml.example)) can hold keys,\nmodel aliases, and settings instead of env vars.\n\n## Local diagnostics and operations\n\nRun `freellmpool doctor` for a no-network local check of package version, config\npaths, configured provider count, routing mode, quota/cache locations, external\ncatalog cache age, and bundled catalog validity.\n\nResponse caching is off unless `FREELLMPOOL_CACHE_TTL` (seconds) or\n`[settings] cache_ttl` is positive. When enabled, cache rows live in SQLite with\nWAL mode and TTL pruning; `FREELLMPOOL_CACHE_MAX_ENTRIES` caps retained rows\n(default `10000`, set `0` to disable size pruning).\n\nQuota counters are written immediately by default. Long-running proxy/MCP\nprocesses can reduce file churn with `FREELLMPOOL_QUOTA_FLUSH_EVERY=N`, which\nbatches up to `N` successful requests before flushing. Shutdown paths and\n`quota.snapshot()` flush pending counts, so dashboards and process exits still\nsee current totals.\n\nRecent per-route health is persisted at\n`~/.config/freellmpool/route_health.json` (override with\n`FREELLMPOOL_HEALTH_FILE`). The bounded, atomic state contains route names,\ntimings, counters, and normalized failure classes only—never prompts, responses,\nheaders, or credentials. `/status` reports circuit state, sample age, and reset\ntime for each enabled route.\n\n## How routing works\n\nFor each request, freellmpool builds the list of `(provider, model)` pairs you\nhave access to, then orders providers least-used-first and picks a least-used\nmodel inside that provider. This keeps providers with large catalogs, like\nNVIDIA, from receiving more traffic only because they expose more models. A\nprovider that returns a 429 is set aside for its advertised `Retry-After` or\nrate-limit reset window. Repeated availability failures open a per-route circuit;\nafter cooldown, one half-open probe can restore the route. Daily counts are kept\nin `~/.config/freellmpool/quota.json` and reset at UTC midnight.\n\nEvery call records latency and success per model target. A provider whose targets\nare currently failing sinks to the back automatically; with\n`FREELLMPOOL_ROUTING=fast` the fastest measured provider goes first instead.\n`FREELLMPOOL_ROUTING=fair` spreads requests across providers to preserve daily\nquota. `freellmpool benchmark` warms these metrics on demand. To restore the old\nper-model balancing behavior, set `FREELLMPOOL_ROUTING=legacy` or\n`FREELLMPOOL_ROUTING=model` (or `FREELLMPOOL_ROUTING=model-fast` for the old\nper-model fastest-first ordering).\n\n**Quality routing (`FREELLMPOOL_ROUTING=quality`).** Free tiers' strongest models\nhave the smallest daily caps, so a naive pool gets weaker as the day fills. Quality\nrouting matches each prompt's *difficulty* to each model's *capability*: hard\nprompts (long input, code, reasoning cues) go to the strongest available model, and\neasy ones go to lightweight models — which rations scarce strong-model quota so the\npool stays sharp for longer. It also recognizes high-confidence grounded Markdown\nreading/extraction requests locally. When repeated evidence from the versioned,\nsanitized fixture exists for a candidate's exact model identity, a bounded task-fit\nterm influences quality ordering. Unmeasured models remain reachable, and if no\ncandidate has current evidence the ordering is unchanged. Capability is grounded in\nreal benchmark data, not guessed from names; models that no benchmark lists cover\nfall back to a name heuristic.\n\nUse `--task grounded-reading` to declare that intent, `--task general` to suppress\nautomatic classification, or `--task auto` to classify locally. OpenAI-compatible,\nResponses, and Anthropic proxy clients can send the body extension\n`\"task\": \"grounded-reading\"` or the `X-Freellmpool-Task` header. Explicit intent\nwins over automatic classification. Task evidence stores aggregate pass counts,\nfixture hashes, and scores only—never prompts, documents, responses, or provider\nsecrets.\n\nThe bundled, offline scores come from [LMArena](https://lmarena.ai/) Elo (an\nMIT-licensed snapshot) and the [Aider](https://aider.chat/) code-editing\nleaderboard (Apache-2.0), normalized to a common percentile scale. For much\nbroader coverage, run `freellmpool capability sync` with a free\n[Artificial Analysis](https://artificialanalysis.ai/) API key\n(`FREELLMPOOL_AA_API_KEY`) — its Intelligence Index covers most current and\nopen-weight models and takes precedence. The fetched AA data is cached locally\nunder your own key (never bundled, per AA's terms). `freellmpool capability\nstatus` shows current coverage. Scores via LMArena and Aider; intelligence index\nvia Artificial Analysis when keyed.\n\n**Context windows.** Free models often have small context windows. freellmpool\nnever truncates your input; instead, when a model rejects a request as too long,\nit learns that model's limit and stops routing oversized requests there, escalating\nonly to larger-window models. If nothing fits it raises a clear\n`ContextWindowExceeded` (with the estimated input size) instead of a generic\nfailure — over the proxy that's a `413`. You can declare a model's window with\n`context = N` in `providers.toml` to skip it proactively.\n\nArchitecture notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).\n\n## Limitations\n\n- Free-tier models are smaller than frontier models. They're good for drafting,\n  summarizing, classification, triage, and everyday coding — not a replacement\n  for GPT-class reasoning on hard problems.\n- Quality and capacity vary through the day as high-cap tiers exhaust.\n  freellmpool's local per-day counters reset at UTC midnight; upstream\n  providers use their own limit and reset windows.\n- Free tiers change without notice. When a model id or limit goes stale, a\n  one-line PR to `providers.toml` fixes it for everyone.\n- The proxy is meant for local/single-user use. It binds to `127.0.0.1` by\n  default; if you expose it, set a key (`--api-key`).\n- The Claude Code / Anthropic path is experimental (text and tool use; no vision).\n- These are free tiers shared by everyone — don't abuse them.\n- Upstream providers receive and may prompt-title/moderate your requests; use\n  the FAQ privacy table and each provider's current terms before sending\n  sensitive prompts.\n- Free-tier availability, model IDs, and rate limits drift without notice.\n  freellmpool does not bypass provider limits, rotate accounts, or evade\n  quotas.\n\n## How it compares\n\nComparison snapshot: 2026-07-19. Competitor rows link to the exact README commit\nused for the claims; scopes change quickly, so treat this as a dated map rather\nthan a permanent ranking.\n\n| Tool | Keyless start | # providers | Failover | MCP server | CLI | Transcription | Local/self-hosted | License |\n|---|---|---:|---|---|---|---|---|---|\n| **freellmpool** | Yes, when a configured keyless provider is available | 22 chat providers cataloged locally | Yes: retryable provider failures, empty replies, and transport errors | Yes: `freellmpool mcp` | One-shot CLI plus profiles, library, and proxy commands | Yes: `/v1/audio/transcriptions` with failover | Yes: small Python package and local proxy | MIT |\n| [OpenRouter free models](https://openrouter.ai/openrouter/free/providers) | No: hosted API use requires an account/key | Hosted router; the linked free router listed 22 models at snapshot | [Yes](https://openrouter.ai/docs/guides/routing/model-fallbacks): hosted provider/model fallbacks | [Yes](https://openrouter.ai/blog/announcements/openrouter-mcp-server/): hosted remote MCP server | Hosted API/SDK and agent SDK, not a local gateway CLI | [Audio input/transcription](https://openrouter.ai/docs/guides/overview/multimodal/audio) through multimodal chat | No: hosted service | Proprietary service |\n| [LiteLLM](https://github.com/BerriAI/litellm/blob/5d4c4d0fce45c73c4b56b48e46dfc4e56e8b0aa5/README.md) | No: bring provider or hosted-gateway credentials | README claims 100+ LLMs/providers | Yes: router retries/fallbacks | Yes: AI Gateway includes an MCP Gateway | Python SDK and proxy/gateway CLI surface | Yes: `/audio/transcriptions` | Yes: self-hosted proxy or hosted offering | MIT core; commercial enterprise features |\n| [OmniRoute](https://github.com/diegosouzapw/OmniRoute/blob/d8ff51874c8add566d43225988b9bc67e0542d65/README.md) | Yes: its setup documents a no-auth OpenCode option | README claims 268 integrations/providers and 90+ free options | Yes: layered fallback/circuit-breaker routing | Yes: MCP and A2A control planes | Broad management CLI and one-command agent setup | Audio translation is documented; other media capabilities vary by provider | Yes: Node application with dashboard, Docker, desktop/PWA paths | MIT |\n| [FreeLLMAPI](https://github.com/tashfeenahmed/freellmapi/blob/759de8e7ed1edc1cd513c9777cd0a807fb5ceee3/README.md) | Server starts with a Docker one-liner; inference capacity is configured afterward | README claims **28 free LLM providers** and 339 endpoints | Yes: provider/key routing and fallback | Yes: **MCP (Streamable HTTP)** | Dashboard/server and desktop apps | Not documented; audio speech/TTS is supported | Yes: Node/Docker server and desktop apps | MIT |\n\nFreeLLMAPI predates this project, and the overlap is independent convergence\naround legitimate free tiers. OmniRoute\noptimizes for an expansive control plane and many protocols; FreeLLMAPI for a\ndashboard-centric self-hosted router with encrypted key management; freellmpool\nfor the smallest Python/CLI/library path with a keyless-capable first reply.\nThose are product-scope choices, not a claim that one tool is best for every\ndeployment.\n\n## FAQ\n\n**Is there a free, OpenAI-compatible LLM API gateway?** Yes — freellmpool is a\nfree, MIT-licensed gateway that exposes one OpenAI-compatible endpoint over the\nenabled routes you can access. Its 22 cataloged provider groups span recurring\nfree tiers, keyless endpoints, finite trials, pin-only routes, and disabled\ncandidates. `pip install freellmpool` and point any OpenAI client at the local\nproxy.\n\n**How do I use multiple free LLM APIs at once?** freellmpool pools them: each request\ngoes to a provider you have access to, fails over to the next when one is rate-limited\nor down, and tracks per-day usage so load spreads across tiers.\n\n**Can I run Claude Code or Codex on free models?** Yes — the proxy speaks the\nOpenAI API and has an experimental Anthropic-compatible path. Set\n`OPENAI_BASE_URL=http://localhost:8080/v1` for OpenAI-compatible tools or\n`ANTHROPIC_BASE_URL=http://localhost:8080` for Anthropic-compatible tools, then\nrun Codex, Claude Code, aider, Cline, Continue, or Cursor against pooled free tiers. For Claude Code, set\n`CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=1` so `/v1/models` is discovered\nthrough the Anthropic bridge. See `freellmpool code <agent>`. (Claude Code path is\nexperimental: text + tools, no vision.)\n\n**Do I need an API key?** Not while an enabled keyless route is available:\nPollinations, OVHcloud, and Kilo Gateway expose keyless routes, and LLM7 is\nkey-optional. Add applicable free-tier or trial credentials for more routes and\ncapacity; availability and terms remain provider-specific.\n\n**Is it free and open source?** Yes, MIT-licensed. More at the\n[project page](https://0xzr.github.io/freellmpool/).\n\n## Featured in\n\n- Community videos (Spanish, by lytohlg AI): [\"Accede a 18 modelos de IA GRATIS con 1 solo comando\"](https://www.youtube.com/watch?v=1UfIlWoedho) and [\"Prueba 18 IAs GRATIS sin API key en 30 segundos\"](https://www.youtube.com/watch?v=oaM_E92WVGQ) (from an earlier catalog; freellmpool now catalogs 22 providers).\n- Directory: [FreeLLM Pool on MCP Market](https://mcpmarket.com/server/freellm-pool).\n\n## Contributing\n\nNew providers and fixes to stale limits are the most useful contributions, and\nboth are usually a small change to `providers.toml`. See\n[CONTRIBUTING.md](CONTRIBUTING.md). Maintainer-ready newcomer tasks are drafted in\n[docs/GOOD_FIRST_ISSUES.md](docs/GOOD_FIRST_ISSUES.md). Tests run with no network access:\n\n```bash\npython -m pip install -e \".[dev]\" && ruff check . && pytest\n```\n\nSource-first verification in this repo uses `PYTHONPATH=src` so `pytest` exercises\nthe checkout without requiring an editable install first; CI runs the same\nconfiguration. Release readiness uses `PYTHONPATH=src python3 scripts/check_release_ready.py`.\nSecurity scanners, severity gates, the expiring exception process, local\nreproduction, SBOMs, and provenance verification are documented in\n[SECURITY.md](SECURITY.md).\n\n## License\n\nMIT\n",
  "bytes": 37288,
  "sha": "9f418fed62c75a0e5e7c05656782458de8ccc3c9e143a820cc5413cbb036ac4e",
  "repo_slug": "0xzr/freellmpool",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_0xzr_freellmpool_b8340e3e/readme"
}