{
  "markdown": "# Multi-LLM Provider for Go\n\n[![CI](https://github.com/manishiitg/llm-provider-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/manishiitg/llm-provider-mcp/actions/workflows/ci.yml)\n[![Release](https://img.shields.io/github/v/release/manishiitg/llm-provider-mcp)](https://github.com/manishiitg/llm-provider-mcp/releases)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n\n`multi-llm-provider-go` is a Go library for using hosted LLM APIs and local\ncoding agents through a shared set of provider interfaces.\n\nIt supports two complementary ways to run a model:\n\n- **API providers** call hosted models through their normal SDK or HTTP\n  transport, including OpenAI, Anthropic, Bedrock, Vertex AI, Azure, and\n  OpenRouter.\n- **Coding-agent providers** run Claude Code, Codex CLI, Cursor Agent, or Pi in\n  local tmux sessions, preserving their native tools, subscriptions, project\n  context, and authenticated sessions.\n\nThe repository also includes `llm-provider-mcp`, an optional MCP server for\ndelegating asynchronous work between coding agents. It is one way to expose the\nprovider library—not the library's only use case.\n\n## Why This Exists\n\nApplications should be able to choose the right model and transport for each\ntask without rebuilding their orchestration layer.\n\nUse a direct API when you want a conventional request/response integration,\npredictable infrastructure, or model-level features such as structured output,\nembeddings, and media generation. Use a coding-agent CLI when you want an agent\nthat can inspect a repository, edit files, run commands, and reuse an existing\nlocal subscription. Both fit behind the same Go model abstraction.\n\n```mermaid\nflowchart LR\n    App[Go application] --> Provider[Shared provider interfaces]\n    Provider --> API[API and cloud adapters]\n    Provider --> CLI[Local coding-agent adapters]\n    API --> Hosted[Hosted models]\n    CLI --> Tmux[tmux sessions]\n    Tmux --> Agents[Claude Code / Codex / Cursor / Pi]\n    MCP[Optional MCP server] --> CLI\n```\n\n## Supported Providers\n\nThe core `InitializeLLM` factory returns the same `llmtypes.Model` interface for\ntext and coding-agent providers:\n\n| Provider ID | Integration | Transport |\n|---|---|---|\n| `openai` | OpenAI | OpenAI Go SDK |\n| `anthropic` | Anthropic | Anthropic Go SDK |\n| `openrouter` | OpenRouter | OpenAI-compatible API |\n| `bedrock` | AWS Bedrock | AWS SDK |\n| `vertex` | Google Vertex AI and Gemini | Google Gen AI SDK |\n| `azure` | Azure AI | Azure/OpenAI-compatible API |\n| `z-ai` | Z.AI | OpenAI-compatible API |\n| `kimi` | Kimi/Moonshot | OpenAI-compatible API |\n| `minimax`, `minimax-coding-plan` | MiniMax | Provider API |\n| `claude-code` | Claude Code | Local CLI in tmux |\n| `codex-cli` | Codex CLI | Local CLI in tmux by default |\n| `cursor-cli` | Cursor Agent | Local CLI in tmux |\n| `pi-cli` | Pi | Local CLI in tmux by default |\n\nSpecialized factories expose capabilities that do not fit the text-model\ninterface:\n\n| Capability | Providers |\n|---|---|\n| Embeddings | OpenAI, OpenRouter, Vertex AI, Bedrock |\n| Image generation | Vertex AI, MiniMax Coding Plan, Codex CLI |\n| Video generation | Vertex AI (Veo and Gemini Omni) |\n| Text to speech | Vertex AI, MiniMax, ElevenLabs, Deepgram |\n| Audio transcription | Deepgram |\n| Music generation | ElevenLabs, MiniMax |\n\nGemini models are available through Vertex AI for direct API access or through\nPi as a coding agent. The old Gemini CLI adapter has been removed.\n\n## Common Capabilities\n\nProvider support varies, but the shared interfaces cover:\n\n- Text generation and streaming\n- Tool calling and structured output\n- Token usage, model metadata, logging, and event emission\n- Embeddings\n- Image input and generation\n- Video generation and conversational video editing\n- Audio generation and transcription\n- Music generation\n- Stateful coding-agent sessions, continuation, and terminal progress\n\n## Quick Start: Go Library\n\nInstall the module:\n\n```bash\ngo get github.com/manishiitg/multi-llm-provider-go@latest\n```\n\nThe current module requires Go 1.25.12 or newer.\n\nInitialize a provider and use the returned `llmtypes.Model`:\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"log\"\n\n    llmproviders \"github.com/manishiitg/multi-llm-provider-go\"\n    \"github.com/manishiitg/multi-llm-provider-go/llmtypes\"\n)\n\nfunc main() {\n    model, err := llmproviders.InitializeLLM(llmproviders.Config{\n        Provider: llmproviders.ProviderOpenAI,\n        ModelID:  \"gpt-4.1-mini\",\n    })\n    if err != nil {\n        log.Fatal(err)\n    }\n\n    response, err := model.GenerateContent(\n        context.Background(),\n        []llmtypes.MessageContent{\n            llmtypes.TextParts(llmtypes.ChatMessageTypeHuman, \"Explain tmux in one sentence.\"),\n        },\n    )\n    if err != nil {\n        log.Fatal(err)\n    }\n\n    if len(response.Choices) == 0 {\n        log.Fatal(\"provider returned no choices\")\n    }\n    fmt.Println(response.Choices[0].Content)\n}\n```\n\nSet the credential expected by the selected provider—for example,\n`OPENAI_API_KEY` for OpenAI. Credentials can also be supplied explicitly with\n`Config.APIKeys`. See the [examples](examples/README.md) for streaming, tool\ncalls, custom logging, Bedrock, and Vertex AI.\n\n## Configuration\n\n`llmproviders.Config` controls provider initialization:\n\n| Field | Purpose |\n|---|---|\n| `Provider` | Selects the API, cloud platform, or coding CLI |\n| `ModelID` | Selects a model; provider defaults apply when supported |\n| `Temperature` | Sets sampling temperature for providers that expose it |\n| `APIKeys` | Supplies credentials explicitly instead of using the environment |\n| `FallbackModels` / `MaxRetries` | Configures retry and fallback behavior |\n| `Logger` / `EventEmitter` | Connects host logging, tracing, and model events |\n| `Context` | Controls initialization lifetime and cancellation |\n\nCommon credential sources include:\n\n| Provider | Environment or native authentication |\n|---|---|\n| OpenAI | `OPENAI_API_KEY` |\n| Anthropic | `ANTHROPIC_API_KEY` |\n| OpenRouter | `OPENROUTER_API_KEY` |\n| AWS Bedrock | Standard AWS credential chain and `AWS_REGION` |\n| Vertex AI | `VERTEX_API_KEY`, `GOOGLE_API_KEY`, or Google application credentials |\n| Azure AI | `AZURE_AI_ENDPOINT` and `AZURE_AI_API_KEY` |\n| Z.AI / Kimi | `ZAI_API_KEY`, `KIMI_API_KEY` |\n| MiniMax | `MINIMAX_API_KEY` or `MINIMAX_CODING_PLAN_API_KEY` |\n| ElevenLabs / Deepgram | `ELEVENLABS_API_KEY`, `DEEPGRAM_API_KEY` |\n| Coding-agent CLIs | Existing native CLI login or provider configuration |\n\nSee [.env.example](.env.example) for common provider credentials. Model,\nendpoint, fallback, and test-specific variables are documented next to the\nprovider adapters and tests that consume them.\n\nChanging transports starts with changing the provider:\n\n```go\nconfig.Provider = llmproviders.ProviderAnthropic  // direct API\nconfig.Provider = llmproviders.ProviderBedrock    // cloud API\nconfig.Provider = llmproviders.ProviderCodexCLI   // local coding agent\nconfig.Provider = llmproviders.ProviderCursorCLI  // local coding agent\n```\n\nBounded coding-agent calls can use the process working directory and a temporary\nsession automatically. Long-lived host applications should explicitly pass\n`CodingAgentWorkingDirOption`, `CodingAgentInteractiveSessionOption`, and\n`CodingAgentPersistentInteractiveOption`. Provider-specific options additionally\ncontrol the model, approval policy, resume ID, tools, and streaming behavior.\n\n## Coding Agents And tmux\n\nThe coding-agent adapters turn native coding CLIs into providers without\nreimplementing their agent loops. They use each CLI's existing login and model\naccess, and run in a local project with that CLI's native file and shell tools.\n\ntmux is the default transport because it supports long-lived interactive\nsessions, multi-turn continuation, live terminal capture, control-key input,\nand recovery after a caller disconnects.\n\n| CLI | Provider ID | Authentication |\n|---|---|---|\n| Claude Code | `claude-code` | Existing Claude Code login or scoped OAuth token |\n| Codex CLI | `codex-cli` | Existing Codex login |\n| Cursor Agent | `cursor-cli` | Existing Cursor login |\n| Pi CLI | `pi-cli` | Existing Pi/provider configuration |\n\nRequirements for this transport:\n\n- macOS or Linux\n- tmux 3.x or newer\n- At least one installed and authenticated coding CLI\n\nThe library exposes session lifecycle, resume, input, interrupt, pane capture,\nand cleanup helpers so a host application can manage coding agents as part of\nits own workflow.\n\n## Module Layout\n\n```text\nmulti-llm-provider-go/\n├── providers.go                 # Provider IDs, configuration, initialization\n├── provider_*.go                # Shared provider behavior and media factories\n├── interfaces/                  # Logging, events, and public support contracts\n├── llmtypes/                    # Messages, responses, tools, streams, options\n├── pkg/adapters/                # API, cloud, media, and coding-CLI adapters\n├── pkg/codingagentjob/          # Durable asynchronous job execution\n├── pkg/codingagentmcp/          # Optional MCP tool surface\n├── pkg/tmuxcapture/             # Terminal progress capture and cleanup\n├── cmd/llm-chat/                # Local provider chat client\n├── cmd/llm-test/                # Manual provider contract runner\n└── cmd/llm-provider-mcp/        # Optional delegation MCP server\n```\n\n`llmtypes.Model` is the central text request/response interface. Additional\ninterfaces cover embeddings, image generation, video generation, audio\ngeneration and transcription, and music generation.\n\n## Optional: Asynchronous Delegation Over MCP\n\n`llm-provider-mcp` packages the coding-agent providers as a local stdio MCP\nserver. A Codex or Claude Code host can queue work in another coding CLI,\ncontinue working, and retrieve the result later. Jobs are persisted in SQLite\nand executed in detached tmux sessions.\n\nInstall it in the project where you want delegation:\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/manishiitg/llm-provider-mcp/main/scripts/install-mcp.sh | sh\n```\n\nThe setup detects installed CLIs, registers selected hosts and targets for the\ncurrent project, verifies authentication, and installs the delegation skill.\nThe server is also published in the\n[official MCP Registry](https://registry.modelcontextprotocol.io/docs) as\n`io.github.manishiitg/llm-provider-mcp`.\n\nIt exposes five tools:\n\n| Tool | Purpose |\n|---|---|\n| `list_coding_agents` | List enabled targets and capabilities |\n| `list_coding_agent_models` | Discover available model selectors |\n| `delegate_coding_agent` | Start an asynchronous coding job |\n| `get_coding_agent_job` | Read progress, terminal output, or the final result |\n| `cancel_coding_agent_job` | Stop a queued or running job |\n\nSee [Installation](docs/installation.md) and\n[Delegation workflow](docs/delegation.md) for the complete MCP workflow.\n\n## Security And Trust\n\n- Direct API credentials remain in the host process or the provider's normal\n  credential chain.\n- Coding-agent credentials remain owned by the native CLI.\n- tmux-backed agents have the local user's filesystem and process permissions;\n  tmux is a transport, not a sandbox.\n- A coding agent can modify the working tree and run commands. Review and test\n  delegated changes before accepting them.\n- `LLM_PROVIDER_MCP_WORKSPACE_ROOTS` can restrict directories accepted by the\n  MCP server, but it does not create an operating-system sandbox.\n\nRead [Security and trust](docs/security-and-trust.md) before enabling unattended\ncoding-agent execution in sensitive repositories.\n\n## Testing And Coverage\n\nThe repository uses several layers of testing because hosted APIs and terminal\nTUIs fail in different ways:\n\n| Layer | What it verifies | Normal CI |\n|---|---|:---:|\n| Unit and adapter tests | Request conversion, event parsing, metadata, pricing, options, cleanup | Yes |\n| Replay and fixture tests | Provider responses and terminal transcripts without network access | Yes |\n| Contract tests | Shared behavior across API providers and coding agents | Yes |\n| Real API tests | Authentication, live response shape, streaming, tools, media | Opt-in |\n| Real coding-agent E2E | tmux launch, prompts, tools, resume, live input, cancellation, isolation | Opt-in |\n| Downstream compile checks | Public API compatibility with MCP Agent and MCP Agent Builder | Yes |\n\nAPI-provider coverage is not uniform. This inventory reflects the tests and\nmanual commands currently present in the repository:\n\n| Provider | Deterministic or replay coverage | Opt-in live Go tests | Manual `llm-test` commands |\n|---|:---:|:---:|:---:|\n| OpenAI | Yes | Yes | Yes |\n| Anthropic | Yes | Yes | Yes |\n| Bedrock | Yes | Yes | Yes |\n| Vertex AI | Yes | Yes | Yes |\n| Azure AI | Replay | Not yet | Yes |\n| OpenRouter | Replay | Not yet | Yes |\n| Z.AI | Limited | Yes | Yes |\n| Kimi | Model metadata | Yes | Not yet |\n| MiniMax | Yes | Credential-gated | Yes |\n| ElevenLabs / Deepgram | Not yet | Not yet | Not yet |\n\n“Yes” does not mean every capability is covered. The\n[API provider test contract](docs/api_provider_test_contract.md) distinguishes\nautomated Go tests, replay/manual smoke coverage, partial coverage, and known\ngaps at feature level.\n\nThe coding-agent certification suite covers all four active CLI providers:\n\n| Contract area | Claude Code | Codex CLI | Cursor Agent | Pi CLI |\n|---|:---:|:---:|:---:|:---:|\n| tmux launch and working directory | ✓ | ✓ | ✓ | ✓ |\n| Native system instructions and prompt paste | ✓ | ✓ | ✓ | ✓ |\n| Terminal progress and done detection | ✓ | ✓ | ✓ | ✓ |\n| MCP bridge and tool policy | ✓ | ✓ | ✓ | ✓ |\n| Persistent sessions and continuation | ✓ | ✓ | ✓ | ✓ |\n| Live input, cancellation, and cleanup | ✓ | ✓ | ✓ | ✓ |\n| Parallel/session isolation | ✓ | ✓ | ✓ | ✓ |\n\nThe coding-agent matrix shows the release-blocking contract areas. Broader\nnon-P0 certification gaps remain explicitly tracked in\n`knownCertificationGaps`. These checks do not promise that every upstream CLI\nversion behaves identically. Real tests are gated by explicit environment\nvariables and require the relevant CLI login or provider credentials.\n\nRun the offline suite and build the manual test client:\n\n```bash\ngo test -p 1 ./...\nmake build\n./bin/llm-test --help\n```\n\nDetailed, provider-by-provider coverage and real-test commands live in:\n\n- [API provider test contract](docs/api_provider_test_contract.md)\n- [Coding-agent tmux contract](docs/coding_sdk_tmux_contract.md)\n- [Structured coding-agent contract](docs/coding_sdk_structured_contract.md)\n- [Testing guide](docs/TESTING.md)\n\n## Code Quality And Secret Scanning\n\nThe project uses `golangci-lint` for static analysis and `gitleaks` for secret\nscanning:\n\n```bash\nmake lint\nmake scan-secrets\n```\n\n## Documentation\n\n- [Go provider API](docs/go-provider-library.md)\n- [Examples](examples/README.md)\n- [Coding-agent providers and models](docs/providers.md)\n- [MCP installation](docs/installation.md)\n- [Delegation workflow](docs/delegation.md)\n- [Architecture](docs/architecture.md)\n- [Security and trust](docs/security-and-trust.md)\n- [API provider test contract](docs/api_provider_test_contract.md)\n- [Coding-agent tmux contract](docs/coding_sdk_tmux_contract.md)\n- [Replay and manual test runner](docs/TESTING.md)\n- [Roadmap](ROADMAP.md)\n\n## Development\n\n```bash\nmake build\nmake build-mcp\ngo test -p 1 ./...\ngolangci-lint run --timeout=5m ./...\n```\n\nCI also compile-checks MCP Agent and MCP Agent Builder against the current\ncheckout to prevent accidental public API breakage.\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md) before opening a pull request. Report\nsecurity issues using [SECURITY.md](SECURITY.md), not a public issue.\n\n## License\n\n[MIT](LICENSE)\n",
  "bytes": 15689,
  "sha": "5fa5209d077e342844b623c558e59825b24f5f3f37398b0983c3f13252bfbff7",
  "repo_slug": "manishiitg/llm-provider-mcp",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_manishiitg_llm_provider_mcp_92c51e84/readme"
}