convergence-debugger
Expert debugging guide for AI Counsel's convergence detection system. Use when deliberations aren't converging as expected, early stopping f
Open source Repository Open in the app JSON README (API)
About
Expert debugging guide for AI Counsel's convergence detection system. Use when deliberations aren't converging as expected, early stopping fails, semantic similarity seems wrong, or voting vs semantic
Details
- Kind
- Agent skills
- Topic
- Developer tools
- Publisher
- blueman82
- Origin
- majiayu
- Category
- ferramentas
- Stars
- 1
- Forks
- 2
- Last push
- 2026-05-06T18:17:16Z
- Repository state
- ativo
- Language
- Python
- License
- MIT
- Added
- 2026-09-02 18:18:58
- Updated
- 2026-09-02 18:18:58
- Origin id
blueman82/ai-counsel/.claude/skills/convergence-debugger@main
README
<p align="center">
<img src="assets/ai-counsel.png" alt="AI Counsel Logo" width="400">
</p>
# AI Counsel
[](https://smithery.ai/skills?ns=blueman82&utm_source=github&utm_medium=badge)
True deliberative consensus MCP server where AI models debate and refine positions across multiple rounds.
## 🎬 See It In Action
**Cloud Models Debate** (Claude Sonnet, GPT-5.1 Codex, Gemini):
```javascript
mcp__ai-counsel__deliberate({
question: "Should we use REST or GraphQL for our new API?",
participants: [
{cli: "claude", model: "claude-sonnet-4-5-20250929"},
{cli: "codex", model: "gpt-5.2-codex"},
{cli: "gemini", model: "gemini-2.5-pro"}
],
mode: "conference",
rounds: 3
})
```
**Result**: Converged on hybrid architecture (0.82-0.95 confidence) • [View full transcript](transcripts/20251030_153509_Should_we_use_REST_or_GraphQL_for_our_new_API_Con.md)
**Local Models Debate** (100% private, zero API costs):
```javascript
mcp__ai-counsel__deliberate({
question: "Should we prioritize code quality or delivery speed?",
participants: [
{cli: "ollama", model: "llama3.1:8b"},
{cli: "ollama", model: "mistral:7b"},
{cli: "ollama", model: "deepseek-r1:8b"}
],
mode: "conference",
rounds: 2
})
```
**Result**: 2 models switched positions after Round 1 debate • [View full transcript](transcripts/20251030_153834_Should_we_prioritize_code_quality_or_delivery_spee.md)
---
## What Makes This Different
**AI Counsel enables TRUE deliberative consensus** where models see each other's responses and refine positions across multiple rounds:
- Models engage in actual debate (see and respond to each other)
- Multi-round convergence with voting and confidence levels
- Full audit trail with AI-generated summaries
- Automatic early stopping when consensus reaches (saves API costs)
## Features
- 🎯 **Two Modes**: `quick` (single-round) or `conference` (multi-round debate)
- 🤖 **Mixed Adapters**: CLI tools (claude, codex, droid, gemini) + HTTP services (ollama, lmstudio, openrouter, nebius)
- ⚡ **Auto-Convergence**: Stops when opinions stabilize (saves API costs)
- 🗳️ **Structured Voting**: Models cast votes with confidence levels and rationale
- 🧮 **Semantic Grouping**: Similar vote options automatically merged (0.70+ similarity)
- 🎛️ **Model-Controlled Stopping**: Models decide when to stop deliberating
- 🔬 **Evidence-Based Deliberation**: Models can read files, search code, list files, and run commands to ground decisions in reality
- 💰 **Local Model Support**: Zero API costs with Ollama, LM Studio, llamacpp
- 🔐 **Data Privacy**: Keep all data on-premises with self-hosted models
- 🧠 **Context Injection**: Automatically finds similar past debates and injects context for faster convergence
- 🔍 **Semantic Search**: Query past decisions with `query_decisions` tool (finds contradictions, traces evolution, analyzes patterns)
- 🛡️ **Fault Tolerant**: Individual adapter failures don't halt deliberation
- 📝 **Full Transcripts**: Markdown exports with AI-generated summaries
## Quick Start
Get up and running in minutes:
1. **Install** – follow the commands in [Installation](#installation) to clone the repo, create a virtualenv, and install requirements.
2. **Configure** – set up your MCP client using the `.mcp.json` example in [Configure in Claude Code](#configure-in-claude-code).
3. **Run** – start the server with `python server.py` and trigger the `deliberate` tool using the examples in [Usage](#usage).
**Try a Deliberation:**
```javascript
// Mix local + cloud models, zero API costs for local models
mcp__ai-counsel__deliberate({
question: "Should we add unit tests to new features?",
participants: [
{cli: "ollama", model: "llama2"}, // Local
{cli: "lmstudio", model: "mistral"}, // Local
{cli: "claude", model: "sonnet"} // Cloud
],
mode: "quick"
})
```
> **⚠️ Model Size Matters for Deliberations**
>
> **Recommended**: Use 7B-8B+ parameter models (Llama-3-8B, Mistral-7B, Qwen-2.5-7B) for reliable structured output and vote formatting.
>
> **Not Recommended**: Models under 3B parameters (e.g., Llama-3.2-1B) may struggle with complex instructions and produce invalid votes.
**Available Models**: `claude` (opus 4.5, sonnet, haiku), `codex` (gpt-5.2-codex, gpt-5.1-codex-max, gpt-5.1-codex-mini, gpt-5.2), `droid`, `gemini`, HTTP adapters (ollama, lmstudio, openrouter).
See [CLI Model Reference](docs/CLI_MODEL_REFERENCE.md) for complete details.
> **🧠 Reasoning Effort Control**
>
> Control reasoning depth per-participant for codex and droid adapters:
> ```javascript
> participants: [
> {cli: "codex", model: "gpt-5.2-codex", reasoning_effort: "high"}, // Deep reasoning
> {cli: "droid", model: "gpt-5.1-codex-max", reasoning_effort: "low"} // Fast response
> ]
> ```
> - **Codex**: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`
> - **Droid**: `off`, `low`, `medium`, `high`
> - Config defaults set in `config.yaml`, per-participant overrides at runtime
For model choices and picker workflow, see [Model Registry & Picker](docs/model-registry-and-picker.md).
## Installation
### Prerequisites
1. **Python 3.11+**: `python3 --version`
2. **At least one AI tool** (optional - HTTP adapters work without CLI):
- **Claude CLI**: https://docs.claude.com/en/docs/claude-code/setup
- **Codex CLI**: https://github.com/openai/codex
- **Droid CLI**: https://github.com/Factory-AI/factory
- **Gemini CLI**: https://github.com/google-gemini/gemini-cli
### Setup
```bash
git clone https://github.com/blueman82/ai-counsel.git
cd ai-counsel
python3 -m venv .venv
source .venv/bin/activate # macOS/Linux; Windows: .venv\Scripts\activate
pip install -r requirements.txt
python3 -m pytest tests/unit -v # Verify installation
```
✅ Ready to use! Server includes core dependencies plus optional convergence backends (scikit-learn, sentence-transformers) for best accuracy.
## Configuration
Edit `config.yaml` to configure adapters and settings:
```yaml
adapters:
claude:
type: cli
command: "claude"
args: ["-p", "--model", "{model}", "--settings", "{\"disableAllHooks\": true}", "{prompt}"]
timeout: 300
ollama:
type: http
base_url: "http://localhost:11434"
timeout: 120
max_retries: 3
defaults:
mode: "quick"
rounds: 2
max_rounds: 5
```
**Note:** Use `type: cli` for CLI tools and `type: http` for HTTP adapters (Ollama, LM Studio, OpenRouter).
### Model Registry Configuration
Control which models are available for selection in the model registry. Each model can be enabled or disabled without removing its definition:
```yaml
model_registry:
claude:
- id: "claude-sonnet-4-5-20250929"
label: "Claude Sonnet 4.5"
tier: "balanced"
default: true
enabled: true # Model is active and available
- id: "claude-opus-4-20250514"
label: "Claude Opus 4"
tier: "premium"
enabled: false # Temporarily disabled (cost control, testing, etc.)
```
**Enabled Field Behavior:**
- `enabled: true` (default) - Model appears in `list_models` and can be selected for deliberations
- `enabled: false` - Model is hidden from selection but definition retained for easy re-enabling
- Disabled models cannot be used even if explicitly specified in `deliberate` calls
- Default model selection skips disabled models automatically
**Use Cases:**
- **Cost Control**: Disable expensive models temporarily without losing configuration
- **Testing**: Enable/disable specific models during integration tests
- **Staged Rollout**: Configure new models as disabled, enable when ready
- **Performance Tuning**: Disable slow models during rapid iteration
- **Compliance**: Temporarily restrict models pending approval
## Core Features Deep Dive
### Convergence Detection & Auto-Stop
Models automatically converge and stop deliberating when opinions stabilize, saving time and API costs. Status: Converged (≥85% similarity), Refining (40-85%), Diverging (<40%), or Impasse (stable disagreement). Voting takes precedence: when models cast votes, convergence reflects voting outcome.
→ **[Complete Guide](docs/convergence-detection.md)** - Thresholds, backends, configuration
### Structured Voting
Models cast votes with confidence levels (0.0-1.0), rationale, and continue_debate signals. Votes determine consensus: Unanimous (3-0), Majority (2-1), or Tie. Similar options automatically merged at 0.70+ similarity threshold.
→ **[Complete Guide](docs/structured-voting.md)** - Vote structure, examples, integration
### HTTP Adapters & Local Models
Run Ollama, LM Studio, OpenRouter, or Nebius for flexible API costs and privacy options. Mix with cloud models (Claude, GPT-4) in single deliberation.
→ **[Setup Guides](docs/http-adapters/intro.md)** - Ollama, LM Studio, OpenRouter, cost analysis
### Extending AI Counsel
Add new CLI tools or HTTP adapters to fit your infrastructure. Simple 3-5 step process with examples and testing patterns.
→ **[Developer Guide](docs/adding-adapters.md)** - Step-by-step tutorials, real-world examples
## Evidence-Based Deliberation
Ground design decisions in reality by querying actual code, files, and data:
```javascript
// MCP client example (e.g., Claude Code)
mcp__ai_counsel__deliberate({
question: "Should we migrate from SQLite to PostgreSQL?",
participants: [
{cli: "claude", model: "sonnet"},
{cli: "codex", model: "gpt-4"}
],
rounds: 3,
working_directory: process.cwd() // Required - enables tools to access your files
})
```
**During deliberation, models can:**
- 📄 Read files: `TOOL_REQUEST: {"name": "read_file", "arguments": {"path": "config.yaml"}}`
- 🔍 Search code: `TOOL_REQUEST: {"name": "search_code", "arguments": {"pattern": "database.*connect"}}`
- 📋 List files: `TOOL_REQUEST: {"name": "list_files", "arguments": {"pattern": "*.sql"}}`
- ⚙️ Run commands: `TOOL_REQUEST: {"name": "run_command", "arguments": {"command": "git", "args": ["log", "--oneline"]}}`
**Example workflow:**
1. Model A proposes PostgreSQL based on assumptions
2. Model B requests: `read_file` to check current config
3. Tool returns: `database: sqlite, max_connections: 10`
4. Model B searches: `search_code` for database queries
5. Tool returns: 50+ queries with complex JOINs
6. Models converge: "PostgreSQL needed for query complexity and scale"
7. Decision backed by evidence, not opinion
**Benefits:**
- Decisions rooted in current state, not assumptions
- Applies to code reviews, architecture choices, testing strategy
- Full audit trail of evidence in transcripts
**Supported Tools:**
- `read_file` - Read file contents (max 1MB)
- `search_code` - Search regex patterns (ripgrep or Python fallback)
- `list_files` - List files matching glob patterns
- `run_command` - Execute safe read-only commands (ls, git, grep, etc.)
### Configuration
Control tool behavior in `config.yaml`:
**Working Directory** (Required):
- Set `working_directory` parameter when calling `deliberate` tool
- Tools resolve relative paths from this directory
- Example: `working_directory: process.cwd()` in JavaScript MCP clients
**Tool Security** (`deliberation.tool_security`):
- `exclude_patterns`: Block access to sensitive directories (default: `transcripts/`, `.git/`, `node_modules/`)
- `max_file_size_bytes`: File size limit for `read_file` (default: 1MB)
- `command_whitelist`: Safe commands for `run_command` (ls, grep, find, cat, head, tail)
**File Tree** (`deliberation.file_tree`):
- `enabled`: Inject repository structure into Round 1 prompts (default: true)
- `max_depth`: Directory depth limit (default: 3)
- `max_files`: Maximum files to include (default: 100)
**Adapter-Specific Requirements:**
| Adapter | Working Directory Behavior | Configuration |
|---------|---------------------------|---------------|
| **Claude** | Automatic isolation via subprocess `{working_directory}` | No special config needed |
| **Codex** | No true isolation - can access any file | Security consideration: models can read outside `{working_directory}` |
| **Droid** | Automatic isolation via subprocess `{working_directory}` | No special config needed |
| **Gemini** | Enforces workspace boundaries | **Required**: `--include-directories {working_directory}` flag |
| **Ollama/LMStudio** | N/A - HTTP adapters | No file system access restrictions |
**Learn More:**
- [Complete Configuration Reference](CLAUDE.md#configuration-notes) - All config.yaml settings explained
- [Working Directory Isolation](CLAUDE.md#core-components) - How adapters handle file paths
- [Tool Security Model](CLAUDE.md#evidence-based-deliberation) - Whitelists, limits, and exclusions
- [Adding Custom Tools](docs/adding-tool.md) - Developer guide for extending the tool system
### Troubleshooting
**"File not found" errors:**
- Ensure `working_directory` is set correctly in your MCP client call
- Use discovery pattern: `list_files` → `read_file`
- Check file paths are relative to working directory
**"Access denied: Path matches exclusion pattern":**
- Tools block `transcripts/`, `.git/`, `node_modules/` by default
- Customize via `deliberation.tool_security.exclude_patterns` in config.yaml
**Gemini "File path must be within workspace" errors:**
- Verify Gemini's `--include-directories` flag uses `{working_directory}` placeholder
- See adapter-specific setup above
**Tool timeout errors:**
- Increase `deliberation.tool_security.tool_timeout` for slow operations
- Default: 10 seconds for file operations, 30 seconds for commands
**Learn More:**
- [Adding Custom Tools](docs/adding-tool.md) - Developer guide for extending tool system
- [Architecture & Security](CLAUDE.md#evidence-based-deliberation) - How tools work under the hood
- [Common Gotchas](CLAUDE.md#common-gotchas) - Advanced settings and known issues
## Decision Graph Memory
AI Counsel learns from past deliberations to accelerate future decisions. Two core capabilities:
### 1. Automatic Context Injection
When starting a new deliberation, the system:
- Searches past debates for similar questions (semantic similarity)
- Finds the top-k most relevant decisions (configurable, default: 3)
- Injects context into Round 1 prompts automatically
- Result: Models start with institutional knowledge, converge faster
### 2. Semantic Search with `query_decisions`
Query past deliberations programmatically:
- **Search similar**: Find decisions related to a question
- **Find contradictions**: Detect conflicting past decisions
- **Trace evolution**: See how opinions changed over time
- **Analyze patterns**: Identify recurring themes
**Configuration** (optional - defaults work out-of-box):
```yaml
decision_graph:
enabled: true # Auto-injection on by default
db_path: "decision_graph.db" # Resolves to project root (works for any user/folder)
similarity_threshold: 0.6 # Adjust to control context relevance
max_context_decisions: 3 # How many past decisions to inject
```
**Works for any user from any directory** - database path is resolved relative to project root.
→ **[Quickstart](docs/decision-graph/quickstart.md)** | **[Configuration](docs/decision-graph/configuration.md)** | **[Context Injection](docs/decision-graph/using-context-injection.md)**
## Usage
### Start the Server
```bash
python server.py
```
### Configure in Claude Code
**Option A: Project Config (Recommended)** - Create `.mcp.json`:
```json
{
"mcpServers": {
"ai-counsel": {
"type": "stdio",
"command": ".venv/bin/python",
"args": ["server.py"],
"env": {}
}
}
}
```
**Option B: User Config** - Add to `~/.claude.json` with absolute paths.
After configuration, restart Claude Code.
### Model Selection & Session Defaults
- Discover the allowlisted models for each adapter by running the MCP tool `list_models`.
- Set per-session defaults with `set_session_models`; leave `model` blank in `deliberate` to use those defaults.
- Full instructions and request examples live in [Model Registry & Picker](docs/model-registry-and-picker.md).
### Examples
**Quick Mode:**
```javascript
mcp__ai-counsel__deliberate({
question: "Should we migrate to TypeScript?",
participants: [{cli: "claude", model: "sonnet"}, {cli: "codex", model: "gpt-5.2-codex"}],
mode: "quick"
})
```
**Conference Mode (multi-round):**
```javascript
mcp__ai-counsel__deliberate({
question: "JWT vs session-based auth?",
participants: [
{cli: "claude", model: "sonnet"},
{cli: "codex", model: "gpt-5.2-codex"}
],
rounds: 3,
mode: "conference"
})
```
**Search Past Decisions:**
```javascript
mcp__ai-counsel__query_decisions({
query_text: "database choice",
threshold: 0.5, // NEW! Adjust sensitivity (0.0-1.0, default 0.6)
limit: 5
})
// Returns: Similar past deliberations with consensus and similarity scores
// NEW! Empty results include helpful diagnostics:
{
"type": "similar_decisions",
"count": 0,
"results": [],
"diagnostics": {
"total_decisions": 125,
"best_match_score": 0.45,
"near_misses": [{"question": "Database indexing...", "score": 0.45}],
"suggested_threshold": 0.45,
"message": "No results found above threshold 0.6. Best match scored 0.450. Try threshold=0.45..."
}
}
// Find contradictions
mcp__ai-counsel__query_decisions({
operation: "find_contradictions"
})
// Returns: Decisions where consensus conflicts
// Trace evolution
mcp__ai-counsel__query_decisions({
query: "microservices architecture",
operation: "trace_evolution"
})
// Returns: How opinions evolved over time on this topic
```
### Transcripts
All deliberations saved to `transcripts/` with AI-generated summaries and full debate history.
## Architecture
```
ai-counsel/
├── server.py # MCP server entry point
├── config.yaml # Configuration
├── adapters/ # CLI/HTTP adapters
│ ├── base.py # Abstract base
│ ├── base_http.py # HTTP base
│ └── [adapter implementations]
├── deliberation/ # Core engine
│ ├── engine.py # Orchestration
│ ├── convergence.py # Similarity detection
│ └── transcript.py # Markdown generation
├── models/ # Data models (Pydantic)
├── tests/ # Unit/integration/e2e tests
└── decision_graph/ # Optional memory system
```
## Documentation Hub
### Getting Started
- **[Quick Start](README.md#quick-start)** - 5-minute setup
- **[Installation](README.md#installation)** - Detailed prerequisites and setup
- **[Usage Examples](README.md#usage)** - Quick and conference modes
### Core Concepts
- **[Convergence Detection](docs/convergence-detection.md)** - Auto-stop, thresholds, backends
- **[Structured Voting](docs/structured-voting.md)** - Vote structure, consensus types, vote grouping
- **[Evidence-Based Deliberation](README.md#evidence-based-deliberation)** - Ground decisions in reality with read_file, search_code, list_files, run_command
- **[Decision Graph Memory](docs/decision-graph/quickstart.md)** - Learning from past decisions
### Setup & Configuration
- **[HTTP Adapters](docs/http-adapters/intro.md)** - Ollama, LM Studio, OpenRouter setup
- **[Configuration Reference](docs/convergence-detection.md#configuration)** - All YAML options
- **[Migration Guide](docs/migration/cli_tools_to_adapters.md)** - From cli_tools to adapters
### Development
- **[Adding Adapters](docs/adding-adapters.md)** - CLI and HTTP adapter development
- **[CLAUDE.md](CLAUDE.md)** - Architecture, development workflow, gotchas
- **[Model Registry & Picker](docs/model-registry-and-picker.md)** - Managing allowlisted models and MCP picker tools
### Reference
- **[Troubleshooting](docs/troubleshooting/http-adapters.md)** - HTTP adapter issues
- **[Decision Graph Docs](docs/decision-graph/)** - Advanced memory features
## Development
### Running Tests
```bash
pytest tests/unit -v # Unit tests (fast)
pytest tests/integration -v -m integration # Integration tests
pytest --cov=. --cov-report=html # Coverage report
```
See [CLAUDE.md](CLAUDE.md) for development workflow and architecture notes.
### Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/your-feature`)
3. Write tests first (TDD workflow)
4. Implement feature
5. Ensure all tests pass
6. Submit PR with clear description
## License
MIT License - see LICENSE file
## Credits
Built with:
- [MCP SDK](https://modelcontextprotocol.io/) - Model Context Protocol
- [Pydantic](https://docs.pydantic.dev/) - Data validation
- [pytest](https://pytest.org/) - Testing framework
Inspired by the need for true deliberative AI consensus beyond parallel opinion gathering.
---
## Status
**Production Ready** - Multi-model deliberative consensus with cross-user decision graph memory, structured voting, and adaptive early stopping for critical technical decisions!