{
  "markdown": "# TDAD — Test-Driven AI Development\n\n**Minimizing code regressions in AI coding agents using TDD and GraphRAG.**\n\nTDAD builds a code-test dependency graph and uses it to identify exactly which\ntests are impacted by code changes. When integrated into an AI agent's workflow\nas a skill, it enforces regression checking before patch submission.\n\nEvaluated on SWE-bench Verified (100 instances), GraphRAG-based test impact\nanalysis **reduced AI-introduced regressions by 70%** compared to a vanilla\nbaseline, bringing the test-level regression rate down from 6.08% to 1.82%.\n\n## Install as an AI Agent Skill\n\nTDAD ships as an [Agent Skill](https://agentskills.io) that teaches coding\nagents to check impacted tests before submitting patches.\n\n### Claude Code\n\n```bash\n# From the Claude Code skills marketplace\n/skill install pepealonso95/tdad-skill\n\n# Or via skills.sh\nnpx skills add pepealonso95/tdad-skill\n\n# Or manually: copy into your project\nmkdir -p .claude/skills/tdad\ncp SKILL.md .claude/skills/tdad/SKILL.md\n```\n\n### Other agents\n\nAny agent that supports the [Agent Skills spec](https://agentskills.io/specification)\ncan use the `SKILL.md` file directly. Copy it into the agent's skills directory.\n\n### How the skill works\n\nThe skill instructs the agent to:\n1. Look up impacted tests in `.tdad/test_map.txt` after every code change\n2. Run only the impacted tests (not the full suite)\n3. Fix any regressions before submitting the patch\n\nWorks with Python, JavaScript/TypeScript, Go, Java, Rust, and Dart projects.\n\nSee [SKILL.md](SKILL.md) for the full agent-facing instructions.\n\n## Supported Languages\n\nTDAD supports multi-language repositories through a plugin system. Languages\nare auto-detected from file extensions, or you can specify them explicitly.\n\n| Language | Extensions | Test Runner | Parser | Install |\n|----------|-----------|-------------|--------|---------|\n| **Python** | `.py` | pytest | `ast` (built-in) | _(included)_ |\n| **JavaScript/TypeScript** | `.js` `.jsx` `.ts` `.tsx` `.mjs` `.cjs` | Jest / Vitest / Mocha | tree-sitter | `pip install tdad[treesitter]` |\n| **Go** | `.go` | `go test` | tree-sitter | `pip install tdad[treesitter-go]` |\n| **Java** | `.java` | Maven / Gradle | tree-sitter | `pip install tdad[treesitter-java]` |\n| **Rust** | `.rs` | `cargo test` | tree-sitter | `pip install tdad[treesitter-rust]` |\n| **Dart** | `.dart` | `dart test` / `flutter test` | tree-sitter | `pip install tdad[treesitter-dart]` |\n\n```bash\n# Install all language support at once\npip install tdad[treesitter-all]\n\n# Or install only what you need\npip install tdad[treesitter]          # JS/TS\npip install tdad[treesitter-java]     # Java\n```\n\nPython support requires no extra dependencies. For non-Python languages, TDAD\nuses [tree-sitter](https://tree-sitter.github.io/) for parsing. Languages are\nauto-detected by scanning file extensions in the repository, or you can\noverride with `--languages`:\n\n```bash\ntdad index /path/to/repo --languages python,javascript\n```\n\nOr via environment variable:\n\n```bash\nexport TDAD_LANGUAGES=python,java\n```\n\n## Quick Start (CLI)\n\n```bash\n# Install\npip install tdad\n\n# Index your repo (uses NetworkX by default, no external services needed)\ntdad index /path/to/your/repo\n\n# Find impacted tests for changed files\ntdad impact /path/to/your/repo --files src/module.py\n\n# Run impacted tests\ntdad run-tests /path/to/your/repo --tests tests/test_module.py::test_foo\n\n# Check graph stats\ntdad stats /path/to/your/repo\n```\n\n## How It Works\n\n```\nYour Code Changes\n       |\n       v\n+-------------+     +----------------+     +-------------+\n|  Language   |---->| Dependency     |---->|   Impact    |\n|  Plugins    |     | Graph          |     |  Analyzer   |\n| (ast/ts)    |     | File->Func     |     |  4 strategies|\n+-------------+     | Func->Func     |     +------+------+\n                    | Test->Func     |            |\n                    +----------------+            v\n                                       Ranked list of tests\n                                       to run & verify\n```\n\n### Architecture\n\nTDAD has five core components:\n\n1. **Language Plugins** (`languages/`) — Each supported language implements a\n   `LanguagePlugin` protocol that provides parsing, test detection, and test\n   execution. Python uses the built-in `ast` module; all other languages use\n   tree-sitter grammars.\n\n2. **Graph Builder** (`indexer/graph_builder.py`) — Populates the dependency\n   graph with nodes (File, Function, Class, Test) and edges (CONTAINS, CALLS,\n   IMPORTS, INHERITS, TESTS). Supports both full and incremental indexing via\n   content hashing. Language-agnostic — delegates parsing to plugins.\n\n3. **Test Linker** (`indexer/test_linker.py`) — Creates TESTS relationships\n   between test nodes and the code they exercise, using three strategies:\n   naming conventions, static analysis of imports/calls, and optional\n   per-test coverage data.\n\n4. **Impact Analyzer** (`analyzer/impact.py`) — Given a set of changed files,\n   traverses the graph to produce a ranked list of impacted tests sorted by\n   impact score. Works across all supported languages.\n\n5. **Test Runner** (`runner/test_runner.py`) — Delegates test execution to the\n   appropriate language plugin (pytest, Jest, `go test`, Maven/Gradle,\n   `cargo test`, `dart test`/`flutter test`).\n\n### Graph Schema\n\n- **Nodes**: File, Function, Class, Test\n- **Edges**: CONTAINS, CALLS, IMPORTS, INHERITS, TESTS\n\n### Impact Strategies\n\n| Strategy | Weight | Description |\n|----------|--------|-------------|\n| Direct | 0.95 | Test directly tests a changed function |\n| Transitive | 0.70 | Test tests a function that calls changed code |\n| Coverage | 0.80 | Test has coverage dependency on changed file |\n| Imports | 0.50 | Test file imports the changed file |\n\n## Configuration\n\nAll settings via environment variables with `TDAD_` prefix:\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `TDAD_BACKEND` | `networkx` | Graph backend (`networkx` or `neo4j`) |\n| `TDAD_LANGUAGES` | _(auto-detect)_ | Comma-separated languages (e.g., `python,javascript`) |\n| `TDAD_USE_COVERAGE` | `false` | Enable coverage-based test linking |\n| `TDAD_COVERAGE_THRESHOLD` | `0.1` | Minimum coverage to create a link |\n| `TDAD_INDEX_WORKERS` | `4` | Parallel parsing workers |\n| `TDAD_QUERY_TIMEOUT` | `20.0` | Query timeout (seconds) |\n\n### Neo4j backend (optional)\n\nTo use Neo4j instead of the default NetworkX backend, install the optional\ndependency and configure the connection:\n\n```bash\npip install tdad[neo4j]\nexport TDAD_BACKEND=neo4j\n```\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `TDAD_NEO4J_URI` | `bolt://localhost:7687` | Neo4j connection URI |\n| `TDAD_NEO4J_USER` | `neo4j` | Neo4j username |\n| `TDAD_NEO4J_PASSWORD` | `password` | Neo4j password |\n| `TDAD_NEO4J_DATABASE` | `neo4j` | Neo4j database name |\n\n## Experimental Results\n\nEvaluated on SWE-bench Verified (100 instances) using Qwen3-Coder 30B\n(Q4_K_M quantization) via llama.cpp as the AI coding agent.\n\n### Resolution and Regression Rates\n\n| Approach | Resolution Rate | Test-Level Regression Rate | Total P2P Failures |\n|----------|:--------------:|:--------------------------:|:------------------:|\n| Baseline (vanilla) | 31% (31/100) | 6.08% | 562 |\n| TDD Prompting | 31% (31/100) | 9.94% | 799 |\n| **GraphRAG + TDD** | **29% (29/100)** | **1.82%** | **155** |\n\n### Regression Reduction\n\n| Comparison | P2P Failure Reduction | Rate Change |\n|------------|:---------------------:|:-----------:|\n| GraphRAG vs Vanilla | 562 -> 155 | **-72%** |\n| GraphRAG vs TDD Prompt | 799 -> 155 | **-81%** |\n\n### Generation Rates\n\n| Approach | Generation Rate | Empty Patches |\n|----------|:--------------:|:-------------:|\n| Baseline (vanilla) | 86% (86/100) | 14 |\n| TDD Prompting | 75% (75/100) | 25 |\n| GraphRAG + TDD | 74% (74/100) | 26 |\n\n### Key Findings\n\n1. **72% regression reduction** — GraphRAG + TDD reduced total pass-to-pass\n   test failures from 562 to 155 compared to the vanilla baseline, bringing\n   the test-level regression rate from 6.08% to 1.82%.\n\n2. **TDD prompting alone increased regressions** — Prompt-only TDD (9.94%)\n   performed worse than vanilla (6.08%) because more ambitious fixes touched\n   more code. GraphRAG's graph-based localization counteracted this by\n   constraining edits to well-understood areas.\n\n3. **GraphRAG reduces severity, not just frequency** — The instance-level\n   regression count was similar across approaches (~25 instances), but when\n   a GraphRAG patch was wrong it caused far less collateral damage (fewer\n   tests broken per instance).\n\n4. **Modest resolution trade-off** — GraphRAG resolved 29% vs vanilla's 31%\n   (-2pp). The difference is driven by a higher empty-patch rate (26% vs 14%),\n   not by lower patch quality. When GraphRAG generates a patch, it is more\n   likely to be correct and less likely to regress.\n\n5. **Smaller models need context, not procedure** — Verbose, rigid prompts\n   hurt the 30B quantized model. Providing graph-derived context (what tests\n   are impacted) outperformed prescriptive step-by-step instructions.\n\n### Metric Definitions\n\n- **Resolution Rate** — % of instances where the patch fixes the target issue\n  (passes all FAIL_TO_PASS tests without breaking PASS_TO_PASS tests).\n- **Test-Level Regression Rate** — `sum(PASS_TO_PASS failures) /\n  sum(total PASS_TO_PASS tests) * 100` across all evaluated instances.\n- **Instance-Level Regression Rate** — % of evaluated instances with at least\n  one PASS_TO_PASS failure.\n- **Generation Rate** — % of instances where the agent produced a non-empty\n  patch.\n\n## Development\n\n```bash\n# Install dev dependencies\npip install -e \".[dev]\"\n\n# Install all language parsers for testing\npip install -e \".[dev,treesitter-all]\"\n\n# Run tests\npytest tests/\n```\n\nRead the full paper and experimental details in the [TDAD paper](https://arxiv.org/abs/2603.17973).\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n",
  "bytes": 9965,
  "sha": "fe5ee15dfa1576ea24ae85b75570b5b2d60b97d0ca6bf00dfa2444a47e4f3893",
  "repo_slug": "pepealonso95/tdad-skill",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/plg_pepealonso95_tdad_skill_tdad_1e953ca8/readme"
}