{
  "markdown": "# Datanika\n\n[![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL_3.0-blue.svg)](LICENSE)\n[![Python 3.12+](https://img.shields.io/badge/Python-3.12+-blue.svg)](https://www.python.org)\n[![Built with Reflex](https://img.shields.io/badge/Built_with-Reflex-purple.svg)](https://reflex.dev)\n[![dlt](https://img.shields.io/badge/Extract%20%2B%20Load-dlt-orange.svg)](https://dlthub.com)\n[![dbt-core](https://img.shields.io/badge/Transform-dbt--core-green.svg)](https://www.getdbt.com)\n\n**Open-source data pipeline platform — Extract, Load, Transform, and Orchestrate from a single UI.**\n\nDatanika combines [dlt](https://dlthub.com) (extract + load) with [dbt-core](https://www.getdbt.com) (transform) and adds visual pipeline management, scheduling, and monitoring — all in one Python-native platform.\n\n> Think Airbyte + dbt Cloud + Airflow — in one tool, self-hostable with Docker Compose.\n\n---\n\n## Features\n\n🔌 **36 Connectors** — PostgreSQL, MySQL, Oracle, MongoDB, BigQuery, Snowflake, Stripe, HubSpot, Salesforce, Kafka, ClickHouse, and more\n🔄 **dbt Transformations** — SQL models, tests, snapshots, packages, and source freshness built in\n📊 **Visual Pipeline Builder** — DAG editor with dependency management\n⏰ **Scheduling** — Cron-based with APScheduler, persistent across restarts\n📈 **Monitoring** — Run history, streaming logs, and dashboard stats\n🔐 **Enterprise Security** — RBAC, SSO (SAML/OIDC), audit logging, encrypted credentials\n🌍 **9 Languages** — English, German, French, Spanish, Russian, Greek, Chinese, Arabic, Serbian\n🔌 **REST API** — Full CRUD with OpenAPI/Swagger docs, rate limiting, and scoped API keys\n🤖 **AI-Agent Ready** — hosted + local [MCP server](#ai-agent-integration) (25 tools), `/llms.txt`, agent-guide.md, 5-tier capability API, compile+preview validation, typed error codes, `?wait=true`, `Idempotency-Key`, run cancellation\n🚀 **Pipeline Templates** — One-click starter templates (Stripe→Postgres, Postgres→BigQuery, CSV→DuckDB) with prefilled connection configs\n🔔 **Notifications** — Slack, Telegram, email, and webhook alerts on run completion, plus an in-app notification center\n📦 **Self-Hostable** — Single `docker compose up` — no Kubernetes required\n\n---\n\n## Quick Start\n\n### Docker (recommended)\n\n```bash\ngit clone https://github.com/datanika-io/datanika-core.git\ncd datanika-core\ncp .env.example .env\n# Edit .env with your settings\ndocker compose up -d\n```\n\nApp available at http://localhost:3000\n\n### Development\n\n```bash\nuv venv && source .venv/bin/activate\nuv pip install -e \".[dev]\"\ndocker compose up -d postgres redis  # infrastructure only\nuv run reflex run                     # starts on :3000 + :8000\n```\n\n---\n\n## Releases & versioning\n\nDatanika uses **`0.x` SemVer** with tagged releases (`v0.1.0`, …). While pre-1.0, breaking\nchanges bump the **minor**. There is no `1.0` yet — that will mean a committed, stable\npublic API contract.\n\n**Self-hosting? Pin a release instead of tracking `master`.** `master` is continuously\ndeployed to our hosted app and moves several times a day.\n\n```bash\ngit checkout v0.1.0            # source\ndocker compose up -d --build   # build the image from that source\n```\n\n> ⚠️ **There is no publicly pullable image yet.** `ghcr.io/datanika-io/datanika-core`\n> exists but is **private**, so `docker pull` on any tag answers `denied` for everyone\n> outside the org — this README told you otherwise until 2026-09-03. It stays private on\n> purpose: the image grafts in our closed-source cloud plugin, so publishing it as built\n> today would publish that too. **Pin the source tag and build**; that path is exercised\n> by CI.\n>\n> **Building it yourself gives you the same artifact we would publish.** Since\n> 2026-09-04 the Dockerfile carries a `DATANIKA_IMAGE_EDITION` build arg, and the\n> core-only variant is built and asserted on every PR (`core-only-image` in CI) from a\n> context that does not contain the cloud tree at all — which is the context you have.\n>\n> ⚠️ **The Dockerfile expects a monorepo-shaped context**: it does\n> `COPY datanika/pyproject.toml`, so the build context is the **parent** directory and\n> this checkout has to be the `datanika/` inside it. Clone it under that name and build\n> from one level up:\n>\n> ```bash\n> git clone https://github.com/datanika-io/datanika-core.git datanika\n> docker build --build-arg DATANIKA_IMAGE_EDITION=core -f datanika/Dockerfile .\n> ```\n>\n> The resulting image carries no `/cloud` tree and cannot import `datanika_cloud`.\n> Leave `DATANIKA_EDITION` unset — its default is `core`. Setting it to `cloud` on this\n> image fails immediately and loudly (`ModuleNotFoundError: No module named\n> 'datanika_cloud'`), which is deliberate: the alternative is a service that looks\n> healthy and enforces nothing. Publishing this under a public GHCR package is tracked\n> in [#1014](https://github.com/datanika-io/datanika-core/issues/1014).\n\nEvery version's notes are on the [Releases page](https://github.com/datanika-io/datanika-core/releases).\nSecurity advisories cite the first patched release (e.g. `Patched: v0.1.0`), so a pinned\ntag tells you immediately whether you're affected.\n\n> The `datanika-mcp` sub-package is versioned and released independently (`mcp-v*` tags,\n> published to PyPI).\n\n---\n\n## Why Datanika?\n\n| | Datanika | Airbyte | Fivetran | dbt Cloud |\n|---|---|---|---|---|\n| Extract + Load | ✅ 36 connectors | ✅ 600+ [^1] | ✅ 700+ [^1] | ❌ |\n| Transformations | ✅ dbt built-in | ❌ | ❌ (add-on) | ✅ |\n| Scheduling | ✅ Cron + DAG | ✅ Basic | ✅ Basic | ✅ |\n| Pipeline DAG | ✅ Visual | ❌ | ❌ | ❌ |\n| Self-host | ✅ Docker | ⚠️ Needs K8s | ❌ SaaS only | ❌ SaaS only |\n| Open source | ✅ AGPL-3.0 | ⚠️ ELv2 | ❌ | ❌ |\n| Notifications | ✅ Slack/Telegram/Email/Webhook | ✅ | ✅ | ✅ |\n| Pricing | Free forever | Free tier limited | ~$250+/mo | ~$100+/mo |\n\n[^1]: Competitor connector counts as published by each vendor's own connector\ndirectory, **checked 2026-08-30**. Ours is derived from `ConnectionType` in code,\nnot written by hand. We do not track other vendors' catalogues continuously — if\nthis footnote's date looks old, check the source rather than trusting the number.\n\n---\n\n## Tech Stack\n\n| Component | Technology |\n|-----------|-----------|\n| Frontend | [Reflex](https://reflex.dev) (Python → React) |\n| Backend | Starlette (via Reflex) |\n| Extract + Load | [dlt](https://dlthub.com) |\n| Transform | [dbt-core](https://www.getdbt.com) |\n| Database | PostgreSQL 16 |\n| Task Queue | Celery + Redis |\n| Scheduling | APScheduler |\n\n---\n\n## Roadmap\n\n- [x] 36 connectors (databases, SaaS APIs, files, streaming)\n- [x] dbt transformations, tests, snapshots, packages\n- [x] REST API v1 with OpenAPI/Swagger and typed per-connector inline schemas\n- [x] AI-agent compatibility (`/llms.txt`, agent-guide, 5-tier API, golden-path loop, `?wait=true`, `Idempotency-Key`, run cancel, MCP server)\n- [x] Pipeline templates (one-click setup)\n- [x] In-app notification center with Slack, Telegram, Email, Webhook channels\n- [x] SSO (SAML/OIDC) for Enterprise\n- [x] Usage-based billing (cloud plugin)\n- [x] 2,300+ tests across unit, security, and E2E (SQLite in-memory for speed)\n- [x] Kubernetes Helm chart — in-tree at [`deploy/helm/datanika/`](deploy/helm/datanika/); installs the same image as the Compose path. Bundled Postgres/Redis are single-replica and not production-grade — point it at managed databases (see the chart README)\n- [ ] Data lineage visualization\n\n---\n\n## AI Agent Integration\n\nDatanika speaks [MCP](https://modelcontextprotocol.io/), so AI agents (Claude Desktop, Claude Code, Cursor, …) can browse connections, preview data, compile transformations, and monitor runs natively. **25 tools — 17 read-only, 8 write.** There are two ways in.\n\n**Hosted — nothing to install.** Paste this wherever your client accepts a remote MCP server and authorize in the browser:\n\n```\nhttps://app.datanika.io/mcp\n```\n\nOAuth 2.1, no API key to handle. **Read-only unless you grant write at consent** — a client that asks for nothing gets read-only, and a pasted API key stays read-only here even if its own scopes allow writes.\n\n**Local — stdio.** Published on PyPI as [`datanika-mcp`](https://pypi.org/project/datanika-mcp/) and listed on the [official MCP registry](https://registry.modelcontextprotocol.io) as `io.datanika/datanika-mcp`:\n\n```bash\n# read-only by default; add --allow-write to enable the 8 write tools\nuvx datanika-mcp --url https://app.datanika.io --api-key YOUR_KEY\n```\n\nSee [`datanika-mcp/README.md`](datanika-mcp/README.md) for per-client config snippets and the full tool list, or [datanika.io/docs/mcp-server](https://datanika.io/docs/mcp-server) for the hosted walkthrough.\n\nAdditional agent resources:\n- [`/llms.txt`](https://app.datanika.io/llms.txt) — discovery document\n- [`/api/v1/openapi.json`](https://app.datanika.io/api/v1/openapi.json) — OpenAPI spec with typed inline schemas\n- [`/api/v1/meta/agent-tiers`](https://app.datanika.io/api/v1/meta/agent-tiers) — 5-tier capability stack (JSON)\n- [`docs/api_versioning.md`](docs/api_versioning.md) — stability tiers and deprecation policy\n\n---\n\n## Open-Core Strategy\n\nCore platform is open-source (AGPL-3.0).\nCloud version adds billing, quotas, and usage metering via the `datanika-cloud` plugin.\n\n---\n\n## Links\n\n- 🌐 **Website**: [datanika.io](https://datanika.io)\n- 🚀 **Cloud Platform**: [app.datanika.io](https://app.datanika.io)\n- 📖 **Documentation**: [datanika.io/docs](https://datanika.io/docs)\n- 🔌 **Connectors**: [datanika.io/connectors](https://datanika.io/connectors)\n- 📡 **API Reference**: [datanika.io/docs/api](https://datanika.io/docs/api)\n\n---\n\n## Security\n\nFound a vulnerability? See [SECURITY.md](SECURITY.md) for our disclosure\npolicy, supported versions, and reporting instructions.\n\n---\n\n## Contributing\n\nWe welcome contributors and design partners. Open an issue or contact info@datanika.io.\n\n---\n\n## License\n\n[AGPL-3.0](LICENSE)\n",
  "bytes": 9884,
  "sha": "7a19f1628a18d9323291721a264e67cc707b437234efd52d3832355aa7030f5d",
  "repo_slug": "datanika-io/datanika-core",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_datanika_datanika_mcp_e6cded33/readme"
}