{
  "markdown": "<!-- mcp-name: io.github.Ridadata/mcp-data-profiler -->\n\n<div align=\"center\">\n\n# mcp-data-profiler\n\n**Let an AI agent understand a dataset without reading it.**\n\nAn [MCP](https://modelcontextprotocol.io) server that turns a CSV, Parquet, JSON, or Excel file\ninto a compact structured profile — types, ranges, missing values, and likely data-quality\nproblems — instead of raw rows.\n\n[![CI](https://github.com/Ridadata/mcp-data-profiler/actions/workflows/ci.yml/badge.svg)](https://github.com/Ridadata/mcp-data-profiler/actions/workflows/ci.yml)\n[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue.svg)](https://www.python.org/downloads/)\n[![PyPI](https://img.shields.io/pypi/v/mcp-data-profiler.svg)](https://pypi.org/project/mcp-data-profiler/)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n[![MCP Registry](https://img.shields.io/badge/MCP%20Registry-listed-orange.svg)](https://registry.modelcontextprotocol.io/v0/servers?search=mcp-data-profiler)\n\n</div>\n\n---\n\n## Overview\n\nTo let an AI agent reason about a data file, you normally paste rows into the conversation.\nThat is expensive, truncates on anything large, and still leaves the model guessing at column\ntypes and null rates.\n\nThis server answers the question directly. One tool call returns a structured summary that is\norders of magnitude smaller than the data and says more about it:\n\n| Dataset | Raw file | Profile | Reduction | Time |\n| --- | ---: | ---: | ---: | ---: |\n| Google Play Store (2.3M rows × 24 cols) | 645 MB | 13 KB | **49,205×** | 1.9 s |\n| SNCF punctuality (10,687 rows × 26 cols) | 2 MB | 13 KB | 189× | 0.2 s |\n| Orders sample (5,000 rows × 6 cols) | 241 KB | 2.3 KB | 104× | 0.1 s |\n\nThe 645 MB file cannot go into a context window at any price. It is fully characterised here in\nunder two seconds.\n\n## Demo\n\n`python demo.py` generates a deliberately messy dataset, profiles it, and reports what came back:\n\n![A terminal running python demo.py: a 237 KB, 5,000-row CSV is reduced to a 2 KB profile in a\ntenth of a second, with six data-quality findings listed across four columns](docs/demo.gif)\n\nThe same profiler seen from an MCP client — one question, one `profile_dataset` call, and the file\nis characterised without a single row entering the conversation:\n\n![An MCP client is asked what is in orders.csv: it calls profile_dataset, gets a 2.3 KB profile\nback, and reports every column with its type, range and quality flags](docs/mcp-demo.gif)\n\nThree real problems surfaced before any analysis began: a column that never varies, one that is\nentirely empty, and a date column that sorts as text — so `\"2024-10-01\" < \"2024-9-01\"` — silently\ncorrupting any time-based result.\n\n<!-- <sub>Both recordings are [VHS](https://github.com/charmbracelet/vhs) tapes —\n[`docs/demo.tape`](docs/demo.tape) and [`docs/mcp-demo.tape`](docs/mcp-demo.tape) — and regenerate\nwith two `docker run` lines. The conversation in the second one is scripted, but every figure in\nit comes from a real `profile_dataset` call; see [docs/vhs/README.md](docs/vhs/README.md).</sub> -->\n\n## Features\n\n- **Six data-quality flags** — constant, all-null, probable ID, mixed types, and numbers or dates\n  stored as text.\n- **Full column statistics** — dtype, null count and percentage, distinct count, sample values,\n  quartiles for numerics, ranges for dates, frequent values for categories.\n- **Bounded output** — the response stays small no matter how wide the input, and always reports\n  what it truncated.\n- **Honest sampling** — large files are sampled, but never silently; the true row count is always\n  included.\n- **Five formats, eleven extensions** — `.csv` `.tsv` `.txt` `.parquet` `.pq` `.json` `.jsonl`\n  `.ndjson` `.xlsx` `.xlsm` `.xls`, plus `.gz` variants of the text formats.\n- **Path confinement** — optional `--root` restricts profiling to a single directory.\n- **Zero configuration** — no database, no index, no warm-up. Point it at a file.\n\n## Architecture\n\n```mermaid\nflowchart LR\n    A[\"MCP client<br/>Claude Code, Claude Desktop\"]\n    B[\"server.py<br/>MCP adapter\"]\n    C[\"profiler.py<br/>pure pandas, no MCP\"]\n    D[(\"Local files<br/>CSV, Parquet<br/>JSON, Excel\")]\n\n    A -->|\"profile_dataset(path)\"| B\n    B -->|\"validate, confine to --root\"| C\n    C -->|\"sampled read\"| D\n    D -->|\"DataFrame\"| C\n    C -->|\"bounded JSON profile\"| B\n    B -->|\"tool result\"| A\n```\n\nAll profiling logic lives in `profiler.py`, which imports nothing from MCP. It is unit-testable\nwithout a protocol harness and usable as an ordinary Python library. `server.py` is only the\nadapter.\n\n## Installation\n\nRequires **Python 3.10+**.\n\n```bash\npip install mcp-data-profiler\n```\n\n<details>\n<summary>Install the development version</summary>\n\n```bash\npip install git+https://github.com/Ridadata/mcp-data-profiler.git\n```\n\n</details>\n\n### Claude Code\n\n```bash\nclaude mcp add data-profiler -- mcp-data-profiler\n```\n\n### Claude Desktop and other MCP clients\n\nAdd to your client's MCP configuration:\n\n```json\n{\n  \"mcpServers\": {\n    \"data-profiler\": {\n      \"command\": \"mcp-data-profiler\"\n    }\n  }\n}\n```\n\nTo confine the server to one directory, add `\"args\": [\"--root\", \"/path/to/your/data\"]`.\n\n## Usage\n\nOnce registered, ask in plain language:\n\n- *\"Profile `data/orders.csv`\"*\n- *\"Which columns have missing values?\"*\n- *\"Is this dataset clean enough to model?\"*\n\n### Tool reference\n\n**`profile_dataset(path, sample_rows=50000, max_columns=100, top_k=5, sheet=None)`**\n\n| Argument | Type | Default | Description |\n| --- | --- | --- | --- |\n| `path` | `str` | *required* | File to profile; `.gz` is decompressed transparently |\n| `sample_rows` | `int \\| null` | `50000` | Rows to read. `null` reads everything — exact, slower |\n| `max_columns` | `int` | `100` | Cap on columns described, so wide tables stay small |\n| `top_k` | `int` | `5` | Frequent values listed per categorical column |\n| `sheet` | `str \\| null` | first sheet | Which Excel sheet to profile, by name |\n\n### Quality flags\n\n| Flag | Meaning |\n| --- | --- |\n| `all_null` | Column is entirely empty |\n| `constant` | Only ever one value — no signal |\n| `high_cardinality_possible_id` | Nearly all values distinct; an identifier, not a feature |\n| `numeric_stored_as_text` | Numbers typed as strings; comparisons and sorting will be wrong |\n| `date_stored_as_text` | Dates typed as strings; same problem |\n| `mixed_types` | One column holding several unrelated Python types |\n\n### As a Python library\n\n```python\nfrom mcp_data_profiler import profile_dataset\n\nprofile = profile_dataset(\"data/orders.csv\", sample_rows=None)\nprint(profile[\"shape\"])          # {'rows_profiled': 5000, 'total_rows': 5000, 'columns': 6}\nprint(profile[\"duplicate_rows\"]) # 0\n```\n\n## Example output\n\nVerbatim output for the sample dataset produced by `python demo.py`, with three of the six\ncolumns shown:\n\n```json\n{\n  \"file\": { \"name\": \"orders.csv\", \"format\": \"csv\", \"size_bytes\": 247263 },\n  \"shape\": { \"rows_profiled\": 5000, \"total_rows\": 5000, \"columns\": 6 },\n  \"sampled\": false,\n  \"columns\": [\n    {\n      \"name\": \"order_id\",\n      \"dtype\": \"str\",\n      \"null_count\": 0,\n      \"null_pct\": 0.0,\n      \"unique_count\": 5000,\n      \"sample_values\": [\"ORD-000000\", \"ORD-000001\", \"ORD-000002\"],\n      \"flags\": [\"high_cardinality_possible_id\"]\n    },\n    {\n      \"name\": \"amount_eur\",\n      \"dtype\": \"float64\",\n      \"null_count\": 0,\n      \"null_pct\": 0.0,\n      \"unique_count\": 1368,\n      \"stats\": {\n        \"min\": 2.65, \"max\": 1369.65, \"mean\": 684.2364, \"std\": 395.254451,\n        \"q25\": 341.65, \"median\": 683.65, \"q75\": 1025.65\n      },\n      \"sample_values\": [2.65, 39.65, 76.65]\n    },\n    {\n      \"name\": \"currency\",\n      \"dtype\": \"str\",\n      \"null_count\": 0,\n      \"null_pct\": 0.0,\n      \"unique_count\": 1,\n      \"top_values\": [{ \"value\": \"EUR\", \"count\": 5000 }],\n      \"sample_values\": [\"EUR\", \"EUR\", \"EUR\"],\n      \"flags\": [\"constant\"]\n    }\n  ],\n  \"duplicate_rows\": 0\n}\n```\n\nNote that `order_id` carries no `top_values`: for a near-unique column every count would be `1`,\nso the list is omitted rather than padding the response with noise.\n\nWhen a file is sampled, the profile also carries `\"sampled\": true`, the true `total_rows`, and a\n`sampling_note` saying so.\n\n## Design notes\n\n**Bounded output.** The tool must cost less than the data it describes, so the response is capped\nregardless of input width and long strings are truncated. Near-unique columns skip the\nfrequent-values list, since every count would be `1`.\n\n**Honest sampling.** Large files are profiled from a sample, but the result always carries\n`\"sampled\": true` alongside the true row count — a silently sampled statistic is a wrong\nstatistic. Row counts come from Parquet metadata or a raw newline scan, never a full parse into\nmemory.\n\n**No silent wrong answers.** The same rule governs every default that could mislead. A workbook's\nfirst sheet is often a title page, so Excel profiles always name the sheet used and list the\nothers rather than reporting an untouched sheet as a clean dataset. CSV delimiters are inferred by\ntesting candidates for a stable column count, which handles the semicolon files common in European\nopen data without the header-mangling that character-frequency sniffers cause. Compressed files are\ndecompressed before either check, since inspecting gzip bytes as text yields a plausible-looking\nanswer that is entirely wrong.\n\n**Path safety.** `--root` confines profiling to one directory. Paths are canonicalised before the\ncheck, so `..` and symlinks cannot escape it.\n\n## Limitations\n\n- **Read-only, local files.** No databases, no URLs, no writes.\n- **Sampled by default.** Statistics reflect the first 50,000 rows unless you pass\n  `sample_rows=null`.\n- **Row-oriented.** No cross-column correlations, outlier detection, or plots.\n- **pandas parsing rules apply.** The profile shows what pandas sees, which is what your own code\n  will see. Notably `\"NA\"`, `\"N/A\"`, and `\"None\"` are read as *missing*, so a region column\n  containing `\"NA\"` for North America will report nulls. That trap is surfaced, not hidden.\n- **Nested JSON is not flattened.** Unhashable cells make the duplicate check inapplicable, and it\n  is reported as `null`.\n- **One Excel sheet per call.** The profile names the sheet read and lists the rest; pass `sheet`\n  to switch.\n\n## Development\n\n```bash\ngit clone https://github.com/Ridadata/mcp-data-profiler.git\ncd mcp-data-profiler\npip install -e \".[dev]\"\n\npytest                                  # 40 tests\nruff check src tests demo.py            # lint\nruff format --check src tests demo.py   # formatting\npython demo.py                          # profile a generated sample dataset\npython demo.py path/to/your.csv         # profile your own files\n```\n\nCI runs the suite on Python 3.10–3.13 (Linux) plus Windows and macOS, and performs a real stdio\nhandshake against the built server to confirm it starts and advertises its tool.\n\n### Releasing\n\nPublishing to PyPI is automated via [Trusted Publishing][tp], so no API token is stored in this\nrepository. Publishing a GitHub Release triggers `.github/workflows/release.yml`, which builds the\ndistributions, verifies the built wheel actually installs and imports, and uploads it.\n\n[tp]: https://docs.pypi.org/trusted-publishers/\n\nIssues and pull requests are welcome.\n\n## Roadmap\n\n- [x] Publish to PyPI\n- [x] Gzip-compressed inputs (`.csv.gz`, `.jsonl.gz`)\n- [x] List on the official MCP registry\n- [ ] Cross-column correlation summary for numeric features\n- [ ] Multi-sheet Excel profiling in a single call\n- [ ] Remote sources (`s3://`, `https://`)\n\n## License\n\n[MIT](LICENSE) © Rida Aderkane\n",
  "bytes": 11643,
  "sha": "043085aa367611ca94161b5ff112db86091cc438999b8290422506c76510c341",
  "repo_slug": "ridadata/mcp-data-profiler",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_ridadata_mcp_data_profiler_6b5aba04/readme"
}