{
  "markdown": "# WildDatum\n\n[![CI](https://github.com/krnzt/wilddatum/actions/workflows/ci.yml/badge.svg)](https://github.com/krnzt/wilddatum/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](LICENSE)\n[![Status: public alpha](https://img.shields.io/badge/status-public_alpha-6cab70.svg)](#project-status)\n\nWildDatum is a local-first ecological data workbench exposed as a standard Model\nContext Protocol server. It lets Codex, Claude, and other MCP agents discover,\nmaterialize, query, visualize, select, and cite scientific data without putting\ncredentials, private paths, or millions of rows into model context.\n\nNEON plus public ERDDAP services from EMSO ERIC, ICOS Carbon Portal, and\nEuro-Argo/Ifremer are built-in remote providers. Local tabular, raster, vector,\npoint-cloud, image, hyperspectral, and N-dimensional array sources use the same\nprovider-independent manifest, query, view, selection, and provenance model.\n\n> “Compare this NEON LiDAR tile with its hyperspectral cube. Show them together,\n> let me select an individual return or image pixel, then query the exact source\n> point or spectrum behind my selection.”\n\n![WildDatum rendering an official NEON LiDAR tile and hyperspectral cube in Rerun while exposing the selected source point as structured agent context](docs/assets/wilddatum-neon-multimodal.jpg)\n\n_A real WildDatum browser session using Rerun Web Viewer: a 6.6-million-point\nNEON LAS tile, a 500×500×107 reflectance cube, immutable provenance, and a\nverified selection mapped back to LAS source row 2,543,044._\n\n<!-- mcp-name: io.github.krnzt/wilddatum -->\n\n## The human–agent loop\n\n```text\nscientific question\n    → discover products, sites, and local assets\n    → construct and approve a reproducible data plan\n    → materialize immutable, checksum-verified sources\n    → query and render a bounded multimodal Rerun view\n    → human clicks, brushes, filters, or selects\n    → WildDatum records that interaction as scientific state\n    → agent queries the exact selected source data\n    → export results, transformations, provenance, and citation\n```\n\nThe visualization is not just a picture for the model to inspect. `EcoViewSpec`\nis authoritative view state and `SemanticSelection` is authoritative interaction\nstate. A Rerun recording is a regenerable rendering artifact. For verified\nWildDatum point batches, an instance pick maps back to an exact LAS/LAZ source\nrow; for a mapped cube, an image click maps back to the complete source spectrum.\nFor linked trajectories and vertical profiles, a map or profile point maps back\nto the exact physical CSV/TSV, Parquet, or Arrow record, including\nprovider-native QC values.\n\n## What works in the alpha\n\n- A normal MCP 2026-07-28 stdio server registerable with Codex and Claude Code.\n- Public NEON catalog discovery plus reproducible plan, approval, background\n  materialization, checksum, release, license, and citation handling.\n- Public EMSO, ICOS ERDDAP, and Euro-Argo catalog discovery plus validated\n  tabledap/griddap subsets, redirect-aware approval, streaming materialization,\n  license, citation, and live-source provenance.\n- Out-of-band local imports with streaming fingerprints and opaque agent-facing\n  dataset IDs.\n- Arrow/DataFusion tabular queries, spatial raster/vector queries, indexed COPC\n  reads, and bounded HDF5/NetCDF/Zarr N-dimensional slices.\n- Native and browser Rerun views for tables, images, GeoTIFFs, vectors, LiDAR,\n  mapped scientific cubes, and linked geographic trajectories/vertical profiles.\n- Durable human selections that can be converted into provenance-linked source\n  queries and exported as CSV, Parquet, COG, or RO-Crate where applicable.\n- Language-neutral community provider subprocesses with a provider-neutral v2\n  JSON-RPC contract; an RI contributor can work in Rust, Python, R, Go, or\n  another language without adding provider-specific MCP tools.\n\n## Alpha.3: linked scientific views\n\nThe main branch can now inspect the scientific structure of an existing local\nor materialized dataset without exposing its private path:\n\n```bash\nwilddatum inventory ds_...\nwilddatum suggest-views ds_lidar... ds_cube...\nwilddatum create-suggested-view suggest_... ds_lidar... ds_cube...\nwilddatum resolve-selection-links sel_...\n```\n\nThe equivalent registered MCP tools are `inspect_scientific_inventory` and\n`suggest_views`; `create_view_from_suggestion` accepts one of those opaque\nsuggestion IDs, and `resolve_selection_links` evaluates applicable rules after\na human selection. Inventories contain bounded fields, arrays, axes, units,\nCF/QC relationships, evidence, and unresolved decisions. Suggestions remain\ndeterministic and side-effect free until accepted. On acceptance WildDatum\nrecomputes the suggestion, rejects client-invented IDs, and persists an\n`EcoViewSpec` v2 with explicit scientific panels, encodings, and versioned link\nrules. Existing v1 views remain readable.\n\nFor the official NEON teaching pair, WildDatum reads the 107 measured\nwavelengths, selects bands nearest 650/550/450 nm, and ranks a 3D point-cloud +\nRGB + spectrum workspace first. Cube-pixel → spectrum is marked exact. WildDatum\nalso extracts LAS WKT/GeoKey CRS metadata and the HDF5 EPSG, `Map_Info`, spatial\nextent, scale factor, and no-data value. The teaching cube's `Map_Info` origin\ndisagrees with its declared reflectance extent by 500 m, and its footprint does\nnot overlap the teaching LAS tile, so point → image pixel correctly remains\nunavailable for that pair instead of pretending that proximity is\nregistration.\n\nAccepted point-cloud + spectral-cube views render through the same pinned Rerun\nadapter and browser explorer as manually created views. The browser reports the\nview-spec version and panel/link counts. A cube-pixel pick automatically\nmaterializes its exact wavelength-aware spectrum as a durable result with the\nselection and link rule in its provenance. Unavailable link rules remain\nvisible structured state and create no result rather than becoming guessed\ninteractions. Link evaluation refuses stale selections after the view revision\nchanges.\n\nFor datasets with the same authoritative CRS, an internally consistent\nnorth-up affine transform, and overlapping footprints, an exact point pick can\nnow derive the source cube pixel and immediately chain into the complete\nwavelength-aware spectrum. The derived pixel, both link decisions, and the\nspectrum result remain inspectable through the normal MCP, CLI, and browser\ninterfaces.\n\nAfter resolution, the browser regenerates the authoritative view as a complete\nRerun recording with a magenta source/derived pixel marker and the linked\nwavelength/value series inside its spectrum panel. The structured selection and\nresult provenance still drive the overlay; the browser does not reconstruct it\nfrom canvas coordinates.\n\nLinked trajectory/profile views now accept CSV, TSV, Parquet, GeoParquet,\nArrow IPC, and Feather sources. One map can drive up to eight QC-aware value\nprofiles. Inclusive depth/pressure/height ranges and deterministic per-profile\npoint budgets reduce visual load while transparent source slots preserve exact\nRerun-instance → physical-record identity for every displayed format.\n\nDetailed support and caveats are in the [format matrix](docs/FORMATS.md). Design\nboundaries are documented in [architecture](docs/ARCHITECTURE.md) and\n[implementation decisions](docs/DECISIONS.md). Planned Research Infrastructure,\nvisualization, and deployment work is tracked in the public\n[roadmap](ROADMAP.md).\n\n## Install and see it work\n\nThe alpha ships self-contained macOS universal and Linux x86-64 packages. You\ndo not need Rust, Node.js, CMake, or a separate Rerun installation:\n\n```bash\ncurl -fsSL https://raw.githubusercontent.com/krnzt/wilddatum/v0.1.0-alpha.3/scripts/install.sh | sh\n```\n\nThe installer verifies the release SHA-256, installs under `~/.local` by\ndefault, and runs `wilddatum setup`. Set `WILDDATUM_INSTALL_DIR` to choose another\nprefix. If `~/.local/bin` is not already on your `PATH`, add it before continuing.\n\nExisting EcoScope alpha installations remain readable. WildDatum falls back to\nlegacy `ECOSCOPE_DATA_DIR`, `ECOSCOPE_CACHE_DIR`, `ECOSCOPE_WEB_DIST`, default\napplication-data directories, and NEON keychain entries when their WildDatum\nequivalents are absent. The installer also leaves an `ecoscope` command alias\nwhen it can do so without replacing a user-owned file.\n\nCreate a deterministic LiDAR + hyperspectral demonstration and open it in the\nbundled Rerun browser viewer:\n\n```bash\nwilddatum demo synthetic\n```\n\nThe generated LAS and HDF5 files pass through the same import, manifest, cube\nmapping, Rerun recording, and selection-query paths as user data. No network or\ncredentials are needed. An opt-in official NEON teaching-data demonstration is\nalso available (roughly 224 MiB):\n\n```bash\nwilddatum demo neon --accept-download\n```\n\n## Register the normal MCP server\n\nWildDatum is published as `io.github.krnzt/wilddatum` in the official MCP\nRegistry. It is also a normal local stdio server: Codex, Claude Code, and any\ncompatible host launch the same `wilddatum mcp` process and discover its tools.\n\n```bash\nwilddatum register codex\nwilddatum register claude\n```\n\nBoth registration commands are safe to repeat and preserve an existing\nWildDatum entry. Platform-specific MCPB bundles are attached to every release\nfor hosts and registries that install MCPB packages.\n\nEquivalent host commands are:\n\n```bash\ncodex mcp add wilddatum -- /absolute/path/to/wilddatum mcp\nclaude mcp add --scope user wilddatum -- /absolute/path/to/wilddatum mcp\n```\n\nGeneric MCP configuration:\n\n```json\n{\n  \"mcpServers\": {\n    \"wilddatum\": {\n      \"command\": \"/home/you/.local/bin/wilddatum\",\n      \"args\": [\"mcp\"]\n    }\n  }\n}\n```\n\nAfter registration, the host launches WildDatum like any other local MCP server,\nnegotiates the protocol, discovers its tools, and receives bounded structured\nresults rather than bulk scientific files.\n\n## Use local scientific files\n\n```bash\nwilddatum import examples/observations.csv\nwilddatum datasets\nwilddatum preview ds_... --limit 20\nwilddatum create-view --name \"Site comparison\" ds_...\nwilddatum open view_...\n```\n\nLocal paths are selected in the terminal, never passed as an MCP argument. The\nprivate SQLite registry retains the source path; agents receive an opaque ID,\nchecksum, display name, scientific metadata, and provenance. The same path\nsupports the raster, vector, point-cloud, image, and cube formats in the\n[format matrix](docs/FORMATS.md).\n\n## Linked trajectories and vertical profiles\n\nProfile/trajectory rendering is a validated recipe over ordinary CSV, TSV,\nParquet, GeoParquet, Arrow IPC, or Feather data, not a provider-specific\nrenderer. Start locally with the deterministic demo:\n\n```bash\nwilddatum demo profile-trajectory\n```\n\nThe agent workflow is the same for a local import or materialized ERDDAP table:\n\n```text\nuser imports a compatible local table, or agent materializes a provider subset\n  → create_view\n  → configure_profile_trajectory_view\n  → open_view\n  → human selects a map or profile observation\n  → inspect_view\n  → query_selection\n  → exact source record with native identifiers, values, units, and QC\n```\n\nThe recipe explicitly names trajectory/profile identifiers, time, longitude,\nlatitude, vertical coordinate and direction, one primary plus optional\nadditional displayed values, units, fill values, and accepted native QC codes.\nIt can also apply an inclusive source-coordinate vertical range and a\nper-profile display budget. WildDatum validates those fields against the source\nbefore authoring the exact-row mapping; the browser cannot declare a source\nindex trusted, and sampling never changes the source instance slots.\n\n![WildDatum linked trajectory map and vertical temperature profile with a real Rerun point selection serialized as exact-row agent context](docs/assets/profile-trajectory-explorer.png)\n\n_The shipped synthetic profile demo in Rerun Web Viewer. The selected profile\nobservation is persisted as a `rows` selection containing only its Rerun entity,\ninstance, mapping kind, and pinned version; the service independently resolves\nthat instance to the original delimited source row._\n\n## NEON\n\nMetadata discovery does not require credentials. Exact file planning and\ndownloads use a NEON API token stored outside model context:\n\n```bash\n./target/release/wilddatum connect-neon\n```\n\nThe prompt does not echo the token. WildDatum stores it in the operating-system\nkeychain and sends it upstream only in the `X-API-Token` header. Headless systems\ncan inject `NEON_API_TOKEN` through their secret manager.\n\n`wilddatum doctor` time-boxes its noninteractive keychain probe. A\n`neon_connected: null` result with `neon_credential_probe: \"timed_out\"` means the\noperating system did not answer the readiness probe; it does not expose or erase\nthe stored credential.\n\n## Public ERDDAP infrastructures\n\nThe normal registered MCP exposes three credential-free public presets through\nthe same tools used for NEON and community providers:\n\n| Provider ID | Public surface | Boundary |\n|---|---|---|\n| `emso` | [EMSO ERIC ERDDAP](https://erddap.emso.eu/erddap/) | Federated public datasets; approved redirect chains identify the regional server that returns the bytes |\n| `icos-erddap` | [ICOS Carbon Portal ERDDAP](https://erddap.icos-cp.eu/erddap/) | Public ERDDAP only; authenticated Carbon Portal objects are a separate future integration |\n| `euro-argo` | [Ifremer ERDDAP](https://erddap.ifremer.fr/erddap/) | Catalog search is scoped to Argo/Euro-Argo data on the shared service |\n\nAn agent can call `search_catalog` and `inspect_resource` without credentials,\nthen construct a typed table subset:\n\n```json\n{\n  \"provider\": \"emso\",\n  \"resource_id\": \"OBSEA_seabed_station_TS_L1c\",\n  \"variables\": [\"time\", \"TEMP\", \"TEMP_QC\"],\n  \"temporal_start\": \"2025-01-01T00:00:00Z\",\n  \"temporal_end\": \"2025-01-02T00:00:00Z\",\n  \"provider_options\": {\n    \"protocol\": \"tabledap\",\n    \"output_format\": \"csv\",\n    \"constraints\": [\n      {\"variable\": \"TEMP_QC\", \"op\": \"eq\", \"value\": 1}\n    ]\n  }\n}\n```\n\nThat is the input to `plan_materialization`; planning does not download the\ndataset. WildDatum validates every variable against ERDDAP `info` metadata,\ntranslates neutral temporal bounds into `time` constraints for tabledap, probes\nthe exact redirect chain, and returns one URL for approval. Call `approve_plan`\nwith the returned hash and then `materialize_dataset`. The stored object is named\nby its BLAKE3 digest, while the manifest retains the decoded query, redirect\nchain, ETag, Last-Modified value, access time, server version, global attributes,\nvariable-level CF attributes, license, and citation. Downloads stream through a\nconfigurable hard byte ceiling (512 MiB by default) and failed or oversized\npartials are removed.\n\nGrid subsets use `protocol: \"griddap\"` and explicit arrays. Each axis can use\ninteger indices or ERDDAP value coordinates:\n\n```json\n{\n  \"protocol\": \"griddap\",\n  \"output_format\": \"netcdf\",\n  \"arrays\": [{\n    \"variable\": \"temperature\",\n    \"slices\": [\n      {\"start\": \"0\", \"stop\": \"23\", \"stride\": 1},\n      {\"start\": \"10\", \"stop\": \"30\", \"stride\": 2}\n    ]\n  }]\n}\n```\n\nERDDAP subsets are live generated results rather than fixed releases. File size\nis generally unknown at approval time, so constrain variables, rows, time, and\ngrid axes carefully. Materialization freezes the exact returned bytes locally;\nrepeating the same upstream query later may produce a different checksum.\n\n![WildDatum rendering EMSO OBSEA seabed temperature and native quality-control data in Rerun with provenance and structured agent context](docs/assets/wilddatum-erddap-obsea.jpg)\n\n_A real EMSO workflow executed through WildDatum's registered stdio MCP:\n`plan_materialization` → `approve_plan` → `materialize_dataset` → `create_view`\n→ `render_view`. The browser shows the public OBSEA temperature and native QC\nchannels, the regenerable `EcoViewSpec`, and selection events returned as\nstructured agent context._\n\n## Multidimensional data\n\nWildDatum inventories cube arrays without guessing ambiguous scientific meaning.\nWhen X, Y, and spectral axes are unambiguous, common NEON reflectance conventions\nare inferred. Otherwise the mapping is explicit and revisioned:\n\n```bash\n./target/release/wilddatum configure-cube view_... \\\n  --layer-id layer_1 \\\n  --cube-array /SITE/Reflectance/Reflectance_Data \\\n  --y-axis 0 --x-axis 1 --spectral-axis 2 \\\n  --wavelength-dataset /SITE/Reflectance/Metadata/Spectral_Data/Wavelength \\\n  --red-band 14 --green-band 9 --blue-band 5\n```\n\nThe provider-neutral MCP equivalent is `configure_cube_view`.\n\n## Community providers\n\nThe maintained remote providers are built in, but the architecture is not\ninstitution-shaped. Install a trusted\nlanguage-neutral provider executable with:\n\n```bash\n./target/release/wilddatum provider install ./my-provider.json\n./target/release/wilddatum provider list\n```\n\nInstallation performs a protocol handshake and validates provider identity,\ncapabilities, response bounds, and declared HTTPS origins. Provider executables\nare trusted local code, not sandboxed plugins, and never receive credential\nvalues. See the [provider SDK](docs/PROVIDER_SDK.md) for the complete wire\ncontract, conformance fixture, and security model. The canonical\n[`DatasetRequest` v2](schemas/dataset-request-v2.schema.json) and\n[provider manifest v2](schemas/provider-manifest-v2.schema.json) schemas keep\nRI-native names inside adapters while `plan_materialization` exposes one typed\nMCP input across providers.\n\n## Known limitations\n\n- WildDatum is local stdio MCP today; remote Streamable HTTP and OAuth are not\n  implemented yet.\n- The alpha macOS executables are not Apple-notarized. macOS may require an\n  explicit first-run approval; release checksums still protect artifact integrity.\n- The browser explorer uses Rerun's WebGL renderer for reliable software and CI\n  support. Native Rerun remains the higher-ceiling surface for very large scenes.\n- Rerun exposes entity/instance selection events, but not one universal brush\n  protocol for every view. Explicit interval, map, raster, spectral, and row\n  selections use the same `record_selection`/`query_selection` path.\n- GeoParquet supports exact GeoDataFusion queries but not direct geometry logging\n  into Rerun yet; GeoPackage currently has inspection but no query adapter.\n- WildDatum-derived COPC indexes provide full-resolution spatial access but do\n  not yet contain a provider-quality multiresolution hierarchy.\n- Generic ERDDAP planning does not infer institution-specific station/location\n  dimensions or translate arbitrary polygons. Use typed tabledap constraints or\n  griddap slices. CF profile roles are preserved, but choosing the scientific\n  value/QC policy remains an explicit recipe rather than an automatic guess.\n- Linked profile rendering accepts up to eight displayed values from CSV/TSV,\n  Parquet/GeoParquet, Arrow IPC, or Feather, but rejects sources above 100,000\n  rows rather than streaming or tiling them. Rerun's raw-value 2D aspect can make\n  a narrow value range look horizontally compressed against a deep vertical range.\n- The pure-Rust NetCDF-3 adapter bounds whole-variable decoding because its\n  reader does not currently provide subset I/O.\n\n## Development\n\nBuilding from source requires Rust 1.95, Node.js 22, CMake, and a C/C++\ncompiler. Linux uses a vendored static D-Bus client for keychain access.\n\n```bash\ngit clone https://github.com/krnzt/wilddatum.git\ncd wilddatum\nnpm --prefix viewer/web-bootstrap ci\nnpm --prefix viewer/web-bootstrap run build\ncargo build --release\n./target/release/wilddatum setup\n```\n\nRun the complete validation suite with:\n\n```bash\ncargo fmt --check\ncargo clippy --workspace --all-targets -- -D warnings\ncargo test --workspace\nnpm --prefix viewer/web-bootstrap run check\nnpm --prefix viewer/web-bootstrap run build\nnpm --prefix viewer/web-bootstrap exec -- playwright install chromium\nnpm --prefix viewer/web-bootstrap run test:e2e\nnpm --prefix viewer/web-bootstrap audit --omit=dev\n```\n\nAn opt-in integration test exercises published NEON teaching subsets: a\n6,609,829-point LAS tile and a 500×500×107 HDF5 reflectance cube.\n\n```bash\ncurl -fL -o /tmp/neon-point-cloud.las https://ndownloader.figshare.com/files/7024955\ncurl -fL -o /tmp/neon-hyperspectral.h5 https://ndownloader.figshare.com/files/21754221\nNEON_POINT_CLOUD_FIXTURE=/tmp/neon-point-cloud.las \\\nNEON_HYPERSPECTRAL_FIXTURE=/tmp/neon-hyperspectral.h5 \\\ncargo test -p wilddatum-rerun --test official_neon_fixtures -- --ignored\n```\n\nThe maintained ERDDAP presets also have opt-in live drift checks. They search\nand inspect all three services, materialize a tiny redirected EMSO subset, and\nrun a bounded current Euro-Argo profile through CF discovery, materialization,\nrecipe validation, and Rerun rendering:\n\n```bash\ncargo test -p wilddatum-provider-erddap --test live -- --ignored\ncargo test -p wilddatum-provider-erddap --test argo_profile_smoke -- --ignored\n```\n\nBuild local MCPB and archive artifacts after the Rust and browser builds with\n`scripts/package-release.sh target/release/wilddatum dist macos-arm64 darwin`\n(substitute `linux-x86_64 linux` on Linux). The tag workflow builds and combines\nboth macOS architectures, verifies the Linux linkage, publishes checksummed\nrelease assets, and submits the generated `server.json` using GitHub OIDC.\n\n## Project status\n\nWildDatum is an early public alpha. The scientific data model, provider contract,\nand Rerun boundary are designed for extension, but APIs and packaging may still\nchange before the first stable release.\n\nContributions are welcome from ecological researchers, data stewards, Research\nInfrastructure teams, visualization developers, and scientific-format experts.\nGood first collaborations include representative metadata fixtures, format\nadapters, selection semantics, accessibility, and reproducible ecological\ndemonstrations. Read [CONTRIBUTING.md](CONTRIBUTING.md) and open an issue before\nstarting a large provider or viewer change.\n\nWildDatum is MIT licensed. Rerun is used under its MIT/Apache-2.0 license; built\nbundles retain the required notices in [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).\n",
  "bytes": 22240,
  "sha": "e264516013a7a0dc94f6d6677bd675ac6526100e094f9e691d426650d9b7d91f",
  "repo_slug": "krnzt/wilddatum",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_krnzt_wilddatum_3d88cd4d/readme"
}