{
  "markdown": "# MapSmith\n\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.22213091.svg)](https://doi.org/10.5281/zenodo.22213091)\n\n[![CI](https://github.com/mapsmith-ai/MapSmith/actions/workflows/ci.yml/badge.svg)](https://github.com/mapsmith-ai/MapSmith/actions/workflows/ci.yml)\n[![PyPI](https://img.shields.io/pypi/v/mapsmith)](https://pypi.org/project/mapsmith/)\n[![Container](https://img.shields.io/badge/ghcr.io-mapsmith--ai%2Fmapsmith-2496ED?logo=docker&logoColor=white)](https://github.com/mapsmith-ai/MapSmith/pkgs/container/mapsmith)\n[![MCP](https://img.shields.io/badge/Model_Context_Protocol-server-654FF0)](https://modelcontextprotocol.io)\n[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)\n\n**Professional-grade GIS geoprocessing for AI agents — with provenance you can verify.**\n\n**[mapsmith.dev](https://mapsmith.dev)** — a real terrain analysis and the manifest that came\nwith it. Both are build products: the figure is rendered from GeoTIFFs MapSmith writes, so the\npage cannot drift from what the software does.\n\nMapSmith is an open-source [MCP](https://modelcontextprotocol.io) server that gives an AI\nagent real GIS analysis — buffers, overlays, reprojections, zonal statistics, terrain and\nhydrology — executed by GeoPandas, DuckDB Spatial, exactextract and Whitebox Workflows,\nnever written by the model. Every dataset it produces lands on disk next to a lineage\nmanifest: inputs with checksums, the exact parameters, the CRS decisions and *why*, engine\nversions, and the deterministic checks that ran on the result.\n\n> Ask for the result. The agent picks the tools. You can check the work afterwards.\n\nThe manifest is a [specified format](https://github.com/mapsmith-ai/manifest-spec), not\nMapSmith's private output: JSON Schema, a toolchain-free validator, a conformance suite, and a\nhundred-line emitter that never imports MapSmith. Records carry `spec_version`, and CI validates\nreal MapSmith output against the spec's own validator. The specification is archived and citable\nas [10.5281/zenodo.22205213](https://doi.org/10.5281/zenodo.22205213).\n\nEvidence before promises: an [A/B on GABench](docs/benchmarks.md) whose headline is a null\nresult — with the analysis that took our own positive number apart — a correctness suite in\nits own organisation, [**Argleton**](https://argleton.org), whose published run grades\nMapSmith on thirty-one traps with answers computed on paper and has already sent six defects\nback here, [notebooks](examples/) on a real USGS DEM of Mount St. Helens, an\n[in-chat map panel](#see-results-inside-the-chat) that shows the verification status of\nevery layer it draws, and a\n[measurement of our own tool discovery](#finding-the-right-operation) that retracted two numbers\nthis page had already published — including the one in the bullet list below.\n\n## Quickstart\n\nAdd MapSmith to any MCP client over stdio (Claude Desktop, Claude Code, Cursor, VS Code):\n\n```json\n{\n  \"mcpServers\": {\n    \"mapsmith\": {\n      \"command\": \"uvx\",\n      \"args\": [\"mapsmith\"]\n    }\n  }\n}\n```\n\nDocker is the supported path, and confines the server to the directory you mount:\n\n```json\n{\n  \"mcpServers\": {\n    \"mapsmith\": {\n      \"command\": \"docker\",\n      \"args\": [\"run\", \"-i\", \"--rm\",\n               \"-v\", \"/absolute/path/to/your/data:/data\",\n               \"-e\", \"MAPSMITH_WORKSPACE=/data\",\n               \"ghcr.io/mapsmith-ai/mapsmith\"]\n    }\n  }\n}\n```\n\nOne-click installs:\n\n[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/install-mcp?name=mapsmith&config=eyJjb21tYW5kIjoidXZ4IiwiYXJncyI6WyJtYXBzbWl0aCJdfQ%3D%3D)\n[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_MapSmith-0098FF?logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=mapsmith&config=%7B%22command%22%3A%22uvx%22%2C%22args%22%3A%5B%22mapsmith%22%5D%7D)\n\nor from a terminal: `code --add-mcp '{\"name\":\"mapsmith\",\"command\":\"uvx\",\"args\":[\"mapsmith\"]}'`\n\nTo check it runs before wiring a client, `uvx mapsmith` starts the server on stdio\n(Ctrl-C to quit) — it speaks MCP, not a CLI, so a silent prompt means it is working.\n\nThis page describes **0.4.0**, which is what that command installs. When `main` runs ahead of\nthe published artifact this paragraph says so and names the difference — a reader should never\nhave to find out by calling a tool that is not there.\n\nThen ask your agent things like:\n\n> \"Take parcels.gpkg, keep only the parcels within 300 m of the river in rivers.gpkg, and\n> give me the result with the analysis lineage.\"\n\nThe Docker image includes the `[raster]` and `[whitebox]` extras. With `uvx`, pick your\nown: `uvx --from \"mapsmith[raster,whitebox]\" mapsmith`. **Docker — or `uvx` on a machine\nwith working wheels — is the only supported installation path**: geospatial native\ndependencies across three OSes are a support black hole, and issues about broken local\nenvironments will be redirected here.\n\nTwo things about the image, because they change what happens on your machine: it sets\n`MAPSMITH_WORKSPACE=/data` itself (the `-e` above is explicit, not required) and runs as\nuid 1000, so pass `--user $(id -u):$(id -g)` if the directory you mount belongs to another\nuser; and it is built for amd64 only, so on Apple Silicon it runs under emulation.\n\n## What you get back\n\nEvery dataset comes with the file below, written next to it as\n`<output>.provenance.json` — enough to re-run the analysis without the model that asked\nfor it:\n\n```json\n{\n  \"spec_version\": \"1.0.0-draft.3\",\n  \"producer\": {\"name\": \"mapsmith\", \"version\": \"0.4.0\"},\n  \"operation\": \"buffer_layer\",\n  \"parameters\": {\"distance_meters\": 300.0},\n  \"inputs\": [{\n    \"path\": \"rivers.gpkg\",\n    \"sha256\": \"b24b884f49eee431133d443d842557d3ed21f3978e2c4b1e8e072fd1240effe8\",\n    \"crs\": \"EPSG:4326\"\n  }],\n  \"crs_decisions\": {\"analysis_crs\": \"EPSG:32632\", \"reason\": \"estimated UTM zone for metric buffering\"},\n  \"engine\": {\"name\": \"geopandas\", \"version\": \"1.0.1\"},\n  \"environment\": {},\n  \"verification\": [\n    {\"name\": \"crs_matches\", \"passed\": true, \"detail\": \"expected EPSG:4326, got EPSG:4326\"},\n    {\"name\": \"feature_count_exact\", \"passed\": true, \"detail\": \"expected 1, got 1\"}\n  ],\n  \"started_at\": \"2026-08-18T10:15:03Z\",\n  \"finished_at\": \"2026-08-18T10:15:04Z\"\n}\n```\n\nThis block is checked against the specification's own validator by\n`tests/test_showcase.py`, because the page that says records carry `spec_version`\nhad an example without one for two releases — and this is the record a third-party\nimplementer copies.\n\nTrimmed for the page, not for the file: the real record also carries the output's own path\nand hash, any geometry MapSmith had to repair, and the notes it made about how the inputs\nwere handled — and every check says whether it was critical and, when it failed, what to do\nabout it. `get_provenance` returns it for any output.\n\n## Why MapSmith\n\n- **Real geoprocessing, not map CRUD.** Built on the proven open geospatial stack: GDAL,\n  GeoPandas, Shapely, DuckDB Spatial, Whitebox Workflows and exactextract ship today\n  (more to come: QGIS Processing via sidecar).\n- **Provenance by design.** Every layer MapSmith produces ships with a machine-readable\n  lineage manifest — source datasets with checksums, tools executed, exact parameters, CRS\n  decisions, software versions, timestamps. Everything needed to re-run the analysis\n  without the LLM is in there. No AI slop.\n- **The engines compute, the model orchestrates.** Geometry and numbers only ever come\n  from deterministic tool executions — never from model output.\n- **Semantic tools, not a tool dump — and a catalog built for thousands.** 28 goal-level\n  tools plus a searchable operation catalog, because tool-selection accuracy degrades once\n  a few dozen tools are exposed at once, and fastest when two of them apply to the same\n  input. Capability count has no such ceiling, so capability lives in the catalog. Search\n  **narrows** it on what you declare and then hands over what survives rather than ranking it\n  for you, because measurement said ranking is the wrong verb — see\n  [Finding the right operation](#finding-the-right-operation).\n- **Model-agnostic infrastructure.** Claude, GPT, Qwen, Kimi, GLM — anything that speaks\n  MCP, cloud or local. The leverage is better contracts (typed plans, actionable error\n  codes, a searchable catalog), not weights we would have to maintain. See\n  [the manifesto](MANIFESTO.md).\n\n## Tools\n\n| Tool | What it does |\n|---|---|\n| `describe_dataset` | CRS, schema/bands, extent, nodata and statistics of any vector or raster dataset |\n| `buffer_layer` | Metric buffer with automatic UTM estimation for geographic CRS |\n| `clip_layer` | Clip a layer with a mask layer |\n| `overlay_layers` | Set-theoretic overlay (intersection/union/difference/…); dropped lower-dimension pieces are declared in the manifest |\n| `dissolve_layer` | Merge features per key; the aggregation is recorded in the manifest and the group count verified |\n| `nearest_join` | Nearest neighbour with the distance in meters, UTM-measured on geographic CRS (decision recorded) |\n| `explode_layer` | Multi-part to single-part, with the part count verified in closed form |\n| `measure_area` | Area in m², always: ground on the ellipsoid, or planar converted with the CRS's own declared linear unit (survey feet are not metres). Invalid rings repaired *before* measuring, and a plane that is not equal-area here comes back with the ratio against the ground area |\n| `merge_layers` | Append layers (schema union); null-filled columns are named in the manifest, the count verified against the sum |\n| `simplify_layer` | Douglas-Peucker with the drift measured: area/length before and after recorded in the manifest |\n| `centroid_layer` | Geometric centroids computed in a metric CRS, never on degrees (decision recorded) |\n| `convert_format` | Convert between GeoParquet/GeoPackage/GeoJSON by output extension, re-read and verified (count and CRS). Two conversions are refused with the reason rather than performed: shapefile output, which truncates field names to 10 characters silently, and GeoJSON for a non-WGS84 layer |\n| `reproject_layer` | Reproject to any CRS (EPSG code or WKT) |\n| `spatial_join` | Join by spatial predicate, auto-routed to the fastest engine (SedonaDB > DuckDB > GeoPandas) |\n| `run_sql` | Spatial SQL (DuckDB dialect) over GeoParquet and GDAL formats |\n| `zonal_statistics` | Raster statistics per vector zone with exact fractional pixel coverage (`[raster]` extra) |\n| `hillshade` | Shaded relief from a DEM, in-memory Whitebox engine (`[whitebox]` extra) |\n| `slope` | Slope gradient from a DEM in degrees, percent or radians; geographic-CRS DEMs refused (`[whitebox]` extra) |\n| `aspect` | Downslope azimuth from a DEM, 0 = north; flat cells are −1, not nodata (`[whitebox]` extra) |\n| `flow_accumulation` | D8 flow accumulation with automatic depression filling (`[whitebox]` extra) |\n| `watershed` | Watershed delineation from a DEM and pour points (`[whitebox]` extra) |\n| `preview_map` | Interactive in-chat map (MCP Apps) of any datasets, with a provenance card and verification status per layer |\n| `validate_plan` | Statically validate a multi-step plan before running anything: operations, arguments, references, input files, simulated CRS flow |\n| `execute_plan` | Validate then run a plan step by step, with per-step provenance and a plan-level manifest |\n| `get_provenance` | Return the full lineage manifest of any MapSmith output |\n| `list_operations` | Catalog search: narrows on what you declare, then returns the surviving set to choose from (`status: \"choose\"`) or a ranking by `engine` — BM25, embeddings, or auto; `detail=true` returns parameters and worked examples |\n| `run_operation` | Run any catalog operation by name, including those with no tool of their own; arguments validated against the catalog before anything runs |\n| `server_info` | Version, license, available engines |\n\n### Finding the right operation\n\nThose are the tools an agent chooses between. Behind them the **catalog** holds every\noperation MapSmith can perform — 74 today, and 49 of them have no tool of their own — and\nit is built to hold thousands. (Two of the 74 are marked `planned` and say so when asked:\nthe roadmap is in the catalog on purpose, so an agent can answer \"not yet\" instead of\ninventing a call.)\n\nThe split is the design: tool-selection accuracy degrades past a few dozen *exposed* tools,\nwhile capability count has no such ceiling. That makes reaching scale a retrieval problem, so\nit is treated as one — and measured like one.\n\n**First it narrows, deterministically, on things the caller already knows.** Every entry\ndeclares what data it takes (`vector`, `raster`, `dataset`, `plan`, `none`), what it hands back\n(`dataset:vector`, `dataset:raster`, `answer`, `description`), whether it demands a projected\nCRS, and which family it belongs to.\n\nMeasured over 118 answerable requests written by two other model families from job scenarios — a\nhydrologist with a flood report, a surveyor arguing with a field measurement — neither of which\nwas shown this catalog, because a model handed the entry writes a paraphrase of the entry:\n\n| what the caller declares | candidates left | BM25, found@3 | embeddings, found@3 | **right answer in what comes back** |\n|---|---|---|---|---|\n| nothing — words alone | 74 | 31% | 18% | 31% |\n| what data I have | 48 | 32% | 21% | 33% |\n| + what I want back | 30 | 45% | 38% | 53% |\n| **+ how many datasets I have** | **16** | **58%** | **53%** | **98%** |\n\n**Two ranking columns, and that is a correction.** This table used to carry one,\ncomputed with the default engine — which is the embedding one where its model\nloads and BM25 where it does not. So the published figures were a measurement of\nwhat the machine could download, and a CI run that met a 429 from Hugging Face\nrecomputed the first row as 28% where this page said 18%. Not a flaky test: a\nnumber that had never been reproducible on a machine without the model,\npublished under a sentence promising it could be checked.\n\nThe two also differ in a way worth seeing, and this page had it backwards\nuntil 2026-08-30. It said the embedding engine overtakes BM25 once the facets\nhave narrowed. It does not overtake it anywhere: BM25 leads at every row of the\ntable above, by seven to twelve points, and the gap is widest at the fullest\ndeclaration. An exact term either matches or does not, and the entries that\nsurvive a full declaration are told apart by the words that distinguish them —\nwhich is what `distinguishes` is for. The embedding engine earns its place on\nthe phrasings it has never seen, not on the ranking once the set is small.\n\nThe last column is not an accuracy figure — it is a property, and the 98% rather than 100% is\nworth a sentence. The narrowing never drops the right operation: that is asserted per entry and\nholds for all 74. What the column measures is whether the surviving set was small enough to hand\nover WHOLE, and for a handful of requests it still is not, so those fall back to a ranked\nshortlist and the answer can be outside the top three. Ranking decides the order; it does not\ndecide membership; and the 3% is the gap between \"cannot lose the answer\" and \"can show you all\nof it\".\n\n**The third row is the scaling wall, and we hit it in one afternoon.** On 2026-08-29 the\ncatalogue went from 51 operations to 61. Two rows of that table got worse: the commonest\nsurviving set went from 26 candidates to 34, past the point where the whole set can be handed\nover, and *delivered* fell from 100% to 45% while found@3 fell from 48% to 36%. **Adding\ncapability had made discovery worse** — the failure this page had predicted at eight hundred\noperations and met at sixty-one.\n\nRaising the threshold would have postponed it by about ten operations. What fixed it is the\nfourth row: how many datasets you are holding. That is a fact about your situation — one layer\nor two — not a guess about our vocabulary, it is derivable from each operation's own signature\nso a test can check the declaration against the code, and it takes the median surviving set from\n34 to 9. The catalogue grew by a fifth and discovery got better, but only because a facet\narrived with it. That is the trade this design makes, stated rather than discovered later.\n\n**It happened again the next day, and this is what watching a curve is for.** On 2026-08-30 the\ncatalogue went from 61 operations to 71. Every ranking figure in that table fell — 28% to 25%\nbare, 34% to 27% on the input kind — and the *delivered* column of the second row fell from 48%\nto 27%, because more requests now leave a set too large to hand over whole. The bottom row did\nnot move: **97%, the same as at 61.** Ten more operations, no new facet, and the guarantee held,\nwhich is the first time growth has been absorbed by the facets already there. (Not «and at 51»:\nthat bottom row is the arity facet, and `dataset_inputs` did not exist at 51 operations. The 100%\nquoted above at that size is the row above it. Two different rows under one sentence is the kind\nof comparison this page exists to refuse.)\n\n**And again at 74, with two operations that a caller is unusually likely to want.** `select_features` and `extract_layer` are the remedies MapSmith's own error messages had been recommending, so they sit in the busiest corner of the facet space: the average surviving set went from 16 to 17 and the second row's *delivered* did not move. The bottom row held at 97% for the third catalogue size running. The margin to the wall is now 13. *(Those are the figures as measured on 2026-08-31. They were recomputed on 2026-09-01 against human answers — see the note under the table — which moved them without any catalogue change: the surviving set reads 16 and the bottom row 98%. A paragraph about a transition keeps the figures of the transition.)*\n\nThat is the shape of the trade, and it says when the next facet is due. The figure to watch is\nnot found@3 — a ranker will always get worse as the catalogue grows, and it is a hint. It is the\n**average** surviving set at the fullest declaration, the fourth column of that table:\n9 at 51 operations, 14 at 61, 16 at 72, 16 at 74. When that crosses 30, delivery stops being a\nproperty and starts being a ranking again, and the answer is another fact the caller already\nknows, not a bigger threshold. (It said *median* until 2026-08-29, and published the mean:\nthe median at 74 is 14. The distribution is skewed — most requests leave a small set and a few\nleave a large one — so the two numbers say different things and the mean is the pessimistic\none, which is the right one to watch.)\n\nThe requests, both labels and the harness are all in the repository:\n[`tests/data/discovery_queries.json`](tests/data/discovery_queries.json) and\n[`benchmarks/discovery_report.py`](benchmarks/discovery_report.py), which recomputes every\nnumber above from those files with no network and no model — so they can be checked rather than\nbelieved, and `tests/test_discovery_report.py` fails if this page and the harness disagree. The\none exception is the 69%: reproducing that needs the model that did the choosing, and the report\nsays so where it stops.\n\n**So it hands over the set instead of picking for you.** Below thirty survivors `list_operations`\nanswers with `status: \"choose\"`: every candidate, ordered as a hint that says it is a hint, each\ncarrying the sentence that separates it from its neighbours. The threshold is 30 because that is where the\nsurviving set almost always sits: over those 118 requests its median is 14 and it exceeds 30\nfor two of them — which is the 98% in the table above, seen from the other side. The payload\nis about 2,100 tokens, less than one wrong operation costs to run and undo.\n\n*(That sentence used to say the set had a median of 26 and never exceeded 30. It was false\nbefore this catalogue reached 72 operations and nobody noticed, because the test that checks\nthis page against the harness read the table and not the prose around it. It reads both now.)*\n\nThree measurements say this is the right shape, and the third is the one that settles it:\n\n| | |\n|---|---|\n| our ranking puts the answer in the top three | **58%** |\n| a model handed the same candidates and asked to *choose* gets its first pick right | **69%** |\n| the two labellers who wrote the ground truth agree **with each other** | **70%** |\n\n**These figures went up on 2026-09-01 because the measurement changed, not because the ranker\ndid.** Somebody who does this work answered all fifty of the requests the two model labellers\nhad disagreed on, and a request can have more than one acceptable answer: two experienced\nanalysts reach the same result with different tools. So a hit is now counted against every\noperation a professional would accept rather than against one label, and found@3 rose four to\nfive points. Nothing in the ranking code changed. Read the other way round, the older figures\nwere understating by that much — they scored a system as having failed when it returned the\nother defensible answer — and the honest description of this table is *the answer is among the\nones a professional would accept*, which is a property, not an accuracy.\n\nWhere the secondary answers came from is recorded rather than smoothed over: the primary on\neach of those fifty is a human choice, and the secondaries were proposed by a third model and\nadopted wholesale rather than judged one at a time. Two different strengths of evidence, and\nthe file says which is which.\n\n**The two model figures are dated: the labels were written on 2026-08-28, against a catalogue of\n51 operations.** It now has 74, so for any request whose right answer is one of the 23 added since,\nneither labeller *could* have been right — the answer was not in the catalogue to name. Measured on\nthe first four requests a person has answered by hand, two of the four have both labellers wrong,\nand both of those two name operations that did not exist on the 28th. So \"both labellers wrong\" and\n\"both labellers chose badly\" are not the same number, and the honest thing is to say when the labels\nwere made rather than to quietly benefit from the difference. Human answers are replacing them one\nat a time — `benchmarks/ingest_answers.py` is how they get in, and every figure above is computed\nagainst a human answer where there is one.\n\nAll three are over the same 118 requests, which matters: agreement measured over all 155 requests\nin the file is 68%, and the difference is the 21 pairs where both labellers agreed a request was\nunanswerable — true, and the easy half. Quoting that 68% beside a 58% computed over the 118 would\nbe comparing two populations, which this table did for half a day.\n\n**The last row is a ceiling, not a baseline**, and the second row sits at it rather than below it.\nWhen two competent labellers disagree three times in ten about which operation answers a request,\n\"the right one\" is not a single value to rank toward, and a system scoring above that is fitting\none annotator rather than getting better. Two GIS analysts with thirty years each do the same job\nwith different tools and neither is wrong.\n\nThat is why the answer is a set and why its `reason` field says, in words, that the order is a\nhint and that **two defensible candidates are a question for the person who made the request**.\nThe caller — an agent with the conversation in context — knows things no ranking can. Where it\ndoes not, the human does.\n\nThe remaining honesty: the ceiling was measured between two language models. Whether human GIS\nanalysts agree with each other more, less, or about the same is unmeasured, and until it is, these\nnumbers are reported as *agreement with model-written labels* and never as accuracy.\n\n**The family is the one facet that orders instead of filtering, and that is a correction.** It\nused to be a hard filter like the others. It is not like the others: input kind and projected-CRS\nare facts about the data in hand and output kind is what the caller wants, but *family* is a guess\nabout our taxonomy, which the caller cannot see. Measured, it removed six candidates out of\nsixteen — and when the guess was wrong it removed the right operation, with no error, leaving a\nconfident answer assembled from neighbours. Every request in the independent set has 4.4 plausible\nfamilies. That is the silent-failure class [Argleton](https://argleton.org) measures in other\npeople's systems, sitting in our own discovery layer, so it now sorts: declaring the family lifts\nit to the front and costs positions when wrong, never the answer. The hard cut stays available on\n`catalog.applicable`, where asking for it means it.\n\n**We do not need a model to extract those facets, because the caller is one.** An MCP client is an\nLLM with the context we lack — it knows what file it is holding and what it is trying to produce.\nSo `list_operations` asks for them in its schema, and its description leads with why. This is the\nsame shape as LlamaIndex's Auto-Retrieval or LangChain's Self-Querying, minus the model those have\nto host: here it is already on the other end of the protocol. A geographic raster is never offered\n`slope`, because `slope` refuses one — a property of the data, checked in code, no model in the\nloop.\n\n**Then it ranks, with two engines that both always run.** `list_operations` takes `engine`:\n`auto` (the default), `lexical`, or `vector`. Every result carries the engine that produced it,\nbecause a BM25 score of 10.03 and a cosine of 0.38 are not on the same scale.\n\n| engine | what it is | what it guarantees |\n|---|---|---|\n| `auto` — **the default** | The embedding engine, falling back to BM25 when the model cannot be loaded | An answer on a machine with no network, and a field saying which engine gave it |\n| `lexical` — words | Okapi BM25, ~40 lines, no model and no network ever | Identical scores on every machine; term-sorted accumulation, because float addition is not associative |\n| `vector` — meaning | Static embeddings — a token lookup plus pooling, no transformer, no GPU. Model **revision pinned in the source**, 512 dimensions, ~130 MB fetched once | Bit-identical across calls in one process (measured, multiprocessing off), with the vectors pinned by a golden-vector test — so a change in the model, the tokenizer or the pooling fails a test instead of an analysis |\n\n**The default was lexical until the measurement said otherwise**, and the measurement is the\ninteresting part. Golden queries written by whoever wrote the catalog share its vocabulary, so\nthey test word overlap dressed as retrieval: on those, BM25 scores 100% found@1 and embeddings\n60%. Re-phrased the way somebody with a problem actually phrases it — *\"the coastline is 400000\nnodes and the browser dies\"* rather than *\"simplify the geometry\"* — the finding reversed, on a\ncatalogue of fifty-one entries. It has since reversed back, and both engines degrade as the\ncatalog grows:\n\n| catalog size | BM25 found@3 | embeddings found@3 |\n|---|---|---|\n| 10 | 77% | 80% |\n| 30 | 65% | 58% |\n| 74 | 50% | 40% |\n\n**This table used to say the opposite, and the reversal is the finding.** Published at 10/30/51\nit read 78/83, 47/65, 40/55 — embeddings ahead at every size — and the sentence under it said\nBM25 degrades faster, which is why the embedding engine became a dependency rather than an extra.\nRecomputed today the crossover has moved: embeddings still lead on ten entries, and from thirty\nup BM25 leads by a margin that widens with size. Part of that is the catalogue itself, because\nthe distractors are drawn from it and it has grown from fifty-one entries to seventy-four — which\nis the point rather than a caveat. **The near-neighbour effect the eight-hundred-operation test\npredicted has arrived in our own catalogue**, and the two tables that used to disagree now agree.\n\nThe curve is recomputed by `tests/test_retrieval_degradation.py` and compared with this table, so\nit cannot go stale again in silence — which it did for three catalogue sizes.\n\n**And that finding does not survive being scaled up — measured the same day it was\npublished.** The distractors above are drawn from our own seventy-four entries, which are\nsemantically spread out. Growing this catalog means adding *near neighbours*: hundreds of\nraster and terrain operations that resemble each other. Re-run against 800 real GIS\noperations, taken from a library that ships them with their own descriptions, the ranking\nreverses and the embedding engine degrades **faster**:\n\n| catalog size | BM25 found@3 | embeddings found@3 |\n|---|---|---|\n| 74 — our own entries, no foreign distractors | 50% | 40% |\n| 200 | 48% | 25% |\n| 800 | **35%** | **20%** |\n\nEmbeddings blur near neighbours; an exact term either matches or does not. These two\nmeasurements used to disagree, and both were kept because they answered different questions:\nwhich engine suits the catalog we have, and which survives the catalog we plan. **They agree\nnow** — the near-neighbour effect this one predicted has arrived in our own catalogue, so BM25\nleads at both scales, and the second question has the answer *neither*. The embedding engine is\nstill the default, and the measurement that made it one no longer says so: that is a decision to\ntake rather than a number to quietly restate. At 800 entries the better engine is wrong two\ntimes in three, so **scale will not be bought by choosing a better ranker**.\n`test_retrieval_at_scale.py` keeps the projection under measurement rather than under opinion.\n\n**And the narrowing does not scale on its own either — this page claimed otherwise and was\nwrong.** It said the facets leave sixteen candidates at 800 operations just as they do at 200. The\nsixteen is real and it is produced almost entirely by *family*: those 803 operations are all\nraster-in, raster-out, so input kind and output kind cut nothing at all, and only the taxonomy\ndoes — a choice among 43 families that the caller has to guess. Which is exactly the facet that\nmust not filter.\n\nSo the open problem has a sharper shape than \"ranking is hard\". What is needed at a thousand\noperations is **more facts a caller can state without knowing our taxonomy** — how many inputs an\noperation takes, whether it changes geometry or only attributes, whether the output has the same\nnumber of features as the input. Those are structural properties of the operation, they are\ncheckable against the code rather than declared by hand, and they separate the pairs a bag of\nwords cannot: `spatial_join` from `overlay_layers`, `flow_accumulation` from `extract_streams`.\nThat work is not done, and until it is, the honest claim is the measured one: the guarantee above\nholds at seventy-four operations, not at eight hundred.\n\n**How an entry has to be written is a published specification**, not a convention:\n[`docs/catalog-entry-spec.md`](docs/catalog-entry-spec.md), with a normative\n[JSON Schema](schema/operation.schema.json) that every entry validates against in CI. Each field\nis there because a measurement said so — including the two that measured to nothing and are\ndocumented as such, because a spec that only reports what worked is an advertisement.\n\n**And discoverability is a contract per operation, not an average.** A catalog-wide 90% found@3\nover fifty entries means five are invisible and the average will not say which. So every available\nentry is probed with its own first worked example, with its own facets declared\n(`test_discovery_contract.py`, parameterised over the catalog, so a new operation is under\ncontract the moment it is added). Two things are required of it: the facets the entry declares\nmust never drop that entry, and the entry must reach the caller.\n\n**Its rank is no longer one of them, and removing that is the point.** The contract used to demand\nthe top three. That looks like a discovery contract and is a ranking contract, with one bad\nproperty: the only way to repair a failure is to reword the entry until the ranker likes it. Fifty\nentries tuned that way score nineteen points better on examples we wrote than on requests written\nby anyone else — that gap is measured, and it is where a published 70% on this page turned into\n51% overnight. A test whose repair procedure is *fit the text to the scorer* manufactures the\nnumber it reports. What remains under contract is the part that is deterministic and ours; rank\ninside the delivered set is still measured, and no longer fails a build.\n\nThe old form still earned its place the first time it ran. `centroid_layer` advertised *“label\npoints for a polygon layer”* and ranked below `point_on_surface`. The ranking was right: a centroid\ncan fall outside its own polygon, which is [Argleton](https://argleton.org) trap 014 — our catalog\nwas recommending the defect our own suite measures. The example changed, not the score.\n\n**And when the two engines agree on nothing, the search says so instead of answering.** This is\nthe failure that measurement turned up in our own product: asked *\"send an email to my\naccountant\"*, the embedding engine returned `idw_interpolation` with the same confidence as a\nreal answer — a silent error in the layer whose job is to prevent them. A similarity threshold\ndoes not fix it, because there is no line to draw: *\"convert this mp4 to a gif\"* scores above\nsixteen of twenty genuine queries. What does separate them is the two rankers landing on\n**nothing in common** — mean top-3 overlap 0.90 of 3 when an answer exists, 0.18 when it does\nnot. So a query the catalog cannot place comes back as `status: \"unsure\"`, carrying both\nengines' guesses and the question that narrows the catalog deterministically: what kind of data\ndo you have. It fires on 9 of 11 unanswerable queries and suppresses 1 correct answer in 20.\n\n**And when the facets leave nothing at all, it says which declaration did it.** Zero\ncandidates used to fall through the branch above and come back as *\"0 operations survive,\nwhich is few enough to read\"* — prose that means nothing and, worse, an empty candidate\nlist, which an agent reads as *MapSmith cannot do this*. It was found by the discovery log\nbelow on its first real session: *\"how much land is in each of these parcels\"* with\n`produces=\"answer\"` left nothing, while `measure_area` computes exactly that and declares\n`dataset:vector` because it writes the areas into a column. So that case is now its own\nanswer — each declaration with the number of operations that would survive without it,\nsmallest first — and it is arithmetic, not ranking.\n\nBelow the choose threshold it stops refusing and becomes a warning instead — `order_is_weak` on\nthe delivered set. Refusing made sense while the search was deciding; handing over every candidate\nis not deciding, so the disagreement reverts to being evidence about the *order*, which is the only\nthing it was ever evidence about.\n\nThe applicability filter above runs first for **both** engines — otherwise the guarantee would\nonly be true of one of them, and there is a test that says so.\n\n**Then it runs, tool or no tool.** Most catalog operations have a tool of their own; the newer\nones increasingly do not, and `run_operation(operation, arguments)` runs those by name. This is\ndeliberate: capability count has no ceiling, but the *exposed tool list* has one, so the catalog\nis allowed to grow faster than the tool list. Arguments are checked against the catalog before\nanything executes — unknown operation (with a \"did you mean\", from the same ranking), missing or\nmisnamed argument, wrong type, path outside the workspace — and every error carries a stable\ncode. Execution goes through the same path as `execute_plan`, so an operation cannot behave one\nway alone and another way inside a plan.\n\nBoth engines embed the *identical* document text (`catalog.document_text`), so a comparison\nbetween them measures the ranking and nothing else. Three test files keep the rest under\nmeasurement rather than under opinion: the degradation curve over our own catalog, the projection\nagainst 800 real neighbouring operations, and a discoverability contract per entry. That is what\nturns the scaling limit into a curve you can watch rather than a number someone guessed.\n\nDeterminism is the reason for building it this way rather than reaching for a hosted\nembedding API: that would make tool discovery a network call whose answer can change under\nyou, and an agent that finds a different tool tomorrow for the same question is not\nreproducible, whatever its manifest says. The one network access left is the model download\non first use, at the pinned revision; after that the vector engine is local, and an install\nthat never makes it keeps BM25, and the `engine` field of every result says which one\nanswered.\n\n#### Making it better with your own requests, without a model that drifts\n\nThe 155 requests behind those percentages were written by two language models. They are\nthe best set we could build without users, and they are not what users ask: a real request\nnames the file somebody actually has and the words their field actually uses.\n\nSo MapSmith can record its own. Set `MAPSMITH_DISCOVERY_LOG` to a file path and each\nsearch is written as one JSON line together with the operation that was run after it —\nthe query, the facets declared, which engine ranked it, every candidate delivered, and\nwhere in that list the chosen one sat:\n\n```bash\nMAPSMITH_DISCOVERY_LOG=/data/discovery.jsonl   # then work normally for a while\npython benchmarks/log_to_cases.py /data/discovery.jsonl\n```\n\nFor the part that needs eyes rather than a pipe, there is a dashboard — see below.\n\n`log_to_cases.py` prints those lines as rows shaped like `tests/data/discovery_queries.json`\nand flags the two that matter: a run the ranking did **not** put first (the answer was on\nscreen and the order was wrong) and a search nothing followed (a request the catalog did\nnot serve). It prints; it never writes. Which rows become test cases is a person's call.\n\n**None of this trains anything, and that is the design.** A ranker that learns from what\ncallers pick learns from an ordering it produced: the operation shown first gets picked\nmore, gets learned as correct, gets ranked first harder — a confident answer nothing\ncontradicts, which is the exact failure this product exists to measure. The model revision\nstays pinned, held there by a golden-vector test, so the same query gets the same answer\nnext year. What improves instead is the catalog text — a phrasing, a `distinguishes` that\ndoes not distinguish — as a diff somebody can read and revert. That loop is not the weak\noption: it is what took found@3 from 18% to 58% and delivery to 98%.\n\nThe log is off unless the variable is set, holds queries and operation names and nothing\nelse (no dataset paths, no arguments), is guarded by `MAPSMITH_WORKSPACE` like any other\npath MapSmith writes, and never leaves the machine — nothing reads it back. Your queries\ndescribe your work; treat the file that way, and delete it when you are done.\n\n### One question, end to end\n\nEverything above is about one step. Here is a whole question — six parcels, a river, an\nelevation grid, and five operations picked out of 74 — with the search, the arguments and\nthe verification of each step as they were actually recorded.\n\nNothing in this section is drawn. `benchmarks/worked_example.py` builds fixtures whose answer can\nbe worked out on paper, asks the catalogue in the words of the problem, validates and runs the\nplan, reads the manifests, and writes what follows; `tests/test_worked_example.py` fails if this\npage and that script disagree. The position column is BM25's rather than the default engine's,\nbecause a published figure should not depend on whether a model download succeeded on the machine\nthat built the page — the narrowing, which is the point, is identical on both. Two things worth watching: the middle column, where the catalogue\ngoes from 74 operations to a handful the caller can read; and the CRS column, where every\nmetric operation says which coordinate system it moved the data into and why.\n\n<!-- worked-example:start -->\n\n```mermaid\nflowchart TB\n  ASK[\"<b>Parcels within 1.5 km of the river whose mean ground elevation is at most 120 m, with the elevation and the ground area of each</b>\"]\n  ASK --> PLAN{{\"plan validated<br/>before anything runs\"}}\n  PLAN -. \"rejected: FORWARD_REFERENCE\" .-> BAD[\"'mask_path' references '$buffer' which runs later — move step 'buffer' before 'near'\"]\n  BAD:::bad\n  BUFFER[\"<b>buffer_layer</b><br/>74 operations &rarr; 29 candidates &rarr; chosen<br/>CRS EPSG:32610<br/>9/9 checks\"]\n  PLAN --> BUFFER\n  NEAR[\"<b>clip_layer</b><br/>74 operations &rarr; 14 candidates &rarr; chosen<br/>12/12 checks\"]\n  BUFFER --> NEAR\n  HEIGHT[\"<b>zonal_statistics</b><br/>74 operations &rarr; 4 candidates &rarr; chosen<br/>CRS EPSG:4326<br/>7/7 checks\"]\n  NEAR --> HEIGHT\n  AREA[\"<b>measure_area</b><br/>74 operations &rarr; 29 candidates &rarr; chosen<br/>CRS WGS 84 &#40;ellipsoidal&#41;<br/>10/10 checks\"]\n  HEIGHT --> AREA\n  FILTER[\"<b>select_features</b><br/>74 operations &rarr; 29 candidates &rarr; chosen<br/>CRS EPSG:4326<br/>10/10 checks\"]\n  AREA --> FILTER\n  OUT[[\"3 parcels, each with elevation and ground area\"]]\n  FILTER --> OUT\n  classDef bad stroke-dasharray: 4 3\n```\n\n| what the agent asks for | it declares | candidates | picked | at position |\n|---|---|---|---|---|\n| “everything within one and a half kilometres of the river” | vector, dataset:vector, 1 dataset(s) | **29** of 74 | `buffer_layer` | 2 |\n| “keep only the parcels that fall inside that strip” | vector, dataset:vector, 2 dataset(s) | **14** of 74 | `clip_layer` | 1 |\n| “how high is the ground under each of these parcels” | raster, dataset:vector, 2 dataset(s) | **4** of 74 | `zonal_statistics` | 3 |\n| “how big is each one on the ground” | vector, dataset:vector, 1 dataset(s) | **29** of 74 | `measure_area` | 1 |\n| “drop the ones where the ground is above 120 metres” | vector, dataset:vector, 1 dataset(s) | **29** of 74 | `select_features` | 2 |\n\n| step | operation | arguments that mattered | CRS decision, recorded | checks |\n|---|---|---|---|---|\n| buffer | `buffer_layer` | `distance_meters=1500` | `EPSG:32610` — estimated UTM zone for metric buffering on a geographic CRS | 9/9 |\n| near | `clip_layer` | `mask_path=$buffer` | — | 12/12 |\n| height | `zonal_statistics` | `zones_path=$near`, `stats=['mean', 'min']` | `EPSG:4326` — zones and raster share the same CRS | 7/7 |\n| area | `measure_area` | `input_path=$height`, `method=geodesic` | `WGS 84 (ellipsoidal)` — ground area computed on the ellipsoid the layer's CRS names; no map plane is involved, so no projection distortion enters | 10/10 |\n| filter | `select_features` | `input_path=$area`, `by=field_between`, `field=mean`, `maximum=120` | `EPSG:4326` — no CRS change: selecting rows does not touch coordinates | 10/10 |\n\nEvery step is inside the plan, the last one included: `select_features` took 4 rows and returned 3, with a manifest like every other write. This step used to run outside the plan, because the only operation that could answer it was run_sql — which takes its inputs inside a SQL string, declares zero datasets, and therefore cannot join the plan's dataflow. That boundary is deliberate and has not moved: substituting `$step` into arbitrary strings would be a grammar in which a planner assembles a path out of text. What changed is that it is no longer the only way to ask.\n\n**The answer**, which can be worked out on paper before MapSmith sees the files: the parcels are squares of 0.0015° at 46.2°N, so each is about 119 m by 167 m, and the elevation ramps west to east across the fixture.\n\n| name | mean | min | area_m2 |\n|---|---|---|---|\n| North Field | 104.85 | 104.14 | 19303.33 |\n| Mill Meadow | 110.51 | 109.8 | 19303.33 |\n| Old Orchard | 117.58 | 116.87 | 19303.33 |\n\n<!-- worked-example:end -->\n\nThe rejected plan is the honest half. Steps in the wrong order are the dominant failure class in\nthe agent benchmark, so the example includes one and shows what the validator says about it,\nbefore any file is touched. It earned that place while this was being written: the first version\nof the plan passed `distance_m` where the operation declares `distance_meters`, and the validator\nnamed the argument and listed the three it accepts.\n\n\n### Formats\n\n| Format | Read | Write |\n|---|---|---|\n| GeoParquet 1.0 / 1.1 — WKB plus `geo` metadata | yes | yes, every path |\n| **GeoParquet 2.0** — Parquet-native `GEOMETRY`/`GEOGRAPHY` logical types | yes, including files that carry no `geo` key at all | yes on the SQL path: `run_sql` writes **both** layers into one file |\n| GeoPackage, Shapefile, FlatGeobuf, GeoJSON, … | anything pyogrio/GDAL opens | via GDAL |\n| GeoTIFF / COG | yes | outputs of the `[raster]` and `[whitebox]` engines |\n\nGeoParquet [2.0](https://github.com/opengeospatial/geoparquet/releases) moves geometry\ninto Parquet's own logical types and makes the `geo` key optional, so \"a Parquet file with\ngeometry in it\" no longer implies that key. MapSmith reads the CRS from the logical type\nwhen it is the only place it exists — the spec default, an authority string,\n`projjson:<key>`, or the whole PROJJSON document inline, which is what DuckDB writes.\n`run_sql` emits both layers (`geoparquet_version 'BOTH'`), so one output file satisfies a\n2.0-native reader and a GeoPandas 1.x one; the GeoPandas writer path stays 1.x because\nGeoPandas 1.1 caps `schema_version` there.\n\nOne declaration is deliberately refused rather than guessed: `srid:<n>`. The spec defines\nit as a numeric identifier and names no authority — its own example is `srid:0` — so\nreading it as `EPSG:<n>` would be inventing a coordinate system and recording it as fact.\n\n### Choosing the stack, and never swapping it in silence\n\n`MAPSMITH_STACK` picks the geoprocessing stack once, at the start. The default is\n`opensource` — GDAL, GeoPandas, DuckDB, Whitebox — and needs no licence. `esri` routes to\nArcPy on a machine that has ArcGIS Pro installed, through a subprocess and files, because\nArcPy lives in Pro's own interpreter. MapSmith ships no part of it and takes no licence to\nlook: what a session can reach is read from the metadata the installer left on disk, and\n`server_info` reports it, so a caller learns it before planning five steps around it rather\nthan at the first failure.\n\nThe rule that makes the choice worth making is what happens at the edges. When the chosen\nstack cannot do something, MapSmith says which of three things is true — there is no such\ntool, this licence tier does not include it, or it would need an online service — because\nthose lead to three different decisions and one word for all of them leads to none. Where a\nroute exists and cannot run, the manifest names the engine that actually produced the\nnumbers and why the preferred one did not: an engine quietly replaced by another is a record\nthat is true and a number nobody chose.\n\n**One operation is routed today: `buffer_layer`.** Everything else runs on the open source\nstack whatever `MAPSMITH_STACK` says. That is the state of the wiring rather than a property\nof the design, and it is written here because the alternative was a defect: a table\ndeclaring three routes while one operation consulted the router meant requesting the stack\nran two of them elsewhere with nothing in the manifest saying so. It was fixed before this\nrelease by shortening the table, not the sentence.\n\nTwo things this is not. It is not a comparison: MapSmith calls what you have installed, and\nthis repository publishes no scores for anybody's engine but the ones\n[Argleton](https://argleton.org) grades in public. And it is not equivalence: two further\noperations were measured against both stacks on the same fixture and deliberately left\nunrouted, because matching geometry is not the whole story — on a dissolve the other stack\ndrops every attribute, so a pipeline that dissolves and then reads a column would find the\ncolumn on one stack and nothing on the other. That is a difference to record before it is a\nroute to offer.\n\n## Verification, in and out\n\nEvery tool that writes a dataset also writes `<output>.provenance.json` beside it and\nverifies its own work — CRS agreement, geometry validity, raster dimensions, count and\nextent invariants — recording the results in the manifest *before* raising anything, so\nthe audit trail survives the error.\n\nVerification runs on the way in as well. Before an operation touches your data, MapSmith\nchecks the failures that produce *plausible* junk: an input with no CRS is refused\noutright, because metric maths on unknown units is how a confidently wrong answer gets\nmade; an empty input, or two layers whose extents cannot possibly overlap, comes back as\na named warning with a hint — in the tool result, not only in the manifest, so the agent\nsees it instead of assuming success. (The join fast paths, DuckDB and SedonaDB, only ever\nreceive inputs that already share a known CRS; they verify their output and diagnose an\nempty join.)\n\nAn output whose geometry is *mechanically* broken — typically invalidity inherited from an\ninvalid input — is repaired deterministically: `make_valid`, at most two rounds, written\nto a temporary file and swapped in only once it is complete, and skipped rather than\nrisked where a rewrite could drop data (a multi-layer GeoPackage is refused, not\nrewritten — `extract_layer` copies the one you mean into its own dataset, with the\ncontainer and the layers left behind named in its manifest). Every attempt lands in the manifest *and* in the tool result, because a\nrepaired output must never look like one that was right the first time. Failures that need\njudgement are never \"fixed\": an empty result, or geometries eroded away by a wrong\ndistance, come back as warnings with hints for the agent to act on.\n\n**And a manifest can say which configuration produced the numbers** — `environment`,\nsection 3.8 of the manifest specification, empty when there is nothing to say. What made it\nconcrete: a GeoTIFF and the `.aux.xml` file beside it can declare *different*\ngeoreferencing, and GDAL prefers the sidecar by documented design, because that is how\nsomebody overrides georeferencing they know to be wrong. Both readings are the library\nbehaving exactly as written, and on one fixture the same file gives an area four times\nlarger and an origin a hundred kilometres away. There is nothing upstream to fix and\neverything to state, so `describe_dataset` reports both sources when a raster has two, and\n**twelve operations that read a raster's grid directly refuse instead** — zonal statistics,\nresampling, clipping, reclassification, band maths, reprojection, band extraction, band\nstatistics, locating an extreme cell, sampling at points, the elevation profile and the line\nof sight — naming both readings and saying how to choose.\nDescribing is different from computing: a file with two georeferencings is a thing to be\ntold about, not a coin to flip. This is the multi-layer refusal\n([#29](https://github.com/mapsmith-ai/MapSmith/issues/29)) on a second axis — the format's\ndefault answering a question the caller never asked.\n\nThe terrain and sampling operations do not refuse yet, and saying \"any operation that\ncomputes\" would be the promise-with-no-caller this release already found once: the terrain\nengine catches the same file by a different route, because it compares its own reading of\nthe grid against GDAL's and stops when they differ, but its message names neither the\nsidecar nor the way out. Sampling a raster at points does not catch it at all. Extending the\nrefusal to the remaining raster operations is on the roadmap below.\n\n## See results inside the chat\n\n![MapSmith's interactive map panel rendered inside Claude Desktop: OSM basemap, buffer and zone layers, and per-layer provenance cards with verification status](docs/images/map-panel.png)\n\n`preview_map` renders your layers on an interactive map panel *inside* the chat — pan,\nzoom, toggle layers, and read each layer's provenance card (operation, engine, and one of\nthree honest states: `verified ✓`, `verification failed`, or `not verifiable` when no\ncritical check ran) right next to the geometry it explains. Field-tested on Claude\nDesktop; it renders in any client that implements the official\n[MCP Apps](https://modelcontextprotocol.io/extensions/apps/overview) extension, and on\nclients without it the same call returns the preview as structured data.\n\nThe panel is self-contained — no CDN, no bundled libraries, no telemetry — with one\noutbound request named here rather than buried: the OpenStreetMap background tiles, which\nreveal the map view you are looking at (never your data) and which the panel drops to a\nplain backdrop when the host blocks them. The preview is deliberately lossy (simplified\ngeometry, capped feature counts): the dataset of record stays on disk with its manifest.\n\n## Plans: reject wrong analyses before they run\n\nIn [GISAgentBench](https://arxiv.org/abs/2608.01645) — 349 practitioner-sourced tasks over\n128 GIS APIs — the best frontier agent completes 32.7% of tasks under strict scoring, and\nplanning defects dominate the failures: missing operations in 28.3% of failed runs and\nwrong operation order in 18.4% (multi-label, so up to ~47% involve a planning mistake),\nagainst 7.8% for parameter errors. MapSmith attacks this where it is cheapest: the agent\nsubmits a **typed plan**, and static validation rejects unknown operations (with\nsuggestions), missing arguments, forward references, absent input files and CRS-unsuitable\nsteps **before anything executes** — with machine-actionable error codes the agent can\nrepair.\n\n```json\n{\n  \"goal\": \"buildings within 300 m of rivers\",\n  \"steps\": [\n    {\"id\": \"buf\", \"operation\": \"buffer_layer\",\n     \"arguments\": {\"input_path\": \"rivers.gpkg\", \"distance_meters\": 300,\n                   \"output_path\": \"rivers_300m.parquet\"}},\n    {\"id\": \"cut\", \"operation\": \"clip_layer\",\n     \"arguments\": {\"input_path\": \"buildings.parquet\", \"mask_path\": \"$buf\",\n                   \"output_path\": \"at_risk.parquet\"}}\n  ]\n}\n```\n\n`\"$buf\"` consumes the output of step `buf`; references may only point backwards, so plans\nare acyclic by construction. `validate_plan` also simulates the CRS of every intermediate\ndataset from the real input files. `execute_plan` then runs the chain with per-step\nprovenance plus a plan-level manifest (`<output>.plan.json`) fingerprinting the exact plan\nthat produced the result.\n\n## Confinement\n\nUNC hosts and NTFS alternate data streams are refused in every path *argument* of every\ntool call, before anything touches the filesystem (on Windows even an existence check on a\nUNC path talks to an attacker-chosen host). Remote and virtual forms — GDAL `/vsi*`,\n`https://` COGs — are refused by default since 0.2.2 and need `MAPSMITH_ALLOW_REMOTE=1`;\na workspace refuses them whatever that setting says (details below). Validated plans are\nstricter by design and reject every non-local form, opt-in or not.\n\nSet `MAPSMITH_WORKSPACE=/data` to confine the server to one directory:\n\n- every path argument of every tool must resolve inside the workspace (checked at the MCP\n  boundary, and again by plan validation with stable error codes);\n- the `run_sql` DuckDB connection is sandboxed in the engine itself, because SQL text is\n  out of reach of a textual path check: filesystem whitelisted to the workspace\n  (`allowed_directories` + external access off, which also covers GDAL-backed `ST_Read`),\n  memory and temp disk capped (`MAPSMITH_DUCKDB_MEMORY`, default 4GB;\n  `MAPSMITH_DUCKDB_TEMP_LIMIT`, default 8GB), configuration locked. SQL can name any path it\n  likes; the engine refuses to open it.\n\n`MAPSMITH_DISCOVERY_LOG` is the one path MapSmith writes to that no tool argument names,\nso it goes through the same check: outside the workspace it is refused, and the refusal\ndisables the log and says so on stderr rather than failing the search that triggered it.\n\nWithout a workspace, *file* access is deliberately unconfined — fine for a local stdio\nserver on your own files — and plan validation flags `run_sql` steps with a\n`SQL_NOT_SANDBOXED` warning. Code execution is closed **in both modes**, and since 0.4.0 the\nlayer that closes it is the right one: `INSTALL` and `LOAD` in a statement are refused\noutright, because an `INSTALL` is an HTTPS fetch of a native binary run in this process on\nSQL a model wrote. Until 0.4.0 only the *implicit* forms were off, and an audit installed\nDuckDB's `aws` extension and read this machine's real cloud credentials back through a tool\nresult — the whole story is in [SECURITY.md](SECURITY.md), including why none of the four\nexisting layers saw it. Extensions already loaded keep working, `spatial` included; to\nacquire others, name them where the agent cannot reach:\n**`MAPSMITH_ALLOW_EXTENSIONS=postgres,azure`** in the environment of the process that starts\nthe server. Behind that, community extensions stay off (`shellfs` turns a filename into a\nshell command), unsigned extensions are refused, DuckDB's HTTP and S3 filesystems are\ndisabled, and the configuration is locked.\n\n**The network is closed too, unless you open it.** Remote and virtual forms — GDAL\n`/vsi*`, `https://` COGs — are refused by default in path arguments *and* inside `run_sql`\ntext, because the path is written by the model rather than by you: a third-party dataset\ncarrying \"the updated layer lives at `https://evil.tld/x.gpkg`\" was otherwise enough to have\nGDAL parse attacker-chosen bytes in-process. Set **`MAPSMITH_ALLOW_REMOTE=1`** to allow them\n— cloud-native data is a real use case and the capability is gated, not removed. A workspace\nrefuses them regardless, since containment and \"fetch whatever URL the model names\" cannot\nboth be true. The test suite asserts every branch by counting requests at a loopback server\n(`tests/test_duckdb_sandbox.py`). The full threat model —\nand what is explicitly *not* covered — is in [SECURITY.md](SECURITY.md).\n\nFine print, because it changes how you deploy this: the path jail assumes a single trusted\nwriter of the workspace filesystem (paths are resolved at check time, so a symlink swap by\nanother local process is out of scope); the DuckDB spatial extension is fetched once per\nenvironment by MapSmith itself, through the Python API rather than by any statement, so on\nair-gapped machines pre-install it (`python -c \"import duckdb;\nduckdb.connect().install_extension('spatial')\"`) before locking the network down; and the\nHTTP transport has no authentication in this release, so keep it on loopback or a trusted\nnetwork. For real isolation, run the container and mount only the data you want it to see.\n\n## The dashboard\n\nEverything this project knows about itself is computed somewhere and most of it is printed\nonce and lost, which is how a number ages into a claim. `benchmarks/dashboard.py` gathers it\ninto one self-contained HTML file — no CDN, no fonts, no analytics, works with the network\noff:\n\n```bash\npython benchmarks/dashboard.py --log /data/discovery.jsonl --argleton ../argleton\n```\n\n* **Operations** — every entry, and whether a caller actually reaches it. Asked twice: with\n  words alone, and with the facets a caller knows. Three outcomes are kept apart — a rank, an\n  answer that did not contain the entry, and a search that *declined* because the two rankers\n  shared nothing. Collapsing the third into the second is the first thing this page got wrong\n  about itself, and it drew ten working operations as broken.\n* **Search quality** — the facet ablation for both rankers, and the degradation curve as the\n  catalog grows, which is the measurement the embedding engine became a dependency for.\n* **Traps** — [Argleton](https://argleton.org)'s families and what each engine does with them,\n  MapSmith included and not flattered. With `--argleton <path>` it reads a checkout and shows\n  the per-family detail; without one it falls back to the vendored citation and says so.\n* **Answer the open questions** — the requests where the ",
  "bytes": 60000,
  "sha": "23271b3b2b19db8a883cc195caa76b111817ecac49833b8ed7180a97385865c2",
  "repo_slug": "mapsmith-ai/mapsmith",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/mcp_io_github_mapsmith_ai_mapsmith_3ac2d00c/readme"
}