{
  "markdown": "\n<img\n  src=\"logo.png\"\n  alt=\"ECAZ logo\"\n  width=\"100%\"\n  title=\"Rinse the roots clean, then scrape away the outer bark. Slice thinly and dry until brittle. Grind into a coarse powder with a pinch of mineral salt. Steep in hot water until the liquid turns deep crimson. Strain, cool, and store in a dark glass vial.\"\n/>\n\n[![Discord](https://img.shields.io/badge/Discord-Join%20us-5865F2?logo=discord&logoColor=white)](https://discord.gg/6qsdhSPE)\n\nEcaz is a rust based PostgreSQL extension for performant,\nhighly scalable vector storage and retrieval. It supports a broad range\nof quantization and index options rather than a single fixed architecture.\n\n#### Column Types\n\n- `ecvector(dim)` — canonical vector row type\n- `tqvector` — TurboQuant quantized vector storage\n\n#### Quantization Types\n\n- `turboquant` — [TurboQuant](https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/): training-free (data-oblivious) quantization that randomly rotates each vector and scalar-quantizes its coordinates, reaching near-optimal distortion at extreme compression with no learned codebook.\n- `pq_fastscan` — [Product Quantization](https://ieeexplore.ieee.org/document/5432202) in the SIMD FastScan layout: the vector is split into sub-blocks, each mapped to the nearest entry of a small learned codebook; 4-bit codes are scored through in-register lookup tables, with a colder full-precision rerank payload.\n- `rabitq` — [RaBitQ](https://arxiv.org/abs/2405.12497): quantization with a theoretical error bound — a random rotation collapses each dimension toward a sign bit, plus a few per-vector correction scalars that keep the distance estimate unbiased; supports 1–8 bit codes.\n\n#### Index Families\n\n- `ec_hnsw` — HNSW graph index (general-purpose default)\n- `ec_ivf` — IVF posting-list index\n- `ec_diskann` — DiskANN/Vamana-style graph index\n- `ec_spire` — [SPIRE](https://arxiv.org/html/2512.17264v1) partitioned local/distributed IVF-family index\n\n## Quick Start\n\n```bash\ncargo install cargo-pgrx@0.17\ncargo pgrx init --pg18 download\ncargo pgrx run --release pg18\n```\n\n`cargo pgrx run` builds the extension, installs it into the managed PG18\ncluster, starts PostgreSQL if needed, and opens `psql`.\n\n```sql\nCREATE EXTENSION ecaz;\n\nCREATE TABLE memories (\n    id bigint generated always as identity primary key,\n    embedding ecvector(4)\n);\n\n-- Encode and store a canonical vector\n--   args: float4[] input, codebook_bits (4), rng_seed (42)\nINSERT INTO memories (embedding)\nVALUES (encode_to_ecvector(ARRAY[1.0, 2.0, 3.0, 4.0]::float4[], 4, 42));\n\n-- Create HNSW index over the canonical row type\nCREATE INDEX ON memories\nUSING ec_hnsw (embedding ecvector_ip_ops)\nWITH (m = 8, ef_construction = 64);\n\n-- Query nearest neighbors\nSELECT id FROM memories\nORDER BY embedding <#> ARRAY[1.0, 2.0, 3.0, 4.0]::float4[]\nLIMIT 10;\n```\n\nSee [Build From Source](docs/build-from-source.md) for the full repeatable\nsetup path, including native prerequisites, existing-PostgreSQL installs,\noperator CLI setup, and validation commands.\n\n## Compatibility\n\n| Area       | Status                                                |\n|------------|-------------------------------------------------------|\n| PostgreSQL | PG18 primary target; PG17 compatibility target        |\n| pgrx       | `cargo-pgrx` 0.17                                     |\n| Rust       | Stable toolchain                                      |\n| Production | SIMD optimization for Graviton 4 (ARM NEON) & AWS Intel x86_64 (AVX2) |\n| Development | SIMD optimization for linux/x86_64 (AVX2) & Apple Silicon (ARM NEON) |\n\n\n## Build From Source\n\nEcaz targets PG18 by default, with PG17 kept as a compatibility build. A\ncomplete source setup has five parts:\n\n1. Install Rust stable, native build tools, and PostgreSQL build dependencies.\n2. Install the matching pgrx toolchain: `cargo install cargo-pgrx@0.17`.\n3. Initialize pgrx for PG18: `cargo pgrx init --pg18 download`.\n4. Build and install into a pgrx-managed PG18: `cargo pgrx run --release pg18`.\n5. Install the operator CLI for repeatable local SQL, corpus, and benchmark\n   commands: `cargo install --path crates/ecaz-cli`.\n\nFor an already-installed PostgreSQL server, install with an explicit\n`pg_config` instead:\n\n```bash\ncargo pgrx install --sudo --release --pg-config /path/to/pg_config\n```\n\nThe detailed guide is [docs/build-from-source.md](docs/build-from-source.md).\n\n## Performance\n\nResults below come from the standard ecaz sweep — all four index families ×\nquantizations × 10K/50K/100K/1M — run on the AWS production lanes (Graviton 4\nwith ARM NEON, and Intel Sapphire Rapids with AVX2) over 1536-dimensional DBpedia\nOpenAI embeddings. These are engineering measurements, not product claims. See\n[Benchmarks](docs/benchmarks.md), [Benchmark Index](docs/benchmark-index.md), and\n[Benchmark Reporting Standard](docs/benchmark-reporting-standard.md) for the full\nmatrix, source packets, and reporting rules.\n\n### Compression And Storage Format\n\nEncoded payload size per vector, for 1536-dimensional vectors. Lower bit widths\ntrade recall for size; pick the format and bit width that fits your recall and\nstorage budget. (These are the quantized code bytes; the on-disk index adds\nposting-list/graph structure — see the index sizes in the snapshot below.)\n\n| Representation | Bytes per vector | Relative size |\n| --- | ---: | ---: |\n| Raw fp32 | 6,144 B | 1.00x |\n| PQ-FastScan g8 (search code) | 96 B | 64.0x smaller |\n| RaBitQ 1-bit | 204 B | 30.1x smaller |\n| RaBitQ 2-bit | 396 B | 15.5x smaller |\n| TurboQuant 2-bit | 399 B | 15.4x smaller |\n| RaBitQ 4-bit | 780 B | 7.88x smaller |\n| TurboQuant 4-bit | 783 B | 7.85x smaller |\n| RaBitQ 8-bit | 1,548 B | 3.97x smaller |\n| TurboQuant 8-bit | 1,551 B | 3.96x smaller |\n\nA raw fp32 vector barely fits one tuple per 8 KB page; the quantized codes pack\nmany vectors per page, which is what makes large-corpus scans fast.\n\n\n### Index Family Snapshot\n\nAll four index families at the **1M production scale on Graviton 4**\n(`m8g.2xlarge`, AWS arm64), each at its strongest production quantization and a\nrepresentative high-recall operating point. Same corpus (990K real DBpedia\nOpenAI embeddings, 1536-dim), same platform, same `k=10` — so the families are\ndirectly comparable.\n\n| Access method | Quant | Recall@10 | p50 latency | Index size |\n| --- | --- | ---: | ---: | ---: |\n| `ec_hnsw` | turboquant | 0.930 | 13.7 ms | 1.3 GiB |\n| `ec_ivf` | rabitq (1-bit) | 0.980 | 56.8 ms | 290 MiB |\n| `ec_diskann` | rabitq | 0.981 | 5.0 ms | 407 MiB |\n| `ec_spire` ⁽¹⁾ | rabitq | 0.986 | 137 ms | 779 MiB |\nOperating points: DiskANN `list_size=64..128`, IVF/SPIRE `nprobe=16..64`, HNSW\n`ef_search=80..160`. Each family also has a faster lower-recall point on the same\nindex — e.g. DiskANN 0.947 @ 3.6 ms, IVF 0.926 @ 16.1 ms.\n\nHow to read it:\n\n- **`ec_diskann`** — best all-round at scale: the most recall per millisecond and\n  a compact index. Requires unit-normalized source vectors.\n- **`ec_ivf`** — strong recall with the smallest index (RaBitQ 1-bit is the sweet\n  spot); the posting-list model lets you trade recall against latency via `nprobe`.\n- **`ec_hnsw`** — general-purpose graph default with competitive latency; recall\n  tops out lower than the other families at 1M in this sweep.\n- **`ec_spire`** ⁽¹⁾ — a partitioned, **distributed / scale-out** index. The row\n  above is its single-node point; SPIRE's real value is multi-node (below). It\n  trades single-node latency for partitioning, and its latency is still being\n  optimized.\n\nFor context, at 1M `ec_ivf` (RaBitQ 1-bit) serves 0.980 recall at 56.8 ms p50 —\nfaster than the tuned vchord RaBitQ comparator (~90 ms p50, which reaches ~1.0\nrecall) — and every ecaz family is far ahead of the untuned pgvector /\npgvectorscale baselines.\n\nSource: `reviews/task-105/006-full-scale-matrix/` (Task 105 full-scale matrix,\n`main=1345ca603`; G4 + Intel × 10K/50K/100K/1M × all AM/quant).\n\n#### ⁽¹⁾ SPIRE distributed (multi-node)\n\nSPIRE partitions a corpus across nodes and serves queries by fanning out to\nremote leaves. A real **3-node** deployment (1 coordinator + 2 remotes), with the\n1M corpus sharded across the remotes (~505K + ~485K rows) and genuine remote-heap\nreads, at `nprobe=64`:\n\n| Topology | Quant | Recall@10 | p50 | p95 |\n| --- | --- | ---: | ---: | ---: |\n| 3-node distributed | rabitq | 0.951 | 117 ms | 135 ms |\n| 3-node distributed | turboquant | 0.949 | 140 ms | 164 ms |\n\nDistributing across 3 nodes is roughly **5x faster at 1M than the same index on a\nsingle node** (121 ms vs 620 ms at matched `nprobe=32`) — SPIRE trades latency for\nscale-out partitioning. It is currently a **research / scale-out surface**: not yet\non the single-node DiskANN/IVF latency frontier, with latency optimization\nongoing. Source: `reviews/task-107/` (`005-product-decision/`,\n`004-distributed-completion/`).\n\n## Choosing An Index\n\nEach index family implements a different search algorithm. Quantization\n(`storage_format`) is a separate concern — it controls how vectors are\ncompressed inside the index and is independent of the index family. See\n[Usage Guide](docs/usage.md) for full SQL examples,\n[Benchmarks](docs/benchmarks.md) for selected results, and\n[Benchmark Reporting Standard](docs/benchmark-reporting-standard.md) for the\nfields required in new AM, quantizer, storage-format, and option-set\ncomparisons.\n\n| Access method | Best fit | Storage formats | Notes |\n| --- | --- | --- | -- |\n| `ec_hnsw` | General-purpose ANN graph search | `turboquant`, `pq_fastscan`, `rabitq` |  |\n| `ec_ivf` | Posting-list experiments and high-ingest tradeoffs | `turboquant`, `pq_fastscan`, `rabitq` | |\n| `ec_diskann` | Compact graph indexes, strong recall-per-ms at scale | `pq_fastscan`, `rabitq`, `turboquant` | Requires unit-normalized source vectors |\n| `ec_spire` | Partitioned local and distributed search | `turboquant`, `rabitq` |  |\n\n\n## Development\n\nPG18 is the primary target; PG17 is kept as a compatibility build.\n\n- [Rust](https://rustup.rs/) stable\n- [cargo-pgrx](https://github.com/pgcentralfoundation/pgrx) `0.17`\n- Native PostgreSQL build dependencies, or PostgreSQL 18 development headers if\n  using an existing server\n\nThe standard local loop targets PG18:\n\n```bash\ncargo pgrx init --pg18 download\nmake fmt\nmake lint\nmake test\nmake pg-test\n```\n\nPG17 compatibility coverage is optional — run it only when touching\nPG17-specific behavior:\n\n```bash\nmake lint-pg17\nmake pg-test-pg17\n```\n\n## Documentation\n\n| Document | Description |\n| --- | --- |\n| [Getting Started](docs/getting-started.md) | Prerequisites, installation, first query |\n| [Build From Source](docs/build-from-source.md) | Full repeatable local build and setup path |\n| [Usage Guide](docs/usage.md) | Encoding parameters, index tuning, query patterns |\n| [Benchmarks](docs/benchmarks.md) | Measured performance results and methodology |\n| [Benchmark Index](docs/benchmark-index.md) | Packet directory for benchmark lanes and source artifacts |\n| [Benchmark Reporting Standard](docs/benchmark-reporting-standard.md) | Required fields for AM, quantizer, storage-format, and option-set comparisons |\n| [Rust Safety And Quality](docs/hardening.md) | Hardening lanes for linting, unsafe audit, Miri, fuzzing, model checking, sanitizers, and supply-chain checks |\n| [Operator CLI](crates/ecaz-cli/README.md) | `ecaz` corpus, benchmark, compare, stress, and dev command surface |\n| [Architecture](docs/architecture.md) | Compression pipeline, index layout, page format |\n| [PG18 Features](docs/pg18.md) | ReadStream, EXPLAIN hooks, AM callbacks |\n| [Contributing](docs/contributing.md) | Makefile targets, CI, testing, fuzzing |\n| [References](docs/references.md) | Papers and libraries |\n\n## Project\n\n| Resource | Description |\n| --- | --- |\n| [Specification](spec/spec.md) | Master requirements specification |\n| [Implementation Plan](plan/plan.md) | Task board, sequencing, status |\n| [ADRs](spec/adr/) | Architecture decision records |\n| [Reviews](review/) | Review packets and feedback ([workflow](AGENTS.md)) |\n\n## License\n\nMIT\n\n## This software was written 100% by AI\n\nEcaz is an Agentic Engineering experiment: an attempt to develop a complex\ndatabase system written solely by AI. A human worked with AI to design the\narchitecture and navigate the many design decisions, but 100% of the code was\nwritten by GPT >=5.4 and Claude Opus >=4.6.\n\n**The ethos is to pursue quality, testing,\n[Rust safety and hardening](docs/hardening.md), and benchmarking rigorously, but\nthe project should not yet be considered production-ready.**\n\nHaving achieved the initial goal of support for well-known index\nfamilies, the project now aims to build proof-of-concept implementations for\nfrontier vector database research.\n",
  "bytes": 12726,
  "sha": "937167189ade89e8c7e0d783daf3e10065f05d8a10326008aa3290aac7f3dc55",
  "repo_slug": "agent-ix/ecaz",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_agent_ix_ecaz_spec_index_md_952eb8ed/readme"
}