{
  "markdown": "# Self-hosted, license-free SIEM\n\nA NIST 800-53-oriented SIEM built entirely from free/open components:\n**ClickHouse** for storage and SQL, **Vector** for all collection, **Grafana\nOSS** for dashboards and alerting, **HyperDX** for log search, **Keycloak**\nfor SSO with mandatory TOTP MFA. No ingest caps, no license keys, no\nphone-home, no per-seat pricing.\n\nThe project exists because commercial/openish alternatives gate the\ncompliance-critical features (SSO, RBAC, audit trails, >50 GB/day) behind paid\ntiers. Here, the audit evidence *is* the product: every design choice traces\nto an AU-family control in [docs/control-mapping.md](docs/control-mapping.md).\n\n## Dashboard\n\n![800-53 Logging Evidence — Weekly ISSO Review](docs/images/nist-dashboard.png)\n\nThe weekly ISSO review dashboard (shown populated with synthetic demo data):\nat-a-glance control tripwires across the top — audit-log tampering (AU-9), root\nusage (AC-6), failed logons (AC-7), high-severity GuardDuty (SI-4) — then\nWindows, Linux, and AWS sections covering every AU-2-committed event family,\neach panel tagged with its control ID. Querying: [docs/query-guide.md](docs/query-guide.md).\n\n## Architecture\n\n    Windows hosts ── vector agent (Security/System/PowerShell) ──┐\n    Linux hosts ──── vector agent (journald + auditd) ───────────┤ :6000 (native)\n    K8s clusters ─── vector DaemonSet ───────────────────────────┤ :8088 (HEC)\n                                                                 v\n                                             [ vector-hosts aggregator ]\n    AWS CloudTrail/GuardDuty/VPCFlow/S3 ── S3 -> SQS ── [ vector (aws) ]\n                                                                 |\n                                     normalize (VRL) -> INSERT as svc_vector\n                                                                 v\n                              [ ClickHouse ]  siem.* tables + audit.query_archive\n                                   ^     ^\n                     SQL, dashboards,    | search/investigate (svc_hyperdx)\n                     alerts (svc_grafana_*)                    |\n                              [ Grafana ]              [ HyperDX ]\n                                   ^                        ^\n                                   |                 [ oauth2-proxy ]\n                                   +── OIDC ──[ Keycloak + TOTP MFA ]── OIDC ──+\n\nEvery human enters through Keycloak. Every query any UI runs is captured in\n`audit.query_archive` (AU-9). Ingest happens only through the write-only\n`svc_vector` account; analysts can never write, auditors can also read the\nanalyst-activity trail.\n\n## Components\n\n| Service | Image (pinned) | License | Role |\n|---|---|---|---|\n| clickhouse | clickhouse/clickhouse-server:24.8 | Apache-2.0 | storage, SQL, RBAC, audit trail |\n| keycloak (+postgres 16) | quay.io/keycloak/keycloak:26.0 | Apache-2.0 | SSO, mandatory TOTP MFA |\n| grafana | grafana/grafana-oss:11.4.0 | AGPL-3.0 | dashboards, AU-5 alerting |\n| hyperdx (+mongo 7.0) | docker.hyperdx.io/hyperdx/hyperdx:2.19.0 | MIT | log search / investigations UI |\n| hyperdx-auth | quay.io/oauth2-proxy/oauth2-proxy:v7.15.3 | Apache-2.0 | Keycloak SSO gate for HyperDX |\n| vector-hosts | timberio/vector:0.57.0-debian | MPL-2.0 | host/K8s ingestion (always on) |\n| vector | timberio/vector:0.57.0-debian | MPL-2.0 | AWS ingestion (profile `aws`) |\n| k3s (demo) | rancher/k3s:v1.35.6-k3s1 | Apache-2.0 | local test cluster (profile `k3s`) |\n\nLicense note: MongoDB (HyperDX app-state only — no audit data) is SSPL: free\nto self-host, no caps or keys, not OSI-approved; accepted and documented in\ncontrol-mapping.\n\n## Repository layout\n\n    docker-compose.yml           the whole platform\n    bootstrap.sh                 one-command fresh install\n    .env.example                 every secret/setting, annotated\n    clickhouse/\n      ddl/                       schemas, audit trail, RBAC (auto-applied on first start)\n      config.d/ users.d/         listen config, query_log, default-user lockdown\n      initdb/99-init.sh          applies DDL + creates service accounts\n    vector/\n      vector.yaml                AWS pipelines (CloudTrail/GuardDuty/VPCFlow/S3)\n      hosts.yaml                 host+K8s aggregator (HEC :8088, native :6000)\n      agent-linux.yaml           drop-in config for Linux hosts\n      agent-windows.yaml         drop-in config for Windows hosts\n      tests.yaml tests-hosts.yaml  VRL unit tests\n    k8s/\n      vector-shipper.yaml        DaemonSet manifest for any cluster\n      k3s-demo-up.sh             one-command local demo cluster\n    scripts/\n      install-windows-agent.ps1  elevated Windows agent installer\n      fix-windows-agent-addr.ps1 WSL-lab address fix + boot task\n    grafana/provisioning/        datasources, dashboards, AU-5 alert rules\n    keycloak/realm-export/       \"siem\" realm: MFA, roles, OIDC clients\n    docs/                        OKF v0.1 knowledge bundle (runbooks, policies, catalog)\n    scripts/okf-validate.py      OKF conformance checker for docs/\n\n## UIs\n\n| URL | What | Auth |\n|---|---|---|\n| http://localhost:3000 | Grafana — dashboards, SQL (Explore), alerts | Keycloak SSO + TOTP |\n| http://localhost:8081 | HyperDX — log search, investigations | Keycloak SSO first (oauth2-proxy), then HyperDX local account |\n| http://keycloak:8080 | Keycloak admin console | kcadmin (see .env) |\n\nBrowser prerequisite: `127.0.0.1 keycloak` in the hosts file so the browser\nand containers agree on the Keycloak hostname (admin PowerShell:\n`Add-Content $env:SystemRoot\\System32\\drivers\\etc\\hosts \"127.0.0.1 keycloak\"`).\n\nDashboards (Grafana -> SIEM folder):\n- **800-53 Logging Evidence — Weekly ISSO Review**: the weekly audit pass —\n  at-a-glance posture stats (tampering, root usage, failed auth, GuardDuty),\n  then Windows / Linux / AWS sections with every AU-2-committed event family,\n  control IDs in each panel title.\n- **AWS Security Overview**: console logins, AccessDenied trend, root\n  activity, IAM writes.\n- **Pipeline Health (AU-5)**: per-source ingest lag + rate.\n\nNew to querying? Start with [docs/query-guide.md](docs/query-guide.md).\n\n## Fresh install\n\n1. Prereqs: Docker Engine + compose v2, bash, openssl. (This lab runs Docker\n   CE inside WSL2 Ubuntu — no Docker Desktop.)\n2. `./bootstrap.sh you@example.com` — generates `.env` (all secrets), starts\n   the stack, rotates the Grafana + HyperDX OIDC client secrets, creates your\n   admin user (temp password printed once), verifies RBAC.\n3. Add the hosts entry (above), open http://localhost:3000, log in, set a new\n   password, enroll TOTP.\n4. Visit http://localhost:8081, pass Keycloak, register the HyperDX local\n   account — its ClickHouse connection + sources auto-provision at that moment.\n5. Onboard data sources (next section).\n\n`bootstrap.sh` refuses to overwrite an existing `.env`. Fully wipe with\n`docker compose down -v` (destroys data).\n\n## Data source onboarding\n\n| Source | Status here | Runbook |\n|---|---|---|\n| Linux hosts (journald+auditd) | LIVE (WSL host) | [docs/host-ingestion.md](docs/host-ingestion.md) |\n| Kubernetes (any cluster, HEC) | LIVE (local k3s demo) | same |\n| Windows (Security/System/PowerShell) | LIVE (this machine, idle-freeze pilot) | same |\n| AWS CloudTrail/GuardDuty/VPCFlow/S3 | staged — needs account wiring | [docs/aws-ingestion.md](docs/aws-ingestion.md) |\n\nAWS go-live: follow the runbook (S3 -> SQS -> least-privilege IAM), fill the\nPhase 2 block in `.env`, uncomment `COMPOSE_PROFILES=aws`, `docker compose up\n-d`, then un-pause the four AWS alert rules (Alerting -> AU-5 pipeline health).\n\n## Security model\n\n- **Authentication**: everything human-facing sits behind Keycloak (realm\n  `siem`) with TOTP enrollment forced on first login. Grafana local login is\n  disabled; HyperDX (no native SSO in OSS) is unreachable except through\n  oauth2-proxy, which requires a `siem_*` realm role.\n- **Authorization** (ClickHouse enforces, not the UIs):\n\n  | Account | Can | Cannot |\n  |---|---|---|\n  | svc_vector | INSERT siem.* | read anything |\n  | svc_grafana_analyst / svc_hyperdx | SELECT siem.* | write; read audit.* |\n  | svc_grafana_auditor | SELECT siem.* + audit.* | write |\n  | siem_admin (bootstrap) | everything | — |\n\n  Realm roles map to Grafana roles (siem_admin→Admin, siem_analyst→Editor,\n  siem_auditor→Viewer). Readers run under a settings profile: SELECT-only,\n  8 GB / 120 s / 20B-rows per query.\n- **Audit trail (AU-9)**: an incremental MV copies every finished query from\n  system.query_log into append-only `audit.query_archive` (2-year TTL) —\n  including every query Grafana and HyperDX run on anyone's behalf.\n- **Network**: ClickHouse/native+HTTP bound to localhost on the host; the\n  in-container `default` user is loopback-confined; ingest listeners (:6000,\n  :8088) are token/marker-validated at the aggregator and write-only at the DB.\n- **Secrets**: all in `.env` (git-ignored, chmod 600), generated by bootstrap;\n  Keycloak client secrets rotated out of the realm-export placeholders.\n\n## Compliance surface\n\nThe `docs/` directory is an [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md)\n(OKF v0.1) knowledge bundle — every doc carries YAML frontmatter (`type`,\n`title`, `description`, `tags`), `docs/index.md` lists the bundle, and\n`docs/log.md` tracks changes. Conformance is enforced by\n`python3 scripts/okf-validate.py docs` (start at [docs/index.md](docs/index.md)).\n\n- [docs/control-mapping.md](docs/control-mapping.md) — control -> artifact map (auditor-facing)\n- [docs/event-catalog.md](docs/event-catalog.md) — the AU-2 commitment; VRL implements exactly this\n- [docs/retention-policy.md](docs/retention-policy.md) — AU-11 numbers (DRAFT: total-retention needs confirmation)\n- [docs/query-guide.md](docs/query-guide.md) — analyst quick-start for HyperDX + Grafana\n- AU-5: provisioned alert rules fire on source silence (per-table thresholds),\n  and on ClickHouse being unreachable (NoData/Error -> alerting). Notifications\n  route to the `siem-oncall` contact point — set a real address + SMTP.\n\n## Operations\n\n- **Start/stop**: `docker compose up -d` / `docker compose down` (add\n  `--profile aws --profile k3s` to include optional services). On this lab\n  machine a SYSTEM boot task starts WSL -> docker -> stack automatically.\n- **Weekly ISSO pass**: open the 800-53 dashboard, review each section (red\n  stats first), check Alerting for anything firing, spot-check\n  `audit.query_archive` via the audit-trail datasource.\n- **VRL changes**: edit vector/*.yaml, run the unit tests, restart the service:\n\n      docker run --rm -v \"$PWD/vector:/cfg:ro\" -e AWS_REGION=x \\\n        -e SQS_URL_CLOUDTRAIL=x -e SQS_URL_GUARDDUTY=x -e SQS_URL_VPCFLOW=x \\\n        -e SQS_URL_S3ACCESS=x -e CH_VECTOR_PASSWORD=x \\\n        timberio/vector:0.57.0-debian test /cfg/vector.yaml /cfg/tests.yaml\n      docker run --rm -v \"$PWD/vector:/cfg:ro\" -e SPLUNK_HEC_TOKEN=x \\\n        -e CH_VECTOR_PASSWORD=x \\\n        timberio/vector:0.57.0-debian test /cfg/hosts.yaml /cfg/tests-hosts.yaml\n\n- **Break-glass** (Keycloak down): set `GF_AUTH_DISABLE_LOGIN_FORM=false` on\n  the grafana service, `docker compose up -d grafana`, log in with\n  GRAFANA_ADMIN_USER from `.env`. Revert immediately; the event is in\n  Grafana's logs and the audit trail.\n- **Version bumps**: images are pinned; bump deliberately, one at a time, and\n  re-run the vector unit tests + a bootstrap on a scratch host before fleet\n  changes.\n- **Windows idle-freeze pilot** (vector#25194): the AU-5 Windows rule is the\n  tripwire. If it fires while the host is active and the service is Running:\n  `Restart-Service vector` (agent disk buffers prevent loss).\n\n## Field notes / troubleshooting\n\nHard-won lessons encoded in this repo — check here before debugging:\n\n- **ClickHouse in docker listens on loopback only** by default; other\n  containers can't reach it. `config.d/05-listen.xml` sets 0.0.0.0 (`::`\n  crashes on IPv6-less compose networks, exit 210).\n- **Never mount users.d read-only over the whole directory** — the image\n  entrypoint must write default-user.xml there (crash loop otherwise). Mount\n  individual files.\n- **Vector >= 0.57 does not interpolate `${VAR}` in configs by default.**\n  Set `VECTOR_DANGEROUSLY_ALLOW_ENV_VAR_INTERPOLATION=true` (compose sets it;\n  agents need it too). Interpolation is pre-parse text substitution, so a\n  dollar-brace reference **even inside a comment** aborts config load.\n- **Vector's clickhouse sink healthcheck probes without auth** and 403s\n  against the locked-down default user — sink healthchecks are disabled in\n  our configs; inserts are authenticated and unaffected.\n- **Windows services can't use the WSL2 localhost relay** (interactive\n  sessions only). Agents on the docker host must target the WSL NAT IP —\n  `scripts/fix-windows-agent-addr.ps1 -Register` keeps it correct per boot.\n- **PowerShell scripts must stay pure ASCII**: PowerShell 5.1 reads BOM-less\n  UTF-8 as ANSI; an em-dash becomes a smart quote and kills parsing.\n- **Pods inside in-docker k3s can't resolve docker DNS names** (k3s swaps\n  loopback resolvers for a public one) — `k8s/k3s-demo-up.sh` patches the\n  shipper endpoint to the aggregator IP.\n- **HyperDX DEFAULT_CONNECTIONS/DEFAULT_SOURCES seed only when the first user\n  registers**, and malformed JSON is skipped silently.\n- **ClickHouse 24.8 rejects `REFRESH ... APPEND` MVs** (newer + experimental);\n  the audit trail uses a standard incremental MV instead. `system.query_log`\n  doesn't exist until first flush — DDL runs `SYSTEM FLUSH LOGS` first.\n- WSL idle-shutdown stops the stack between sessions; the boot task (or any\n  open WSL shell) keeps it alive.\n\n## Roadmap\n\n- **Phase 5 (remaining)**: S3 tiering for ClickHouse (config staged at\n  `config.d/20-storage-s3.xml.disabled`), Object Lock raw archive (Parquet via\n  aws_s3 sink), query_archive export. Retention numbers pending confirmation\n  in retention-policy.md. AU-5 alerting: DONE.\n- **Phase 6**: detection SQL + alert rules per control family (`detections/`).\n- **Phase 7**: operational cadence — review checklists, evidence exports,\n  annual catalog review.\n",
  "bytes": 14148,
  "sha": "85888d76ea111ab0e2a7ff131fed8ed7cf0281e1b381164788a86a3ae1a341b8",
  "repo_slug": "jake-kelley/ironlog",
  "fonte": "repo",
  "truncated": false,
  "api": "https://agentalog.com/api/listings/okf_jake_kelley_ironlog_docs_index_md_89c3f28d/readme"
}