{
"markdown": "# ๐ฎ๐ฉ indonesia-civic-stack\n\n\n\n[](https://pypi.org/project/indonesia-civic-stack/)\n[](https://registry.modelcontextprotocol.io/servers/io.github.suryast/indonesia-civic-stack)\n[](https://github.com/suryast/indonesia-civic-stack/actions)\n[](https://pypi.org/project/indonesia-civic-stack/)\n[](LICENSE)\n\nProduction-ready scrapers, normalizers, and API wrappers for Indonesian government data sources.\n\nThe infrastructure layer beneath [halalkah.id](https://halalkah.id), [legalkah.id](https://legalkah.id), and a public good for the Indonesian civic tech and developer community.\n\n---\n\n## Why\n\nIndonesian public data is nominally open but practically inaccessible. Every developer building civic tooling re-solves the same scraping problems independently: BPOM product registrations, BPJPH halal certificates, AHU company records. Scrapers bit-rot within months as portals change. There is no shared, maintained layer.\n\n**This repo is that layer.** One `pip install` to query Indonesian government portals โ no more bespoke scrapers.\n\n### AI-Agent First\n\nThis SDK is designed for both humans and AI agents:\n\n- ๐ค **46 MCP tools** โ plug into Claude, GPT, or any MCP-compatible agent\n- ๐ **[SKILL.md](SKILL.md)** โ AI agent skill discovery (AgentSkills format)\n- ๐งโ๐ป **[AGENTS.md](AGENTS.md)** โ architecture guide for coding agents (Claude Code, Codex, Cursor)\n- ๐ **[CLAUDE.md](CLAUDE.md)** โ Claude Code-specific instructions\n- โ
**Typed responses** โ `CivicStackResponse` envelope, never raw dicts\n- ๐ **Consistent patterns** โ every module follows the same contract\n\n---\n\n## Architecture\n\n```mermaid\ngraph TB\n subgraph \"Your App\"\n A[halalkah.id] \n B[legalkah.id]\n C[Your Project]\n end\n\n subgraph \"civic-stack\"\n SDK[Python SDK]\n MCP[MCP Servers]\n API[REST API]\n \n subgraph \"Shared Layer\"\n SC[shared/schema.py
CivicStackResponse]\n HC[shared/http.py
Rate limiting ยท Retries ยท Proxy]\n end\n\n subgraph \"Phase 1\"\n BPOM[bpom
Food & Drug]\n BPJPH[bpjph
Halal Certs]\n AHU[ahu
Company Registry]\n end\n\n subgraph \"Phase 2\"\n OJK[ojk
Financial Licenses]\n OSS[oss_nib
Business ID]\n LPSE[lpse
Procurement]\n KPU[kpu
Elections]\n end\n\n subgraph \"Phase 3\"\n LHKPN[lhkpn
Wealth Declarations]\n BPS[bps
Statistics]\n BMKG[bmkg
Weather & Disasters]\n SIMBG[simbg
Building Permits]\n end\n end\n\n subgraph \"Government Portals\"\n P1[cekbpom.pom.go.id]\n P2[sertifikasi.halal.go.id]\n P3[ahu.go.id]\n P4[ojk.go.id]\n P5[oss.go.id]\n P6[lpse.*.go.id]\n P7[infopemilu.kpu.go.id]\n P8[elhkpn.kpk.go.id]\n P9[webapi.bps.go.id]\n P10[data.bmkg.go.id]\n P11[simbg.pu.go.id]\n end\n\n A & B & C --> SDK & MCP & API\n SDK & MCP & API --> SC\n SC --> BPOM & BPJPH & AHU & OJK & OSS & LPSE & KPU & LHKPN & BPS & BMKG & SIMBG\n BPOM & BPJPH & AHU & OJK & OSS & LPSE & KPU & LHKPN & BPS & BMKG & SIMBG --> HC\n BPOM --> P1\n BPJPH --> P2\n AHU --> P3\n OJK --> P4\n OSS --> P5\n LPSE --> P6\n KPU --> P7\n LHKPN --> P8\n BPS --> P9\n BMKG --> P10\n SIMBG --> P11\n```\n\n---\n\n## Request Flow\n\n```mermaid\nsequenceDiagram\n participant App as Your App\n participant SDK as Civic SDK\n participant HTTP as shared/http.py\n participant Proxy as Proxy (optional)\n participant Portal as Gov Portal\n\n App->>SDK: search(\"paracetamol\")\n SDK->>HTTP: civic_client(proxy_url)\n Note over HTTP: Auto-reads PROXY_URL
from environment\n alt rewrite mode (CF Worker)\n HTTP->>Proxy: GET ?url=encoded_target\n Proxy->>Portal: Forwarded request\n Portal-->>Proxy: HTML/JSON response\n Proxy-->>HTTP: Response\n else connect mode (SOCKS/HTTP)\n HTTP->>Proxy: CONNECT tunnel\n Proxy->>Portal: Proxied request\n Portal-->>HTTP: Response\n else no proxy\n HTTP->>Portal: Direct request\n Portal-->>HTTP: Response\n end\n HTTP-->>SDK: httpx.Response\n SDK->>SDK: Parse + Normalize\n SDK-->>App: CivicStackResponse\n```\n\n---\n\n## Module Status\n\n| Module | Source | Data | Proxy | Status |\n|--------|--------|------|:-----:|--------|\n| [`bpom`](civic_stack/bpom/) | cekbpom.pom.go.id | Food, drug, cosmetic registrations | ๐ | โ
Active |\n| [`bpjph`](civic_stack/bpjph/) | cmsbl.halal.go.id | Halal certificates (1.98M+ records) | ๐ | โ
Active โ migrated to REST API (v1.0.0) |\n| [`ahu`](civic_stack/ahu/) | ahu.go.id | Company registry โ PT, CV, Yayasan, Koperasi | ๐ฎ๐ฉ | โ ๏ธ Page restructured โ search input changed (Apr 2026) |\n| [`ojk`](civic_stack/ojk/) | www.ojk.go.id/waspada-investasi | Licensed financial institutions + Waspada list | ๐ฎ๐ฉ | โ ๏ธ Portal migrated to SharePoint (Apr 2026) โ scraper needs rewrite |\n| [`oss_nib`](civic_stack/oss_nib/) | oss.go.id | Business identity (NIB) | ๐ฎ๐ฉ | โ ๏ธ Page restructured โ Playwright can't find inputs (Apr 2026) |\n| [`lpse`](civic_stack/lpse/) | spse.inaproc.id | Government procurement | ๐ฎ๐ฉ | โ
Active โ un-deprecated (v1.0.0) |\n| [`kpu`](civic_stack/kpu/) | infopemilu.kpu.go.id | Election data โ candidates, results, finance | ๐ | โ
Active |\n| [`bps`](civic_stack/bps/) | webapi.bps.go.id | Statistical datasets (1,000+) | ๐ | โ
Active (requires `BPS_API_KEY`) |\n| [`bmkg`](civic_stack/bmkg/) | data.bmkg.go.id | Weather, earthquake, and disaster data | ๐ | โ
Active |\n| [`simbg`](civic_stack/simbg/) | simbg.pu.go.id | Building permits (PBG) โ multi-portal | ๐ | โ
Active |\n| [`jdih`](civic_stack/jdih/) | peraturan.go.id | National legal database โ UU, PP, Perpres, Permen | ๐ฎ๐ฉ | โ
**New** โ Playwright scraping |\n| [`ksei`](civic_stack/ksei/) | web.ksei.co.id | Securities statistics (62 monthly PDFs) + registered securities | ๐ | โ
**New** โ HTML scraping (no proxy needed) |\n| [`djpb`](civic_stack/djpb/) | data-apbn.kemenkeu.go.id | APBN budget themes โ target/realization/achievement | ๐ฎ๐ฉ | โ
**New** โ clean REST JSON API |\n| [`lhkpn`](civic_stack/lhkpn/) | elhkpn.kpk.go.id | Wealth declarations (officials) | โ | โ
Active โ reCAPTCHA v3 solved via Playwright |\n\n๐ = works globally ๐ฎ๐ฉ = requires Indonesian proxy (set `PROXY_URL`)\n\nEvery module returns the same `CivicStackResponse` envelope โ swap data sources without touching application logic.\n\n### Module Maturity\n\n| Module | Scraper | Normalizer | MCP | Tests | Portal Status |\n|--------|:-------:|:----------:|:---:|:-----:|:------------:|\n| bpom | โ
| โ
| โ
| โ
| โ
|\n| bpjph | โ
| โ
| โ
| โ
| โ
REST API |\n| ahu | โ
| โ
| โ
| โ
| โ ๏ธ page restructured |\n| ojk | โ
| โ
| โ
| โ
| โ ๏ธ SharePoint migration |\n| oss_nib | โ
| โ
| โ
| โ
| โ ๏ธ page restructured |\n| lpse | โ
| โ
| โ
| โ
| ๐ฎ๐ฉ geo-blocked |\n| kpu | โ
| โ
| โ
| โ
| โ
|\n| bps | โ
| โ
| โ
| โ
| โ
|\n| bmkg | โ
| โ
| โ
| โ
| โ
|\n| simbg | โ
| โ
| โ
| โ
| โ
|\n| jdih | โ
| โ
| โ | โ | ๐ฎ๐ฉ Playwright |\n| ksei | โ
| โ
| โ | โ | โ
(no proxy needed) |\n| djpb | โ
| โ
| โ | โ | โ
REST JSON API |\n| lhkpn | โ
| โ
| โ
| โ
| โ
Active (Playwright) |\n\n---\n\n## Quick Start\n\n### Install\n\n```bash\npip install indonesia-civic-stack # Core SDK\npip install \"indonesia-civic-stack[mcp]\" # + MCP server (40 tools)\npip install \"indonesia-civic-stack[api]\" # + REST API (FastAPI + uvicorn)\npip install \"indonesia-civic-stack[all]\" # Everything\n```\n\n### Python SDK\n\n```python\nimport asyncio\nfrom civic_stack.bpom.scraper import search as bpom_search\nfrom civic_stack.bmkg.scraper import get_latest_earthquake\n\nasync def main():\n # Search BPOM product registry\n results = await bpom_search(\"paracetamol\")\n for r in results:\n if r.found:\n print(r.result)\n\n # Get latest earthquake\n eq = await get_latest_earthquake()\n print(eq.result) # {'date': '...', 'magnitude': '5.2', ...}\n\nasyncio.run(main())\n```\n\n### MCP Server (for AI agents)\n\nAll 14 modules expose **46 MCP tools** for use with Claude, GPT, or any MCP-compatible agent.\n\n```bash\n# Install locally:\npip install \"indonesia-civic-stack[mcp]\"\nclaude mcp add civic-stack -- civic-stack-mcp\n\n# Or deploy your own remote server (Railway one-click):\n# See \"Self-Hosted MCP Server\" section below\n```\n\nMCP server classes support two init styles:\n\n```python\n# Style 1: Explicit init\nclass BpomMCPServer(CivicStackMCPBase):\n def __init__(self):\n super().__init__(\"bpom\")\n\n# Style 2: Class attribute\nclass BmkgMCPServer(CivicStackMCPBase):\n module_name = \"bmkg\"\n```\n\n### REST API\n\n```bash\n# Run all modules\nuvicorn app:app --port 8000\n\n# With API key auth (recommended)\nCIVIC_API_KEY=your-secret-key uvicorn app:app --port 8000\n\n# Individual module\nuvicorn modules.bpom.app:app --port 8001\n\n# With proxy\nPROXY_URL=socks5://id-proxy:1080 uvicorn app:app --port 8000\n```\n\n```bash\n# Endpoints\nGET /bpom/check/MD123456789012\nGET /bpom/search?q=paracetamol\nGET /bpjph/check/BPJPH-12345\nGET /ahu/search?q=PT+Contoh+Indonesia\nGET /ojk/check?name=Bank+BCA\nGET /kpu/candidate/search?q=Joko\nGET /lhkpn/search?q=Anies # โ
reCAPTCHA v3 solved via Playwright\nGET /bps/search?q=inflasi # Requires BPS_API_KEY\nGET /bmkg/weather?city=jakarta\nGET /simbg/search?q=Jakarta+Selatan\n```\n\n---\n\n## Response Envelope\n\nEvery module returns `CivicStackResponse`:\n\n```json\n{\n \"result\": {\"product_name\": \"...\", \"registration_status\": \"ACTIVE\"},\n \"found\": true,\n \"status\": \"ACTIVE\",\n \"confidence\": 1.0,\n \"source_url\": \"https://cekbpom.pom.go.id/...\",\n \"fetched_at\": \"2026-03-14T06:30:00Z\",\n \"module\": \"bpom\"\n}\n```\n\nStatus values: `ACTIVE`, `EXPIRED`, `SUSPENDED`, `REVOKED`, `NOT_FOUND`, `ERROR`.\n\nWhen a module can't reach its portal or is missing configuration (e.g., `BPS_API_KEY`), it returns an error envelope instead of crashing:\n\n```json\n{\n \"result\": null,\n \"found\": false,\n \"status\": \"ERROR\",\n \"confidence\": 0.0,\n \"source_url\": \"https://webapi.bps.go.id\",\n \"module\": \"bps\",\n \"detail\": \"BPS_API_KEY not set. Register at https://webapi.bps.go.id/developer/register\"\n}\n```\n\n---\n\n## Module Internals\n\n```\ncivic_stack/bpom/\nโโโ __init__.py\nโโโ app.py # FastAPI application\nโโโ normalizer.py # Raw HTML/JSON โ structured dict\nโโโ router.py # FastAPI routes\nโโโ scraper.py # fetch() + search() โ core logic\nโโโ server.py # FastMCP MCP server\nโโโ Dockerfile\nโโโ README.md\n```\n\nThe `shared/` layer provides:\n- **`schema.py`** โ `CivicStackResponse` Pydantic model, status enum, helper constructors\n- **`http.py`** โ `civic_client()` factory with auto-proxy, rate limiter, exponential backoff retry, URL rewriting for CF Worker proxies\n- **`mcp.py`** โ `CivicStackMCPBase` abstract base class for MCP servers\n\n---\n\n## Deployment Notes\n\n### Geo-blocking & Proxy Requirements\n\nMost Indonesian government portals (`*.go.id`) restrict access to Indonesian IP addresses. If deploying outside Indonesia, you **must** set `PROXY_URL` to route requests through an Indonesian endpoint.\n\n```bash\n# Option 1: Indonesian VPS/SOCKS proxy (recommended for production)\nexport PROXY_URL=\"socks5://id-proxy.example.com:1080\"\nexport PROXY_MODE=\"connect\"\n\n# Option 2: CF Worker proxy (free, but limited โ see below)\nexport PROXY_URL=\"https://your-proxy.workers.dev\"\n# PROXY_MODE auto-detects \"rewrite\" for *.workers.dev\n```\n\n**Without a proxy, expect:** DNS resolution failures, connection timeouts, or HTTP 403/404 responses from most modules.\n\nThe SDK auto-reads `PROXY_URL` from environment โ no code changes needed in scrapers or MCP servers.\n\n#### Proxy Modes\n\n| Mode | `PROXY_URL` example | How it works |\n|------|---------------------|--------------|\n| `connect` | `socks5://id-proxy:1080` | Standard HTTP/SOCKS CONNECT proxy via httpx transport |\n| `rewrite` | `https://x.workers.dev` | Rewrites URLs to `?url=` (auto-detected for `*.workers.dev`) |\n| `none` | _(unset)_ | Direct connection |\n\nOverride auto-detection with `PROXY_MODE=connect|rewrite`.\n\n#### CF Worker Proxy\n\nA ready-to-deploy CF Worker proxy is included in [`proxy/`](proxy/). Deploy with:\n\n```bash\ncd proxy && npx wrangler deploy\n```\n\n> **โ ๏ธ CF Worker limitation:** Many `.go.id` portals are themselves behind Cloudflare. CF Workers making `fetch()` calls to other CF-protected origins receive 403/522 errors. This is a known Cloudflare limitation.\n\n**Verified through CF Worker proxy:**\n\n| Portal | Status | Notes |\n|--------|--------|-------|\n| data.bmkg.go.id | โ
Works | JSON API, not behind CF |\n| cekbpom.pom.go.id | โ 403/522 | Portal is CF-protected |\n| api.ojk.go.id | โ DNS dead | NXDOMAIN since March 2026 |\n| infopemilu.kpu.go.id | โ 403 | CF-protected |\n| lpse.*.go.id | โ 403 | CF-protected |\n| elhkpn.kpk.go.id | โ
200 | reCAPTCHA v3 solved via Playwright headless browser |\n\n**For production with CF-protected portals**, use an Indonesian VPS with a SOCKS5/HTTP proxy and set `PROXY_MODE=connect`.\n\n### Geo-Restriction Test Results (March 2026)\n\nTested from three locations to map which portals enforce geo-blocking vs WAF:\n\n| Portal | Sydney (AU) | Singapore | Jakarta (ID) | Verdict |\n|--------|:-----------:|:---------:|:------------:|---------|\n| ahu.go.id | โ | โ
| โ
| Geo-blocked (SEA+ OK) |\n| elhkpn.kpk.go.id | โ | โ
| โ
| Geo-blocked (SEA+ OK) |\n| ojk.go.id | โ 403 | โ 403 | โ
| **ID-only** |\n| jaga.id (KPK) | โ
| โ
| โ
| No restriction |\n| data.bmkg.go.id | โ
| โ
| โ
| No restriction |\n| cekbpom.pom.go.id | โ ๏ธ | โ ๏ธ | โ ๏ธ | CF-protected (all locations) |\n| webapi.bps.go.id | โ 403 | โ 403 | โ 403 | WAF, not geo (needs API key) |\n| lpse.lkpp.go.id | โ | โ | โ | Unreliable (all locations) |\n| coretaxdjp.pajak.go.id | โ | โ | โ | Unreliable (all locations) |\n\n**Takeaway:** An Indonesian proxy (e.g., CloudKilat Jakarta) unlocks OJK โ the most important geo-restricted portal. Singapore unlocks AHU + LHKPN. BPS and LPSE failures are not geo-related.\n\n### VPS Hardening Lesson\n\n> โ ๏ธ **Never disable password auth and restart sshd in one automated script on a fresh VPS.** If the SSH key wasn't copied correctly, you're locked out with no recovery path except a web console. Always: (1) copy key, (2) verify key login works in a *separate session*, (3) *then* disable password auth.\n\n### Portal URL Stability\n\nIndonesian government portals frequently change their URL structure without notice. Known changes as of March 2026:\n\n| Module | Old URL | New URL | Status |\n|--------|---------|---------|--------|\n| BPOM | `/index.php/home/produk/1/{keyword}/...` | `/all-produk?q={keyword}` | โ
Updated |\n| KPU | `/Pemilu/caleg/list` | `/Pemilu/Peserta_pemilu` | โ
Updated |\n| BMKG | `/DataMKG/MEWS/Warning/cuacasignifikan.json` | `/DataMKG/TEWS/gempadirasakan.json` | โ
Updated |\n| LHKPN | `/portal/user/check_search_announ` | reCAPTCHA v3 (Playwright) | ๐ข Active |\n\nModules that fail for **60 days** are flagged `DEGRADED` and may be archived.\n\n### Browser-Based Modules\n\nSome portals require a real browser (JavaScript rendering, anti-bot protection):\n\n| Module | Browser | Anti-bot |\n|--------|---------|----------|\n| bpjph | Playwright (Chromium) | Standard |\n| ahu | Playwright + Camoufox | Bot management (datacenter IP blocking) |\n| oss_nib | Playwright (Chromium) | Standard |\n\nInstall browser dependencies:\n```bash\npip install \".[playwright]\"\nplaywright install chromium\n\n# For AHU (optional, improves success rate):\npip install camoufox && python -m camoufox fetch\n```\n\n### API Keys\n\n| Module | Key Required | Env Var | Registration |\n|--------|-------------|---------|--------------|\n| BPS | Yes | `BPS_API_KEY` | [webapi.bps.go.id/developer/register](https://webapi.bps.go.id/developer/register) (free) |\n| All others | No | โ | โ |\n\nWithout `BPS_API_KEY`, the BPS module returns an error envelope (not a crash):\n```json\n{\"status\": \"ERROR\", \"detail\": \"BPS_API_KEY not set. Register at ...\"}\n```\n\n### MCP Tool Inventory\n\nAll 11 modules expose **40 MCP tools** total:\n\n| Module | Tools | Count |\n|--------|-------|:-----:|\n| bpom | `check_bpom`, `search_bpom`, `get_bpom_status` | 3 |\n| bpjph | `check_halal_cert`, `lookup_halal_by_product`, `get_halal_status`, `cross_reference_halal_bpom` | 4 |\n| ahu | `lookup_company_ahu`, `get_company_directors`, `verify_company_status`, `search_companies_ahu` | 4 |\n| ojk | `check_ojk_license`, `search_ojk_institutions`, `get_ojk_status`, `check_ojk_waspada` | 4 |\n| oss_nib | `lookup_nib`, `verify_nib`, `search_oss_businesses` | 3 |\n| lpse | `lookup_vendor_lpse`, `search_lpse_vendors`, `search_lpse_tenders`, `get_lpse_portals` | 4 |\n| kpu | `get_candidate`, `search_kpu_candidates`, `get_election_results_kpu`, `get_campaign_finance_kpu` | 4 |\n| lhkpn | `get_lhkpn`, `search_lhkpn`, `compare_lhkpn`, `get_lhkpn_pdf` | 4 |\n| bps | `search_bps_datasets`, `get_bps_indicator`, `list_bps_regions` | 3 |\n| bmkg | `get_bmkg_alerts`, `get_weather_forecast`, `get_earthquake_history`, `get_latest_earthquake` | 4 |\n| simbg | `lookup_building_permit`, `search_permits_by_area`, `list_simbg_portals` | 3 |\n\n---\n\n## AI Agent Integration\n\nThis repo is built for AI agents as first-class consumers.\n\n### For AI Coding Agents\n\n| File | Purpose | Agent |\n|------|---------|-------|\n| [`AGENTS.md`](AGENTS.md) | Architecture, patterns, critical rules, gotchas | All coding agents |\n| [`CLAUDE.md`](CLAUDE.md) | Commands, do/don't rules, style guide | Claude Code |\n| [`.cursorrules`](.cursorrules) | Project rules for Cursor | Cursor |\n| [`.github/copilot-instructions.md`](.github/copilot-instructions.md) | Instructions for Copilot | GitHub Copilot |\n| [`CONTRIBUTING.md`](CONTRIBUTING.md) | Module contract + PR checklist | All |\n| [`SKILL.md`](SKILL.md) | Skill discovery (AgentSkills format) | Skill-aware agents |\n| [`PROMPTS.md`](PROMPTS.md) | Example prompts + interactive artifact recipes | All AI agents |\n\n### Connect MCP Tools (Pick One)\n\n**Option A โ Self-hosted remote server (deploy your own):**\n\n[](https://railway.com/template)\n\n```bash\n# After deploying to Railway/Fly/Render, add to Claude Code:\nclaude mcp add civic-stack --transport http https://your-deployment.up.railway.app/mcp\n\n# Or Claude Desktop โ add to claude_desktop_config.json:\n```\n\n```json\n{\n \"mcpServers\": {\n \"civic-stack\": {\n \"transport\": \"streamable-http\",\n \"url\": \"https://your-deployment.up.railway.app/mcp\"\n }\n }\n}\n```\n\n> **Note:** There is no shared hosted server. Each user deploys their own instance to control proxy settings, rate limits, and API keys.\n\n**Option B โ Local install via pip:**\n\n```bash\npip install \"indonesia-civic-stack[mcp]\"\nclaude mcp add civic-stack -- civic-stack-mcp\n```\n\n**Option C โ Clone repo (auto-discovery):**\n\n```bash\ngit clone https://github.com/suryast/indonesia-civic-stack.git\ncd indonesia-civic-stack\npip install -e \".[mcp]\"\nclaude # Claude Code auto-detects .mcp.json โ 40 tools available immediately\n```\n\nAll three options give you the same 40 tools. Then ask:\n\n> \"Check if BPOM registration MD 123456789 is still active\"\n> \"Search for companies named 'Maju Bersama' in the AHU registry\"\n> \"What was the latest earthquake in Indonesia?\"\n\nSee [PROMPTS.md](PROMPTS.md) for more example prompts and interactive artifact recipes.\n\n### REST API\n\n```bash\npip install \"indonesia-civic-stack[api]\"\ncivic-stack api --port 8000\n# GET http://localhost:8000/bpom/search?q=paracetamol\n```\n\n### Example Prompts\n\nOnce MCP tools are connected, try these with your AI agent:\n\n> **Food Safety**\n> \"Check if BPOM registration number `MD 123456789` is still active\"\n> \"Search for all paracetamol products registered with BPOM\"\n\n> **Halal Verification**\n> \"Is product XYZ halal certified? Cross-reference with BPOM registration\"\n> \"Find all halal certificates issued to PT Indofood\"\n\n> **Company Due Diligence**\n> \"Look up PT Maju Bersama in the AHU company registry and check who the directors are\"\n> \"Is this company OJK-licensed? Check both the license registry and the waspada (warning) list\"\n\n> **Public Finance**\n> \"Search LHKPN wealth declarations for officials in Jakarta\"\n> \"Find government procurement tenders for road construction on LPSE\"\n\n> **Disaster & Weather**\n> \"What was the latest earthquake in Indonesia?\"\n> \"Get the weather forecast for DKI Jakarta from BMKG\"\n\n> **Statistics**\n> \"Find BPS datasets about poverty rates by province\"\n> \"Get the inflation indicator for the last 5 years\"\n\n> **Multi-Source Queries**\n> \"I want to verify a food company: check AHU for registration, OJK for financial license, BPOM for product registrations, and BPJPH for halal certificates\"\n> \"Compare LHKPN wealth declarations for these two officials over the last 3 reporting periods\"\n\n### Design Decisions for AI Agents\n\n1. **Uniform response envelope** โ every tool returns `CivicStackResponse` with the same fields. Agents don't need module-specific parsing logic.\n2. **Error envelopes, not exceptions** โ agents receive structured error info they can reason about, not stack traces.\n3. **Self-documenting tools** โ MCP tool descriptions include parameter types, expected values, and response format.\n4. **Deterministic naming** โ `check_`, `search_`, `get__status` pattern across all modules.\n\n---\n\n## Security\n\n| Feature | Config | Default |\n|---------|--------|---------|\n| **API key auth** | `CIVIC_API_KEY` env var | Disabled (open) |\n| **Rate limiting** | `CIVIC_RATE_LIMIT` env var | 60 req/min per IP |\n| **Proxy allowlist** | `CIVIC_ALLOWED_PROXIES` env var | Any non-private IP |\n| **SSRF prevention** | Built-in | Blocks RFC 1918 + localhost |\n| **Container user** | Dockerfile | Non-root (`civicapp`, uid 1000) |\n\n```bash\n# Production deployment\nexport CIVIC_API_KEY=\"your-secret-key\"\nexport CIVIC_RATE_LIMIT=30 # 30 req/min\nexport CIVIC_ALLOWED_PROXIES=\"proxy.example.com\" # optional proxy allowlist\nexport PROXY_URL=\"socks5://id-proxy:1080\" # Indonesian proxy\nuvicorn app:app --host 0.0.0.0 --port 8000\n```\n\n---\n\n## Docker\n\n```bash\ndocker compose up # All modules\ndocker build -t civic-bpom civic_stack/bpom/ # Individual\ndocker run -p 8001:8000 -e CIVIC_API_KEY=secret -e PROXY_URL=socks5://proxy:1080 civic-bpom\n```\n\n---\n\n## Development\n\n```bash\ngit clone https://github.com/suryast/indonesia-civic-stack.git\ncd indonesia-civic-stack\npython -m venv .venv && source .venv/bin/activate\npip install -e \".[all,dev]\"\nplaywright install chromium\n\npytest -v # VCR replay โ no live portal calls\nruff check . # Lint\nruff format --check . # Format check\nmypy shared/ # Type check\n```\n\n---\n\n## Tests\n\n```bash\npytest -v # 89 tests, VCR replay (no live calls)\npytest tests/bpom/ -v # Single module\npytest --tb=short -q # Quick summary\n```\n\n```mermaid\npie title Test Coverage (89 tests)\n \"BPOM\" : 7\n \"BPJPH\" : 8\n \"AHU\" : 12\n \"OJK\" : 4\n \"KPU\" : 5\n \"LPSE\" : 9\n \"OSS-NIB\" : 6\n \"LHKPN\" : 10\n \"BPS\" : 7\n \"BMKG\" : 8\n \"SIMBG\" : 7\n \"Schema\" : 6\n```\n\n---\n\n## Contributing\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md). Every module PR must include:\n- `fetch()` and `search()` returning `CivicStackResponse`\n- FastAPI router + FastMCP server\n- 3+ VCR test fixtures\n- Module README\n\nA module that breaks for **60 days** is flagged `DEGRADED` and archived.\n\n---\n\n## Used By\n\n- [**halalkah.id**](https://halalkah.id) โ Halal product verification (9.57M products)\n- [**legalkah.id**](https://legalkah.id) โ Financial institution legality checker\n- [**datarakyat.id**](https://datarakyat.id) โ Landing page & documentation\n\n## Sample Architectures\n\n### Simple: Halal Product Checker\n\nA single-page app that checks if a product is halal-certified. One module, no proxy needed for Indonesian users.\n\n```mermaid\ngraph LR\n subgraph Client\n A[Mobile App / Web]\n end\n\n subgraph Your Server\n B[FastAPI]\n C[bpjph module]\n end\n\n subgraph Government Portal\n D[sertifikasi.halal.go.id]\n end\n\n A -->|POST /check| B\n B --> C\n C -->|scrape| D\n D -->|HTML| C\n C -->|CivicStackResponse| B\n B -->|JSON| A\n\n style A fill:#f9f9f9,stroke:#333\n style B fill:#e8f5e9,stroke:#2e7d32\n style C fill:#e8f5e9,stroke:#2e7d32\n style D fill:#fff3e0,stroke:#e65100\n```\n\n```python\n# app.py โ 15 lines, production-ready\nfrom fastapi import FastAPI\nfrom civic_stack.bpjph.scraper import fetch\n\napp = FastAPI()\n\n@app.get(\"/check/{product_id}\")\nasync def check_halal(product_id: str):\n result = await fetch(product_id)\n return {\"halal\": result.found, \"data\": result.result}\n```\n\n---\n\n### Intermediate: Multi-Source Due Diligence API\n\nA compliance tool that cross-checks a company across multiple government databases. Runs behind a proxy for overseas deployment.\n\n```mermaid\ngraph TB\n subgraph Client\n A[Compliance Dashboard]\n end\n\n subgraph Your Infrastructure\n B[API Gateway]\n C[Due Diligence Service]\n D[ahu module]\n E[ojk module]\n F[bpom module]\n G[oss_nib module]\n H[(Redis Cache)]\n end\n\n subgraph Proxy Layer\n I[CF Worker Proxy]\n end\n\n subgraph Government Portals\n J[ahu.go.id]\n K[www.ojk.go.id]\n L[cekbpom.pom.go.id]\n M[oss.go.id]\n end\n\n A -->|GET /company/:name| B\n B --> C\n C --> H\n C --> D & E & F & G\n D & E & F & G -->|via PROXY_URL| I\n I --> J & K & L & M\n\n style A fill:#f9f9f9,stroke:#333\n style B fill:#e3f2fd,stroke:#1565c0\n style C fill:#e8f5e9,stroke:#2e7d32\n style D fill:#e8f5e9,stroke:#2e7d32\n style E fill:#e8f5e9,stroke:#2e7d32\n style F fill:#e8f5e9,stroke:#2e7d32\n style G fill:#e8f5e9,stroke:#2e7d32\n style H fill:#fce4ec,stroke:#c62828\n style I fill:#fff8e1,stroke:#f57f17\n style J fill:#fff3e0,stroke:#e65100\n style K fill:#fff3e0,stroke:#e65100\n style L fill:#fff3e0,stroke:#e65100\n style M fill:#fff3e0,stroke:#e65100\n```\n\n```python\n# due_diligence.py โ parallel checks across 4 portals\nimport asyncio\nfrom civic_stack.ahu.scraper import search as ahu_search\nfrom civic_stack.ojk.scraper import search as ojk_search\nfrom civic_stack.bpom.scraper import search as bpom_search\nfrom civic_stack.oss_nib.scraper import search as nib_search\n\nasync def check_company(name: str) -> dict:\n ahu, ojk, bpom, nib = await asyncio.gather(\n ahu_search(name),\n ojk_search(name),\n bpom_search(name),\n nib_search(name),\n )\n return {\n \"company\": name,\n \"registered\": any(r.found for r in ahu),\n \"ojk_licensed\": any(r.found for r in ojk),\n \"bpom_products\": len([r for r in bpom if r.found]),\n \"nib_valid\": any(r.found for r in nib),\n \"risk_flags\": _assess_risk(ahu, ojk, bpom, nib),\n }\n```\n\n---\n\n### Advanced: AI Agent with MCP Tools\n\nAn AI assistant that answers natural language questions about Indonesian civic data using MCP tools. The agent reasons about which portals to query.\n\n```mermaid\nsequenceDiagram\n participant User\n participant Agent as AI Agent (Claude/GPT)\n participant MCP as MCP Server\n participant SDK as civic-stack modules\n participant Proxy as CF Worker Proxy\n participant Gov as Government Portals\n\n User->>Agent: \"Is PT Maju Bersama a legitimate company
with halal certification?\"\n\n Note over Agent: Agent reasons: need AHU (company)
+ BPJPH (halal) + OJK (finance)\n\n Agent->>MCP: search_companies_ahu(\"PT Maju Bersama\")\n MCP->>SDK: ahu.search()\n SDK->>Proxy: GET ahu.go.id/...\n Proxy->>Gov: Forward request\n Gov-->>Proxy: HTML response\n Proxy-->>SDK: Response\n SDK-->>MCP: CivicStackResponse\n MCP-->>Agent: {found: true, status: \"ACTIVE\", ...}\n\n Agent->>MCP: check_halal_cert(\"PT Maju Bersama\")\n MCP->>SDK: bpjph.fetch()\n SDK->>Proxy: GET sertifikasi.halal.go.id/...\n Proxy-->>SDK: Response\n SDK-->>MCP: CivicStackResponse\n MCP-->>Agent: {found: true, status: \"ACTIVE\", ...}\n\n Agent->>MCP: check_ojk_license(\"PT Maju Bersama\")\n MCP->>SDK: ojk.fetch()\n SDK-->>MCP: {found: false, status: \"NOT_FOUND\"}\n\n Note over Agent: Agent synthesizes results\n\n Agent->>User: \"PT Maju Bersama is a registered company (AHU โ
)
with active halal certification (BPJPH โ
).
No OJK financial license found โ this is normal
for non-financial companies.\"\n```\n\n```bash\n# Connect MCP servers to Claude Desktop โ one command per module\nclaude mcp add civic-ahu -- python -m civic_stack.ahu.server\nclaude mcp add civic-bpjph -- python -m civic_stack.bpjph.server\nclaude mcp add civic-ojk -- python -m civic_stack.ojk.server\n\n# Or run unified REST API for HTTP-based agents\nPROXY_URL=https://your-proxy.workers.dev uvicorn app:app\n```\n\n---\n\n## Related\n\n- [**indonesia-civic-signal-monitor**](https://github.com/suryast/indonesia-civic-signal-monitor) โ Anomaly detection engine built on this SDK, monitors 11 government data sources for newsworthy changes\n- [**indonesia-gov-apis**](https://github.com/suryast/indonesia-gov-apis) โ Reference docs for 50+ Indonesian government APIs\n- [**datarakyat.id**](https://datarakyat.id) โ Project homepage with full module documentation\n\n## License\n\nMIT โ see [LICENSE](LICENSE)\n",
"bytes": 29821,
"sha": "4d25eb5f8b1c7cce619d4e9fe58c0acc5ba153f930331be28787ab8aef096876",
"repo_slug": "suryast/indonesia-civic-stack",
"fonte": "repo",
"truncated": false,
"api": "https://agentalog.com/api/listings/mcp_io_github_suryast_indonesia_civic_stack_a9fcca5e/readme"
}