datapulse-my

DataPulse is an open, read-only verification layer for Malaysian public data. It does not republish official datasets; instead it repeatedly probes the sources, records what it actually observes, and publishes machine-readable evidence about each dataset so that humans and AI agents can judge whether a figure is safe to reuse or cite. The live dashboard is at data-pulse.my. Catalogue The project tracks 418 official Malaysian datasets, organised by stable publisher ID (custodian). Publishers include Bank Negara Malaysia, the Department of Statistics Malaysia (DOSM/OpenDOSM), the Accountant General's Department, the Department of Environment, the Department of Agriculture and Fisheries, the National Anti-Drugs Agency, and others. The inventory spans economic and financial series (OPR, exchange rates, interest rates, inflation, money aggregates, payment systems), demographic and health statistics, crime, agriculture, and environmental monitoring. Thirty of the entries are GTFS transit feeds (KTMB, Prasarana, BAS.MY), covering both static schedules and realtime vehicle positions. Each dataset has a documentation page, and some include CSV or JSON samples. Health taxonomy Instead of a blanket green checkmark, every dataset carries one of ten honest statuses: fresh, aging, stale, discontinued, degraded, browser-dependent, unreachable, unknown, unknown-freshness, or reference. A source that works but cannot be shown to be current is labelled unknown-freshness rather than assumed healthy; a series that a publisher has stopped updating is marked discontinued (a publisher decision, frozen data) rather than treated as a freshness failure. Reference rows describe versioned lookup data where date-based freshness does not apply, with variants such as policy-reference and reference-current. A public _trust_summary reports the current distribution and explicitly counts missing freshness and row-count signals. Health changes can be followed via an RSS feed. Licences and provenance Every dataset declares its licence in machine-readable form, reproducible from datapulse.json. The stated distribution includes Creative Commons Attribution 4.0, Open Government Licence (Malaysia), MIT, Singapore Open Data Licence, MBPP Government Open Data Terms, and some rows where the publisher licence is not stated. Probing follows a tiered, cadence-aware schedule (for example daily, weekly for fuel prices, monthly for surveys), and each dataset carries a human-readable steward plus a stable custodian ID for publisher provenance. Verification The project describes three checkable layers: an Ed25519-signed envelope over the exact content of each observation, a source-of-record check where the served observation byte-matches the versioned Git source, and a temporal witness in the form of a Rekor/Sigstore public-log inclusion proof. A standalone script (scripts/verify_external.py) can be downloaded and run without checking out the repository or using any DataPulse code. The README is explicit about scope: the platform proves the integrity and timing of its observations, not that an upstream government figure is semantically true. Agent access DataPulse exposes a read-only MCP server at https://mcp.data-pulse.my/mcp over Streamable HTTP with no authentication. It advertises 19 read-only tools over the 418-dataset catalogue, including search_datasets, get_dataset, get_data_passport, find_stale, find_anomalies, find_deteriorating, find_recovering, find_unreliable, find_schema_drift, check_reconciliation, get_provenance, get_evidence, verify_dataset, get_freshness_summary, verify_evidence, trust_verdict, verify_attestation, find_by_licence and usage_summary. Connection from Claude Desktop, Cursor or Cline is a single JSON config block. Public discovery files include llms.txt, agent.json, mcp.json and a sitemap. Browser-dependent sources Five sources (about 1.4% of the catalogue) render client-side JavaScript and are probed with Camofox, a self-hosted patched headless-Chromium sidecar configured through the CAMOFOX_BASE_URL environment variable. Without it those datasets honestly remain at browser-dependent status. Intended audience The project is aimed at AI builders and agent developers who want freshness and licence checks before a model cites a Malaysian figure; researchers, analysts and journalists who need to ground work in verifiable data; compliance and regulatory-monitoring teams needing a tamper-evident record that a figure was checked at a known time; and civic technologists seeking a transparent view of how discoverable and reliably described public data is. Probing is described as read-only and lawful: publicly available sources only, rate-limited, self-identifying, and never bypassing authentication.