Data
VERSION 0.1 · 2026-10-12 · developer.nrgon.net/data
The second layer: how data is stored and where each kind goes. Everything above reads from here.
Formats
| format | used for | rule |
|---|---|---|
| JSON | one message, one record, one API response | UTF-8, one object per file, keys in snake_case, units in the key name |
| JSONL | many records of the same kind in one file: a day of telemetry, an event log | one JSON object per line, append only, no header |
| CSV | tables people open in a spreadsheet: interval load, site lists, catalogues | UTF-8, comma, header row, ISO 8601 dates, no thousands separators |
| YAML | configuration and definitions a person edits: sites, assets, value cases, data requests | in git, comments allowed, one document per file |
| Markdown | text: READMEs, reports, notes | plain, tables for facts |
| documents as received and reports as delivered | kept unchanged; values extracted into JSON next to it | |
| XLSX, images, drawings | as received from partners and sites | kept unchanged in the bucket or the Drive folder |
Time is ISO 8601 in UTC, 2026-09-24T10:04:43Z. Numbers carry the unit in the field
name: power_kw, energy_kwh, soc_pct, temp_c.
Where data goes
| kind | example | goes to | as |
|---|---|---|---|
| live data | battery state every minute, a swap event, an alarm | bucket, one object per message; Redis, latest value per asset | JSON per message; JSONL per day for analysis |
| one-time dump | 12 months of interval load from a utility, an asset list from a partner | bucket, under <source>/dump/<date>/, as delivered, with a manifest.json naming each file and its origin |
CSV, XLSX, ZIP, as received |
| documents | bills, specifications, single-line diagrams, contracts | Drive folder of the project, so the copy people edit is the copy used; extracted values to the bucket | PDF, DOCX, XLSX |
| site photos and notes | installation photos, meter readings, handover notes from on-site teams | Drive folder, one subfolder per site and visit. Taken on a phone, uploaded from the Drive app | JPG, HEIC, Markdown |
| configuration | sites, assets, value cases, tariffs to apply, data request lists | git, next to the component that reads it | YAML |
| reports | proposal, weekly report, final report | bucket under <project>/reports/; a copy in the Drive folder for the partner |
PDF and Markdown |
On-site operators work in the Drive folder only. They never need a bucket or a key.
Open data
data.nrgon.net publishes reference data for Singapore,
Malaysia, Thailand, Vietnam, Indonesia and the Philippines: countries, utilities, tariff
structures, time-of-use schemes, national statistics and grid reliability. JSON over
HTTPS under /v1/, with a stated review cadence per dataset. Components read tariffs and
time-of-use bands from here rather than carrying a copy.
Rules
- Stored as received. The raw message or file is kept unchanged. Parsing happens on read, so a parser fix applies to history.
- Once. A repeated message, identified by its own id, exists once.
- Dated. Every object carries the time it was received, in UTC, in its key.
- Units stated. A number without a unit is not stored.
- Traceable. A derived value names the objects it was computed from.