THE TRILOGY GOT A SEQUEL NOBODY ORDERED
The last three posts in this series were, in order, about money, bytes and privacy. Who gets paid in a national meter rollout. How heavy a properly described meter reading becomes when you write it down responsibly. And what a fine-grained meter quietly learns about the household on the other end of the wire.
After part three we received a fair question, which we will paraphrase politely: "You keep explaining where the semantic layer belongs. Have you actually built one for meter data, or is this a philosophy blog now?"
Fair. Completely fair.
So this post contains no philosophy. We took some time, generated a million synthetic meter readings, and pushed them through our pipeline twice: once into the format we use in production every day, and once into the format the European energy world is standardising on. Then we let a validator judge the result, because our own opinion of our own output is worth exactly nothing.
Here is what happened, including the part where the ontology surprised us.
THE SETUP: A MILLION READINGS THAT NEVER EXISTED
First, the data. We do not have a utility's permission to publish experiments on real household readings, and after part three of this series you know exactly why we would not want to anyway.
So we simulated. One thousand virtual meters, fifteen-minute intervals, a million readings in total. The payloads are synthetic but the format is not: real DLMS/COSEM OBIS register codes, the same dialect the AMI head-ends emit for the meter families installed across Europe. Register 1-0:1.8.0 for total imported energy. 1-0:2.8.0 for export. 1-0:1.7.0 for instantaneous power. 1-0:32.7.0, 52.7.0 and 72.7.0 for the three phase voltages. If you have ever read a head-end export, you have seen these codes. If you have not, congratulations on your life choices.
Synthetic data with an authentic format is the honest way to do this. The pipeline cannot tell the difference. The privacy regulator can.
RUN ONE: THE PRODUCTION PATH
The first run is the boring one, which is the highest compliment we can give a data pipeline.
CASSIOPEIA, the mapping engine inside our Vela Context Tools line, ingested the OBIS payloads and produced NGSI-LD entities against the Smart Data Models catalogue. The target model is ThreePhaseAcMeasurement from the energy domain, which turns out to be an almost embarrassingly good fit for a meter register set: total energy import and export, active power, and per-phase voltages, all first-class attributes with proper units and timestamps.
The numbers, on a single 16-core machine (an Apple M3 Max laptop, 64 GB of RAM, release build, no cluster, no GPU, no excuses):
- Nested JSON input: around 150,000 readings per second through ingestion, mapping and validation.
- The same data as CSV: around 364,000 readings per second, because flat files do not make the parser think.
A million readings, mapped and validated, in single-digit seconds. For calibration: one million meters at fifteen-minute cadence produce roughly 1,100 readings per second nationally. The mapping stage is not where a national metering backend goes to die. We wrote in part two about where it actually goes to die, and it is the storage and the AI floor above it, not the translation.
One engineering footnote for the three people who will ask: JSON is harder to parallelise because the whole document is parsed before records flow. Newline-delimited JSON would stream. This was a proof of concept, and there is headroom on both paths.
RUN TWO: THE PART WHERE THE ONTOLOGY TALKED BACK
The second run is why this post exists.
The European semantic vocabulary for energy is SAREF, an ETSI standard, with SAREF4ENER as its energy extension. We built a reference implementation of a SAREF output stage and mapped the same source records into SAREF RDF, then validated every generated entity with SHACL shapes.
And here we did something that, judging by a lot of code we have seen in the wild, is apparently optional: we validated against the actual published ontologies. SAREF core v3.2.1 and SAREF4ENER v1.2.1, loaded, with a check that every term we used actually exists in them.
Two findings ruined an afternoon, in the best possible way.
First: saref:Voltage does not exist. It never did. A voltage property feels so obviously like something an energy ontology would define that we had written it into an early draft without checking. The ontology disagreed, politely, by not containing it.
Second: saref:Power is deprecated. Present, but marked as on its way out.
The current SAREF guidance points instead to QUDT quantity kinds and units, which is what we switched to. Voltage becomes a QUDT quantity kind, kilowatt-hours become QUDT units, and the whole graph suddenly agrees with the standard instead of with our assumptions about the standard.
After the correction: every generated entity passes SHACL validation against the published ontologies. The validator's verdict, verbatim, is the single most beautiful word in semantic engineering: conforms.
We are sharing the mistake on purpose. Anyone who tells you their SAREF integration worked on the first try has either validated nothing or is describing a diagram. The entire value of a machine-checkable vocabulary is that it talks back. Ours talked back, we listened, and now the output is right by construction rather than by opinion.
WHAT THIS DOES AND DOES NOT PROVE
Honesty section, as always.
What it proves: the critical path works. Vendor-dialect meter data goes in, and two standards-conformant representations come out of one pipeline: NGSI-LD Smart Data Models, which we run in production, and SAREF RDF, which the validator certifies. The throughput numbers say the translation layer is never the bottleneck, even at cadences nobody has regulated into existence yet.
What it does not prove: that anyone should convert a national meter firehose into RDF and store it. They should not. We wrote an entire post about the coal plant you would need. The semantic layer belongs at the boundaries, where data crosses between organisations that never pre-agreed on a dialect: a household sharing its data with an energy community, an operator publishing aggregates to a market, an event stream telling a third-party service that something on a feeder looks wrong. Low volume, high stakes, exactly where a conformance stamp earns its bytes.
The million-record run was not a deployment recommendation. It was a stress test with a margin note: even if you insisted on doing the wrong thing, the mapper would keep up. Please do not insist.
THE KISS CONCLUSION
The Pareto beat of this series says invest 20 percent, gain 80 percent. This time the split was more extreme.
About 20 percent of the effort in this proof of concept was mapping logic. The other 80 percent was the unglamorous discipline around it: authentic source formats, real published ontologies instead of remembered ones, term-existence checks, SHACL shapes, and measured numbers on named hardware instead of adjectives.
The 20 percent makes a demo. The 80 percent makes a claim you can put your name under.
The meters keep talking. Now, at least in our lab, both of Europe's semantic dialects understand what they are saying.
APPENDIX:
THE BORING TECHNICAL STUFF, FOR THE THREE OF YOU WHO SCROLLED
Everything above was prose. This chapter is receipts. If JSON makes you itch, you already have everything you need, thank you for reading, the appointment button is below.
1. What goes in: one reading, as the head-end speaks it
One synthetic payload, byte-compatible with what an AMI head-end emits downstream. The register codes are real DLMS/COSEM OBIS. The values are invented. The format cannot tell you which is which, and neither could our pipeline.
JSON
{
"deviceId": "SIM-AM550-0000000001",
"vendor": "IskraAM550",
"gateway": "dc-gw-01",
"protocol": "DLMS/COSEM",
"ts": "2026-08-10T00:00:00+02:00",
"registers": [
{
"obis": "1-0:1.8.0",
"value": 19332.701,
"unit": "kWh",
"desc": "Active energy import, total"
},
{
"obis": "1-0:1.7.0",
"value": 0.34,
"unit": "kW",
"desc": "Instantaneous active power import"
},
{
"obis": "1-0:32.7.0",
"value": 228.4,
"unit": "V",
"desc": "Voltage L1"
}
],
"meta": {
"intervalMin": 15,
"fw": "1.2.5",
"signalQuality": 91
}
}2. The mapping: declarative, versioned, diffable
The mapping is data, not code. A rule file tells the engine which register lands on which Smart Data Models attribute, with which unit code and which timestamp. Excerpt from the mapping used in the run.
JSON
{
target: "ThreePhaseAcMeasurement", // dataModel.Energy
id: "urn:ngsi-ld:ThreePhaseAcMeasurement:{deviceId}",
observedAt: "{ts}",
attributes: {
totalActiveEnergyImport: { from: "1-0:1.8.0", unitCode: "KWH" },
totalActiveEnergyExport: { from: "1-0:2.8.0", unitCode: "KWH" },
activePower: { from: "1-0:1.7.0", unitCode: "KWT" },
phaseVoltage: {
L1: { from: "1-0:32.7.0", unitCode: "VLT" },
L2: { from: "1-0:52.7.0", unitCode: "VLT" },
L3: { from: "1-0:72.7.0", unitCode: "VLT" }
}
}
}Two boring details that matter more than they look. Every attribute carries a unitCode and every value carries observedAt, because the SAREF stage downstream refuses measurements without unit and timestamp. Data quality problems are cheapest at the moment of mapping and most expensive three systems later.
3. The run: a million records, stage by stage
One million OBIS readings from nested JSON. Each pipeline stage reports records processed, elapsed time and rate.
The same million readings from CSV. Flat input, no per-record register lookup, roughly 2.4x the transform rate.
How to read the stage lines, since the numbers tell a small story. The Ingestor, Expander and Resolver stages run at about 185k records per second on JSON. Transformer and Validator settle around 150k, which is the number we quote as end-to-end mapping throughput. The Writer stage, which persists entities to the store, finishes at about 71k per second and is the honest bottleneck of the whole run. On CSV the same shape repeats at higher rates: ingest around 663k, transform and validate around 365k, writer around 98k. Parsing is the JSON tax, persistence is everyone's tax.
4. What comes out, dialect one: NGSI-LD Smart Data Models
JSON
{
"id": "urn:ngsi-ld:ThreePhaseAcMeasurement:SI-AM550-0037821460",
"type": "ThreePhaseAcMeasurement",
"totalActiveEnergyImport": {
"type": "Property", "value": 18452.734,
"unitCode": "KWH", "observedAt": "2026-08-10T12:15:00Z"
},
"activePower": {
"type": "Property", "value": 4.382,
"unitCode": "KWT", "observedAt": "2026-08-10T12:15:00Z"
},
"phaseVoltage": {
"type": "Property",
"value": { "L1": 231.4, "L2": 230.9, "L3": 232.1 },
"unitCode": "VLT", "observedAt": "2026-08-10T12:15:00Z"
}
}
5. What comes out, dialect two: SAREF, after the ontology corrected us
The excerpt below is the shape that survived validation. Note what is not there: no saref:Voltage, because the ontology never defined one, and no saref:Power, because the ontology deprecates it. Quantity kinds and units come from QUDT, per current SAREF core guidance.
@prefix saref: <https://saref.etsi.org/core/> .
@prefix quantitykind: <http://qudt.org/vocab/quantitykind/> .
@prefix unit: <http://qudt.org/vocab/unit/> .
@prefix ex: <https://velacontext.com/data/> .
ex:Meter/SI-AM550-0037821460 a saref:Meter ;
saref:makesMeasurement ex:M/SI-AM550-0037821460/Voltage-L1/T1415 .
ex:M/SI-AM550-0037821460/Voltage-L1/T1415 a saref:Measurement ;
saref:hasValue "231.4"^^xsd:decimal ;
saref:hasTimestamp "2026-08-10T12:15:00Z"^^xsd:dateTime ;
saref:relatesToProperty quantitykind:Voltage ;
saref:isMeasuredIn unit:V ;
saref:measurementMadeBy ex:Meter/SI-AM550-0037821460 .
```
6. The judge: SHACL, loaded with the real ontologies
Shapes enforce that every measurement carries a value, a timestamp, a quantity kind, a unit and an originating meter, and a term-existence check confirms that every class and property we use actually exists in SAREF core v3.2.1 and SAREF4ENER v1.2.1 as published by ETSI. The final line of the validation report, unedited:
```
Conforms: True
```
We would like to pretend it said that on the first run. It did not. The first run is where saref:Voltage went to not exist. The validator is the whole point.
7. Reproducibility notes
Hardware: MacBook Pro, Apple M3 Max, 16 cores (12 performance, 4 efficiency), 64 GB RAM, macOS, CASSIOPEIA release build.
Dataset: 1,000 simulated meters, 15-minute cadence, one million records, generated by a standalone publisher.
Ontologies: SAREF core v3.2.1, SAREF4ENER v1.2.1.
Validation: SHACL with term-existence checking, inference off.
Numbers are from single sustained runs, not best-of-N.
DISAGREE, COMMENT, OR WISH TO KNOW MORE?
Curious what a validated semantic boundary layer would look like on your metering data, in your formats, against the standards your regulator names? We have a pipeline warmed up and a validator with opinions.
