MQTT and Sparkplug on a plant: what the broker guarantees, and what a GMP site must add

A weekly reconciliation meeting on a packaging line, the kind every solid-dose plant holds. The MES count of rejected cartons for the night shift does not match the historian, fed by the new edge gateway: the historian has a couple more. Nobody touched anything; the only event in the shift log is a switch that rebooted for a few seconds. The integrator answers at once: “It can’t be the network, it’s MQTT with QoS 1, delivery is guaranteed.”

He is right about the guarantee and wrong about what it means. QoS 1 promises that a message arrives at least once, not that it arrives only once. A few reject events were sent again after the reconnection; the historian stored them twice, the MES, which deduplicated on an event number, did not. Both systems behaved as configured. The site had simply never decided which of them held the record. Most plants meet some version of this scene in their first year with MQTT.

A transport, deliberately left open

MQTT was invented in 1999 by Andy Stanford-Clark of IBM and Arlen Nipper of Arcom (now Eurotech). The Sparkplug specification recalls the original use case: reliable SCADA communication over satellite links, where polling every point was too slow and too expensive, so remote sites published only what had changed. That design shows today. A client opens one outgoing connection to a broker, publishes messages on topics, and subscribes to the topics it needs. The broker sorts and forwards. Nobody polls anybody.

Two versions matter on a plant. MQTT 3.1.1 became an OASIS Standard on 29 October 2014 and was also published as ISO/IEC 20922:2016. MQTT 5.0 became an OASIS Standard on 7 March 2019; it is not an ISO standard. IANA registers TCP port 1883 for MQTT and 8883 for MQTT over TLS, which is a useful first question on any site survey: which of the two is actually open?

What MQTT does not define is just as important. The payload is a string of bytes; the topic is whatever the publisher chooses to write. That freedom is why so many gateways and cloud platforms speak MQTT, and why two MQTT systems on the same plant often cannot understand each other. The protocol moves data. It does not say what the data is or whether its source is still alive.

The three QoS levels, read from the historian’s side

The OASIS specification defines three levels of quality of service. Each one is a promise about a single hop: publisher to broker, or broker to subscriber. It is not a promise from the transmitter to the database.

Three details catch people out. First, the broker delivers to a subscriber at the lower of the QoS the message was published with and the maximum QoS that subscriber was granted. A sensor publishing at QoS 2 to a dashboard subscribed at QoS 0 gets QoS 0 on the last hop. Second, ordering survives resends only loosely: the specification’s own example shows a subscriber receiving 1, 2, 3, 2, 3, 4 after a disconnection. Third, nothing at any QoS level tells the historian that a value it never received should have existed.

For a GMP site, the practical reading is simple. Decide per data type. A trended temperature where the next value comes in a second can live with QoS 0. A count, a reject, an alarm acknowledgement or a batch event cannot, and whichever QoS is used, the consumer needs a key to recognise a duplicate. The consequence for an MES receiving weighing events is discussed in when the MES goes down.

What Sparkplug adds on top

Sparkplug is the Eclipse Foundation’s answer to MQTT’s open payload. Version 3.0.0 of the specification is dated 16 November 2022, and it was transposed into an international standard, ISO/IEC 20237:2023, through the ISO/IEC JTC 1 procedure for publicly available specifications; the Eclipse Foundation announced the publication on 7 November 2023 and keeps stewardship of the text. Products can be tested against the specification with a Technology Compatibility Kit before being listed as compatible.

In plant terms, Sparkplug fixes four things.

• A topic structure. Every topic follows spBv1.0/group_id/message_type/edge_node_id/device_id. A consumer knows where a message comes from without a spreadsheet of conventions.
• A declared data model. When an edge node connects, its birth certificate (NBIRTH, then DBIRTH per device) lists every metric with its name, data type and properties. Data messages that follow only carry what changed.
• A timestamp per value. Each metric carries a UTC timestamp which, in the specification’s words, represents the time at which its value was captured, separate from the time the message was published.
• Session state. The death certificate is registered with the broker as the MQTT Will Message when the node connects, so the broker publishes it even when the node vanishes without saying goodbye.

Here is a point the marketing rarely mentions. Sparkplug requires NDATA and DDATA messages to be published at QoS 0. Integrity does not come from the delivery level; it comes from the sequence number, which runs from 0 to 255 and wraps round. A host application that sees a gap waits for a configurable reorder timeout, then asks the edge node to publish its birth certificates again. Loss is detected and repaired by the application, not hidden by the transport. That is a sounder design for plant data than “QoS 1 everywhere”, provided the host really implements it.

When a node goes quiet

On a conventional SCADA, a dead PLC connection shows up as a communication alarm and greyed values. With MQTT alone, a gateway that loses power simply stops publishing, and every dashboard keeps showing the last value it received, perfectly plausible and completely out of date.

The Sparkplug death certificate exists for exactly that case. When the broker detects the lost session, it publishes NDEATH and every host application can mark that node’s metrics as stale. The question is whether anyone on the floor sees it. Done well, the HMI shows a quality flag and the historian a stale flag, so a trend shows a hole rather than a flat line. Done badly, the death certificate is logged by a service nobody reads and the operator trusts a frozen reading. Pull the power on a gateway during qualification and watch each screen: ten minutes, and more telling than the design document.

Where the timestamp is taken

A value can carry three times: when it was captured, when the edge node published it, and when the historian stored it. In normal running they differ by milliseconds. After an outage bridged by store-and-forward, they differ by the length of the outage, and that decides whether an excursion appears when it happened or when the link came back.

For data integrity this is not a detail. A contemporaneous record is one dated when the event occurred. The consumer must therefore store the capture timestamp carried in the payload, not its own arrival time, and the clocks at the edge must be trustworthy. NIST SP 800-82 Rev. 3 makes the point in its time synchronisation section: an inaccurate device clock makes every timestamp it generates inaccurate, and a common time source should be used across OT devices. On site, that means the edge gateways belong in the NTP design, with their drift monitored like any other GMP clock.

Store-and-forward is a decision, not a feature

Sparkplug lets an edge node keep data while its primary host is offline and send it later, flagged with is_historical so that it is not mistaken for a live value. The specification does not say how much to keep, for how long, or what happens when the buffer is full. Those answers belong to the site.

Work them out before go-live, with the people who own the data. What is the longest outage you plan for? When the buffer fills, does the gateway drop the oldest values or the newest, and does it say so? Does the MES accept late values at all once the batch has moved on? An undocumented default here becomes, sooner or later, a gap in a batch record that nobody can explain.

Who owns the namespace

Sparkplug fixes the shape of the topic. It does not choose the names. group_id and edge_node_id are the site’s to define, and once dashboards, reports and analytics subscribe to them, renaming a node is a change that breaks consumers you may not know about.

Give the namespace an owner, a written naming rule and a place in change control. The same naming then serves security: the Sparkplug specification suggests broker access control lists that let an edge node publish only on its own group and node topics, so that one client cannot easily impersonate another, by malice or by mistake.

The broker is a system to qualify

On many projects the broker starts life as a container on a spare virtual machine and, six months later, carries every value the plant produces. Security first. Neither MQTT nor Sparkplug imposes security; the Sparkplug specification explicitly defers to the underlying protocols and industry practice. In practice that means TLS on 8883 and port 1883 closed, an identity for every client rather than one shared password, access control lists per topic branch, and logs of connections and refused publications. Placement matters as much: NIST SP 800-82 Rev. 3, in its example architectures, routes traffic between the enterprise and operations levels through services in a DMZ. A broker that both plant gateways and cloud applications reach is exactly such a service. How to draw the zones around it is covered in zones and conduits: where to start.

Then GMP. EU GMP Annex 11 (the 2011 text still in force; a revision went to public consultation from July to October 2025) asks, for critical systems, for an up-to-date description of data flows and interfaces (§4.3), for built-in checks when systems exchange data electronically (§5), and for checks that data are not altered in value or meaning when transferred to another system (§4.8). A broker carrying GxP data sits squarely in that flow. The record lives in the historian, the MES or the LIMS, but the path to it runs through the broker, and the data-integrity assessment has to show what happens on that path when things go wrong. The broader argument is in data integrity by design.

And OPC UA, since the question always comes up? It brings a richer information model and, in its usual form, a client reading a server; MQTT with Sparkplug distributes events to many consumers. Many plants run both, OPC UA near the controllers and MQTT further up. The harder question is who needs the data and who answers for it.

Where to start

1. Draw the data path for one GxP-relevant value, from instrument to record, naming every gateway, broker and consumer, and put it into the system description.
2. Fix the QoS per data type and give every event a key the consumer uses to drop duplicates.
3. Pull the plug on a gateway during qualification and record what the HMI, the dashboard and the historian show.
4. Write the store-and-forward parameters in the specification: buffer size, outage covered, overflow behaviour, late-data handling.
5. Bring the edge clocks into the NTP design and check that consumers store the capture timestamp.
6. Treat the broker as a qualified system: TLS, per-client identity, access control lists, logs, a backup and a restore test.

Five common mistakes

• Reading “guaranteed delivery” as “exactly one record”. QoS 1 can duplicate, QoS 0 can lose, and both only cover one hop.
• Dashboards with no stale state. A frozen value that looks live is worse than a blank.
• Storing arrival time instead of capture time, which moves every buffered excursion to the moment the network came back.
• Leaving store-and-forward on its default, so the first long outage discovers the buffer size for you.
• A broker nobody owns: anonymous clients, port 1883 open, no change control on the namespace.

MQTT will carry whatever you give it, as often as the network allows. Deciding what counts as the record, and proving it survives a bad night, is still the plant’s work.

References

• Arlen Nipper, MQTT and Sparkplug 3.0: The Future of Industrial OT-IT Integration, AUTOMATION 2023 Volume 6, November 2023: automation.com (the feature this article starts from).
• OASIS, MQTT Version 5.0, OASIS Standard, 7 March 2019, §1.8, §4.3, §4.6: docs.oasis-open.org.
• OASIS, MQTT Version 3.1.1, OASIS Standard, 29 October 2014: docs.oasis-open.org.
• Eclipse Foundation, Sparkplug Specification, version 3.0.0, 16 November 2022: sparkplug.eclipse.org.
• Eclipse Foundation, Sparkplug as an International Standard (ISO/IEC 20237), 7 November 2023: newsroom.eclipse.org.
• European Commission, EudraLex Volume 4, Annex 11: Computerised Systems (2011), §4.3, §4.8, §5: health.ec.europa.eu.
• NIST, SP 800-82 Rev. 3, Guide to Operational Technology (OT) Security, September 2023, §5.4 and §6.2.12: csrc.nist.gov.