Historian data in the cloud: which copy is the record, and what the analytics are allowed to decide

A deviation meeting on a Thursday afternoon. A temperature excursion on a water-for-injection loop is under investigation, and the reliability engineer has brought the curve from the new cloud dashboard, the one built for predictive maintenance on the loop pumps. The quality lead looks at it, then asks the automation engineer to open the same tag in the site historian. The two curves do not quite agree. The peak is lower in the cloud, it sits an hour away from where the historian puts it, and a short spike visible on site is missing.

Nobody did anything wrong. The cloud copy was built for maintenance: values resampled to one per minute, timestamps converted on the way, quality flags dropped because the model did not need them. The scene is a composite, but its question comes up on every site that sends plant data to the cloud. Which of these two curves is the record?

A sound argument with one subject left out

In a June 2022 InTech feature, Megan Buntain, then director of cloud partnerships at the analytics vendor Seeq, argued against emptying every historian into one central cloud data lake. Her design keeps historians on premises for real-time needs, adds an abstraction layer that queries data where it lives, and streams only selected data to the cloud for machine learning. Her company sells that kind of layer, so it is a vendor’s position, but most automation engineers would recognise the reasoning.

What the feature does not discuss is that on a pharmaceutical site some of that historian data is a GMP record: environmental monitoring of classified rooms, clean utility temperatures and conductivity, autoclave cycles, cold-room mapping. For that data, the freedom to replicate, resample and query from anywhere has to be decided, documented and validated.

One word needs a warning. When cloud vendors say hybrid architecture they mean on-premises plus cloud. When PIC/S PI 041-1 says hybrid system it means an electronic system supplemented by a defined manual system that produces paper records. Both terms will end up in the same validation file, so define them on its first page.

Original, true copy, or neither

The MHRA’s 2018 GXP guidance defines the original record as the first or source capture of data, with all the subsequent data needed to reconstruct the GXP activity (§6.11.1). A true copy is verified, by a dated signature or by a validated process, to hold the same information as the original, including the data that describe its context, content and structure; it may change file format but must keep the metadata and audit trail (§6.11.2). The FDA’s 2018 questions and answers say the same in their own words (question 9).

For most continuous data on a site, the historian archive is the first durable capture, and in practice the original record. Everything downstream is then a true copy, made by a validated process and carrying the metadata; an ordinary copy, useful but unable to stand in for the record; or a view, which reads the original and stores nothing of its own.

The third route is the interesting one for integrity. With no second population of data, the question of which copy is the record does not arise; the burden moves to the query and the calculation. Annex 11 §5 asks for built-in checks wherever systems exchange data, and PIC/S PI 041-1 §9.4 asks interfaces to be assessed during validation. A query layer is an interface.

What a historian actually keeps

The gap between the two curves in the opening scene comes from ordinary historian habits that are rarely written into a cloud project’s requirements.

• Filtering at the source. Many historians do not store every value they receive. In AVEVA’s own description of its PI System, exception reporting applies a simple deadband before values are sent, and compression then uses the swinging-door algorithm to drop points that can be recreated by interpolation. The vendor’s advice is to keep the compression deviation at or below the instrument’s precision. On a GMP tag, those settings are part of the validated configuration, and a change to them is a change to what the record contains.
• Interpolation at retrieval. Ask a historian for one value per minute and it will usually calculate values between archived points. That is what most streaming connectors and dashboards do. The result is a smooth, regular series that was never recorded as such, and a short spike between two sample times simply disappears.
• Context that lives in the historian. Engineering units, quality or status codes (bad input, substituted, questionable), tag descriptions, and their history. A tag renamed or rescaled during a project changes the meaning of the series around it. If the cloud copy keeps the values and not the context, it is the number without the record.

Time explains the hour. Historians, cloud platforms and dashboards each choose a time base: UTC, local time, or local time with an offset. Convert once without the offset and the October clock change gives a duplicate hour in one layer only. PIC/S PI 041-1 asks for clocks synchronised with connected systems (§9.5) and notes that global systems may need the time zone recorded (§9.8); the detail is in our article on metadata. Write down the time base of every layer before the first tag is sent.

Edits come last. Historians allow annotations, substitutions, backfill after a buffer outage and, with the right rights, deletion. If that happens on site after the data was streamed, the cloud holds the pre-edit version, with nothing to tell the reader so.

Insight is not a decision

The useful distinction on a regulated site is not cloud against on-premises. It is what the analytics are used for. A model that predicts pump bearing wear or an energy dashboard for the HVAC produces insight; its output leads to an action recorded elsewhere, a work order or an engineering study. It needs sound engineering and IT controls, not GMP validation of every calculation.

Use the same chart to close a deviation or feed a continued process verification report, and it has become part of a GMP decision: the data must be the record or a true copy, and the calculation validated and reproducible. The pragmatic rule on most sites is simple. The cloud tells you where to look; the decision is taken on the record.

The trouble starts in the grey zone, when a dashboard built for insight is so convenient that people quote it in investigations, as in the opening scene. Label non-GMP dashboards as such, and say in the deviation procedure which sources an investigation may cite.

The provider, the contract and the regulation in force

Once GMP data passes through a cloud service, the provider is a third party under Annex 11 §3.1, which needs formal agreements and treats internal IT the same way; the FDA counts cloud infrastructure among the systems of 21 CFR 211.68 (question 1.f). The MHRA is the most concrete (§6.20): data location and applicable law, timely access to data, metadata and audit trails for the owner and the authorities, archiving for the retention period, and tested business continuity.

The regulation is moving, and its status should be stated exactly. The European Commission consulted on a revised Annex 11, a revised Chapter 4 and a new Annex 22 on artificial intelligence from 7 July to 7 October 2025. The draft Annex 11 cites the growing use of cloud services, asks for oversight of service providers and internal IT through service level agreements and performance indicators (§7.3), and wants the time zone in audit trails where applicable (§12.2). It is still a draft: the Commission’s EudraLex page still lists Annex 11 in its January 2011 revision. Design towards the draft; do not cite it as a requirement.

The connection is an OT conduit too

An abstraction layer that queries plant historians from the cloud is a remote access path into the site, whatever the brochure calls it. The usual answer is a historian replica in the demilitarised zone, fed outbound from the control network, with the cloud reading only the replica (see Purdue, ISA-95 and IEC 62443). That replica is one more hop, with the same questions: complete, synchronised, quality flags kept?

Where to start

1. List the use cases, then the tags. For each cloud use case, write what it is for and whether its output can enter a GMP decision. The tag list follows from that, not the other way round.
2. Draw the data flow per use case. From sensor to the dashboard, with every hop, its time base, what it filters or resamples, and what it drops. This is the method of data integrity by design, applied to one more destination.
3. Name the record. In the system inventory, state which layer holds the original for each GMP data set and what status every other layer has: true copy, copy, or view.
4. Fix the historian configuration. Exception and compression settings, units, quality codes and tag naming for GMP tags under change control, with the reasoning written down.
5. Test the copy against the original. Same period, same tag, archived events against archived events, including a clock-change night and a buffer outage. Do it at validation and at each periodic review.
6. Write the contract last, with the answers in hand. Data location, access for inspectors, retention, restore tests and exit, as in the table above.

Five common mistakes

• Treating the cloud copy as a backup. The FDA uses the word backup for a true copy kept securely for the whole retention period, exact and complete (question 1.e). A resampled stream is neither.
• Comparing interpolated with archived values. Two curves that disagree are often two retrieval modes, not two data sets. Specify the mode in every report that uses historian data.
• Losing the quality flag. A substituted or bad-quality value looks like any other number once the status code is gone, and it then enters a mean or a capability index.
• Letting a maintenance dashboard drift into investigations. Convenience wins unless the deviation procedure says which sources an investigation may cite.
• Signing the cloud contract before the data flow exists. The contract then describes the provider’s standard service, not the record you need to keep.

None of this argues against cloud analytics. Querying data where it lives, and sending only what a model needs, is often the cleaner design for integrity as well as cost.

Before the first tag leaves the site, write down which copy is the record. Everything else in the project gets easier once that sentence exists.

References

• M. Buntain, “Bridging On-Premise and Cloud Data”, InTech (ISA), June 2022, IIoT & Digital Transformation: ISA InTech archives.
• MHRA, GXP Data Integrity Guidance and Definitions, Revision 1, March 2018: gov.uk.
• PIC/S, PI 041-1, Good Practices for Data Management and Integrity in Regulated GMP/GDP Environments, 1 July 2021: picscheme.org.
• European Commission, EudraLex Volume 4, Annex 11: Computerised Systems, 2011: health.ec.europa.eu.
• European Commission, Stakeholders’ consultation on Chapter 4, Annex 11 and new Annex 22, 7 July to 7 October 2025: health.ec.europa.eu.
• FDA, Data Integrity and Compliance With Drug CGMP: Questions and Answers, December 2018: fda.gov.
• AVEVA, B. Bregenzer, Exception, Compression, and their Impacts on PI System Performance, 2023: cdn.osisoft.com.