FAT and SAT: what the supplier’s tests can prove for you, and what stays yours

Two weeks before the factory acceptance test, the machine builder sends the agenda: a long checklist written by its own engineers, its own acceptance criteria, and a box at the bottom for the customer’s signature. Signed as it stands, that document proves one thing. The supplier is satisfied with its own machine.

Factory and site acceptance tests (FAT, SAT) are still worth the trip. They find faults where faults are cheapest to fix, and EU good manufacturing practice (GMP) lets some of their results replace tests in the installation and operational qualification (IQ, OQ). The evidence only counts, though, if you shaped it before the test: the protocol, the criteria, the records.

Two acceptance tests, one conflict of interest

The FAT takes place at the supplier’s works, during and at the end of the build. The SAT takes place on your site after delivery and installation, and is usually still executed by the supplier. Both are commercial acceptance first. They decide whether what was ordered has been delivered, and they often release a payment.

Left to itself, the supplier writes the protocol, runs the tests and concludes against its own specification. It is judge and party, as one qualification trainer used to put it. For a commercial acceptance that is perfectly normal. It becomes a problem when the same paper later turns up in the qualification file as evidence.

EU GMP Annex 15 (2015) treats both tests as optional. Equipment, “especially if incorporating novel or complex technology”, may be evaluated at the vendor before delivery (§3.4). Before installation, compliance with the user requirements specification (URS) or the functional specification should be confirmed at the vendor site, where applicable (§3.5). And a FAT “may be supplemented” by a SAT once the equipment reaches the manufacturing site (§3.7). Not every purchase deserves a FAT, then. For standard catalogue equipment there is often none, and the IQ and OQ carry the whole load.

The reuse condition, read slowly

One sentence decides whether a FAT test saves you work. Annex 15 §3.6 allows documentation review and some tests to be performed at the FAT “without the need to repeat on site at IQ/OQ”, where appropriate and justified, and “if it can be shown that the functionality is not affected by the transport and installation”. Four conditions are hidden in there.

• Appropriate and justified. The justification is written test by test, in the qualification plan or in the IQ/OQ protocol. “It was covered at the FAT” is not a justification.
• Some tests. Reuse is selective: a list of named tests, never a blanket statement about a whole stage.
• Functionality not affected by transport and installation. Software logic, recipes, alarm texts, user roles and calculations usually survive the lorry, provided the version installed is the version tested and you check that it is. Alignment, sensors, fixtures, pneumatics, weighing, anything fed by site utilities and every interface to another machine do not; most of them did not even exist at the works.
• Controls while the data were produced. The General section of the annex accepts data from outside the manufacturer’s own programmes when the approach is justified and there is “adequate assurance that controls were in place” during their acquisition. That is the real price of reuse: an approved protocol, calibrated instruments, raw data, traceable records.

The same reasoning covers commissioning records produced under good engineering practice, the approach of the ISPE commissioning and qualification guide (second edition, 2019). They can support the IQ and OQ. Who produced a record matters less than the controls it was produced under. In practice the four conditions become four questions, asked one test at a time.

Shape the FAT before it happens

What you ask for beforehand decides what the visit is worth. On a machine whose tests you intend to reuse, we ask for the following.

• A protocol written against your URS. Each test cites the requirement it verifies, with your acceptance criterion: a value and a tolerance, not “works correctly”. See a URS that drives the qualification.
• Approval before execution. Annex 15 asks people at the manufacturing site to “confirm suitability and compliance with internal procedures before approval” of a third party’s protocols (§2.6), and lets you add tests of your own.
• Calibrated test instruments, certificates in the file, with a calibrated range that covers what is being measured.
• Raw data rather than ticks: printouts, screenshots, trend files, photographs, and the failed attempts as well as the good ones.
• Deviations recorded on the spot, numbered, with the fix and the retest. The overnight repair nobody wrote down is the one that comes back on your line six months later.
• A configuration baseline at the end: software and firmware versions, checksums, parameter and recipe lists, a program backup. Without it nobody can show on site that the configuration is still the one that was tested.
• Your own people in the room: process, automation, and quality assurance (QA) when results will be reused. They sign as witnesses unless they are also the approvers, and those are two different statements.

A FAT is also the cheapest place to test what you never want to provoke on a running line: alarms, interlocks, emergency stops, reject mechanisms, behaviour at the edges of the range. Test them at the works, then repeat on site the ones installation can affect.

The SAT pays the transport bill

Whatever you agreed to reuse, the SAT is where the §3.6 condition is settled. It covers what the journey and the installation may have changed, what did not exist at the works (utilities, network, neighbouring machines, the systems that collect the data), and the points left open at the FAT.

The IQ itself starts at the loading dock: delivery documents against the order, the state of the crates, the documentation pack, then positioning and connections. On many projects the SAT and the IQ happen in the same week. Executing IQ checks during the SAT is fine, provided the IQ protocol was approved first. The order of the approvals matters more than the order of the visits.

What a supplier qualification package is worth

Many suppliers sell one: pre-written IQ and OQ protocols, execution by their own engineers, sometimes a performance qualification (PQ) service and a “maintenance qualification”. The work can be good. The status of the paper depends on what you did with it. And “MQ” is not a stage in Annex 15; maintenance, calibration and periodic review keep the qualified state, they do not stand in for a stage.

The software inside the machine has its own paragraph in the rules. Annex 11 asks for formal agreements with clear responsibilities when a third party provides, installs, configures, validates or maintains a computerised system (§3.1), and for the documentation supplied with off-the-shelf products to be reviewed by the regulated user against its requirements (§3.3). GAMP 5 second edition goes further and encourages leveraging a competent supplier’s testing, which makes assessing that supplier properly, once, worth more than repeating its tests.

Clauses that belong in the order, not in an e-mail

None of this can be obtained once the contract is signed. These are the clauses we ask buyers to add:

• FAT and SAT protocols submitted for approval a set number of weeks before execution, with tests referenced to the URS;
• executed protocols and raw data handed over, failed runs included, in a readable and durable form;
• in writing: calibrated test instruments with certificates, agreed deviation handling, and a configuration baseline delivered at the FAT and at the SAT;
• a documentation list with formats and language, as-built drawings, a spare-parts list with references;
• evidence that the supplier staff who execute protocols are trained to do so, since qualification is for “suitably trained personnel who follow approved procedures” (Annex 15 §1.2);
• support after the SAT through your controlled remote-access route, not a tunnel opened for the project;
• every change after the FAT notified to you and routed through your change control.

A supplier who refuses these clauses has given you an answer. Plan and price the tests you will have to repeat on site before the order, not after the delivery.

What stays yours

A supplier can execute almost everything. Four decisions stay with the site, and an inspector will ask about each of them: the requirements and the design review; the approval of every protocol before execution; the justification for each reused test; the release decision at the end. Then there is the PQ, run with your product or a qualified substitute (§3.14), your operators and your procedures. The supplier’s sample trials do not answer that question. What each stage has to prove is set out in DQ, IQ, OQ, PQ.

The mistakes we see most often

• Signing the supplier’s report as “approved” when your engineer only watched. Witnessing and approving are different statements, and only one of them opens a stage.
• A FAT on a version that never arrives. The software gets improved after the tests and the machine ships with something else. Compare versions and checksums at the SAT.
• Reuse with no paper. The tests are not repeated and no justification exists, so the file has a hole where the IQ or OQ test should be.
• A SAT that repeats the FAT checklist. The same tests run a second time in a different building, while the site utilities and the interfaces go unchecked.
• Acceptance criteria copied from the datasheet. Some training material even advises being no stricter than the supplier’s specification. The safer rule: criteria come from your process, and the design review should already have shown that the supplier’s specification can meet them. When it cannot, the machine is the problem, not the criterion.

A supplier’s test can become your evidence. It does so on your protocol, with your criteria and your signature, or not at all.

References

• EudraLex Volume 4, EU GMP Annex 15, Qualification and Validation (2015, in operation since 1 October 2015): General section, §1.2, §2.6, §3.4 to §3.7, §3.14.
• EudraLex Volume 4, EU GMP Annex 11, Computerised Systems (2011): §3, suppliers and service providers.
• ISPE, *Baseline Guide Volume 5: Commissioning and Qualification*, second edition (June 2019).