No verdict, no pass: designing the reject loop around an AI vision inspection

The commissioning test had gone well all week. The camera saw every seeded defect and the pusher threw each one into the bin. On the last afternoon, someone from the network team rebooted a switch for an unrelated change, and for a minute the inference service could not be reached. Nobody noticed, because nothing was rejected. The output carrying the verdict to the programmable logic controller (PLC) had stayed in its last state, which meant pass.

We have met variants of that scene on more than one project, and the model was never the problem. An artificial intelligence (AI) vision inspection that feeds a reject gate is a control loop with a deadline. What lets a bad part through is how the loop behaves when the verdict is late, belongs to another part, or never comes. The rule is simple to state and harder to build: each part earns its pass, and everything else rejects.

The time budget between camera and gate

The camera sits at one point on the conveyor, the reject actuator at another, and the belt speed turns the distance into time. With round numbers, for illustration only, a camera 1.2 m upstream of the pusher and a belt at 0.4 m/s leave 3 seconds between image and gate. That is the whole budget.

The image leaves the camera, crosses a network, waits in a queue, goes through the model, and the verdict comes back. Then the PLC needs a scan or two and the pusher needs time to stroke. So the deadline is not the moment the part reaches the gate. It is that moment minus the actuator’s response, minus the scans, minus a margin you can defend in a review.

A 2022 case study in AUTOMATION magazine describes this arrangement on a board production line: a cloud-hosted vision service, an on-site message broker, and a pass/fail signal wired to the PLCs that drive rejection, with about five seconds from capture to reject. Its concern was latency. On a site under good manufacturing practice (GMP), the harder question is what happens when the latency is infinite.

A pass is earned, a reject is the default

Between camera and gate, the PLC already tracks parts in a shift register or a first-in, first-out queue, one slot per part. The design decision is what a slot holds when a part enters it. In a fail-safe loop it holds reject, and only a verdict that passes the four checks below may change it to pass.

• It names this part. The identifier in the verdict matches the slot at the decision point.
• It arrived in time. Before the deadline computed for that part; a late pass is logged and ignored.
• It comes from the approved model. Its version identifier is on the list held under change control.
• It is sure enough. Confidence is above the validated threshold; undecided never passes.

This is the thinking behind a guard door: the safe state must not need a signal to be reached. A static output that means pass while energised fails it, because a frozen relay, a crashed edge device and a lost connection all look like a steady level. A per-part message, or a toggling bit with a part number, can be checked for freshness. The same reasoning runs through the article on wiring the uninterruptible power supply (UPS) signal to the PLC.

The texts point the same way. The draft EU GMP Annex 22, published for consultation on 7 July 2025, asks in §9.2 for a confidence threshold, with very low confidence flagged as undecided. 21 CFR 211.68(b) requires computer input and output to be checked for accuracy, and EU GMP Annex 11 §5 asks for built-in checks wherever systems exchange data.

Keeping the image and the part together

The quietest failure is a verdict applied to the wrong part. An operator lifts a part off the belt between camera and gate, a sensor double-counts, or a verdict delayed by a network hiccup arrives one slot late. Every verdict after that is right for its image and wrong for its part.

The cure is to let the PLC own the identity. At capture, the PLC gives the part a sequence number and sends it with the trigger; the image is stored under that number; the verdict returns it unchanged. If the number at the decision point does not match, the part is rejected and a tracking alarm goes to the operator. Sequence numbers beat timestamps here: they do not need two clocks to agree.

Messaging protocols help without doing the PLC’s job. MQTT (the publish-subscribe protocol standardised by OASIS) offers, in version 5.0, a Correlation Data property returned unchanged with a response, and a Message Expiry Interval. The expiry counts whole seconds and only stops the broker delivering an old message. It cannot tell the PLC that a message which did arrive is stale for this part.

The number also makes the record. Image, part number, verdict, confidence and model version, stored together, are what an investigator needs when a complaint comes back; the article on metadata explains why a value without its context is not a record.

When the loop fails: what the PLC must do

This table is the core of the functional specification. Each line is also a test case for the operational qualification (OQ), and most can be provoked on a running line with known-good parts.

A run of timeouts rejects good product, and someone will want a bypass. The answer is a limit: after a set number of consecutive unverdicted parts, the line stops and the operator decides under a procedure. The last line is not about AI at all: a loop that cannot confirm the part left the belt has only proved that it sent an order.

Edge or cloud: where the model should run

Suppliers offer one of three arrangements. The choice is less about computing power than about who controls the timing, the network path and the model version.

Cloud inference puts a wide-area network inside a loop with a deadline: good latency on average, unbounded in the worst case, and the worst case is what the reject logic must handle. It also needs a permanent path out of the camera network, to be designed as a conduit between zones, as in the article on zones and conduits. Outbound only, through a broker on site, it can be made acceptable.

Edge inference keeps timing local and the line independent of the internet, at the price of an industrial computer to patch and replace, and model files to deploy under change control. The arrangement that survives a GMP review is often the hybrid: a frozen model decides at the edge, and the cloud receives images for monitoring and training the next version.

One trap in the cloud offer has nothing to do with latency. A service whose supplier keeps improving the model is, from the plant’s side, a dynamic model. The draft Annex 22 covers static, deterministic models in §1, and says dynamic models that keep learning in use should not be used in critical GMP applications. The contract must say the version in production changes only when the site says so.

Challenge parts at the start of every batch

EU GMP Annex 1, in operation since 25 August 2023, asks in 8.32 that automated inspection be validated to detect known defects, be equal to, or better than, manual inspection, and be “challenged using representative defects prior to start up and at regular intervals throughout the batch.” Section 8.33 asks for results to be recorded and trended.

For an AI inspection, the challenge set needs more than defects. On the lines we review, a useful set has three parts:

1. Known defects, one or more per defect class in the library, which must all be rejected under their own reason.
2. Known-good parts, including the borderline cosmetic ones operators argue about, which must pass.
3. A silent challenge: a few known-good parts run while the verdict path is deliberately blocked, which must all be rejected as unverdicted.

The third is the test nobody writes, and the one that would have caught the opening scene. It belongs at start-up and after any restart of the inference device or its network path, in the spirit of the article on degraded-mode tests in an OQ.

Trending has a consequence. If timeouts, unusable images and undecided outcomes count as product defects, the defect trend measures the network as much as the product. Give them their own reject categories and treat a rise in timeouts as a system event.

Freezing the model: change control that sees what matters

Section 10 of the draft Annex 22 asks for change control over the model, its system and the process (§10.1, including the physical objects used as input), configuration control that detects unauthorised change (§10.2), and monitoring of performance and input drift (§10.3, §10.4), with a change of lighting given as an example.

On a line, that list is longer than the model file. A new lens, a replaced light bar, another vial supplier, a new format, a camera firmware update: each changes what the model sees. The practical tool is the version identifier carried in every verdict, ideally a hash of the deployed model file, checked against the approved list. An unauthorised change then shows within one part.

Retraining produces a new model, tested again on data it has never seen: §6.1 keeps test data out of development, training and validation, and §4.3 sets acceptance criteria at least as high as the process replaced. So measure the manual inspection first, per defect class, as the article on AI use cases that survive a quality review argues from the quality department’s side.

What the texts ask, and how firm each one is

Annex 22 is a consultation draft, with comments open until 7 October 2025: its wording may change and it binds no one in that form. It is nonetheless the clearest signal of what European inspectors will look for, drafted by the European Medicines Agency’s inspectors with the Pharmaceutical Inspection Co-operation Scheme (PIC/S). Chapter <1790> of the United States Pharmacopeia (USP) is informational, like every USP chapter numbered from 1000 to 1999.

Suppliers often raise the AI Act. Under Article 6, a system is high-risk as a safety component of an Annex I product needing third-party conformity assessment, or in an Annex III use. A filling-line quality inspection is usually neither, and Article 6(1) applies from 2 August 2027. For the lifecycle work, ISPE published its GAMP Guide: Artificial Intelligence in July 2025, to be used alongside GAMP 5 second edition.

Where to start

1. Pull the plug once. At the next planned stop, block the verdict path and run ten known-good parts. Success is ten rejects, each recorded as unverdicted. If one passes, stop there: the rest of this article is now urgent.
2. Write the timing budget on one page. Distance, belt speed, actuator response, PLC scan, measured inference time at its worst, and the margin. Sign it with the automation engineer and the supplier.
3. Put the model version in the verdict. Then in the batch record. It costs a field in a message and closes the most common gap in change control for these systems.

The model will be wrong now and then, and the validation measures how often. The loop must never be wrong about what to do when the model is silent.

References

• Josh Eastburn, “Case Study: Connecting Cloud-based Quality Control to the Edge”, AUTOMATION 2022 Vol 2 (ISA), April 2022, online 6 May 2022: automation.com
• European Commission, EudraLex Volume 4, draft Annex 22 Artificial Intelligence, stakeholders’ consultation opened 7 July 2025, comments until 7 October 2025: health.ec.europa.eu
• EudraLex Volume 4, Annex 1 Manufacture of Sterile Medicinal Products (2022), sections 8.30 to 8.33: health.ec.europa.eu
• EudraLex Volume 4, Annex 11 Computerised Systems (2011), §5 and §10: health.ec.europa.eu
• US 21 CFR 211.68, Automatic, mechanical, and electronic equipment: ecfr.gov
• USP general chapter <1790> Visual Inspection of Injections: doi.usp.org
• OASIS, MQTT Version 5.0, OASIS Standard of 7 March 2019, §3.3.2.3.3 and §3.3.2.3.6: docs.oasis-open.org
• Regulation (EU) 2024/1689 (AI Act), Articles 6 and 113, Annexes I and III: eur-lex.europa.eu
• ISPE, GAMP Guide: Artificial Intelligence, July 2025: ispe.org