Challenge testing: proving that sensors, code readers and metal detectors actually reject

A presence sensor on a cartoner gets knocked out of line while someone clears a jam. The line keeps running, the stack light stays green, and not one carton is rejected for a missing leaflet. Which is exactly what a perfect batch looks like too.

Every detection function on a packaging line has this problem. From the outside, a detector that finds nothing and a detector that has died look the same. A false reject makes noise; a missed defect makes none. The only way to know which one you have is to make the detector fail on purpose, at qualification and again during every batch.

What a challenge test must prove

A challenge test puts a known defect into normal running and watches what the line does with it. The principle is the same on every line we qualify: provoke the defect the sensor must detect, check that it causes the stop or the ejection, and make sure that if the sensor fails, the defective unit can still be found and destroyed. That is three proofs in one test:

• Detection: the sensor sees the defect, at production speed, on the right unit.
• Reaction: the machine does what the specification says: reject, divert or stop.
• Recovery: the test unit ends up back in your hand, counted, and not in a carton on its way to a pharmacy.

Recovery is the one people forget. A test unit is a deliberately defective product on a GMP line, and if the challenge fails, it is travelling with the batch. Mark it so nobody can mistake it for product (a coloured cap, a “TEST” label), and count it out and back in.

These tests belong to the operational qualification (see DQ, IQ, OQ, PQ: what each stage proves) and carry on, in a lighter form, for the life of the line. Write the expected reaction before running the test. Annex 15 asks protocols to define acceptance criteria (§2.4) and a missed criterion to be recorded as a deviation (§2.8). A test line with an empty “expected result” column can neither pass nor fail.

What to challenge on a filling and packaging line

In the 2003 file, the filler’s OQ alone listed about twenty tests on sensors and guards. Strip out the machine tags and you have a checklist for almost any liquid filling line:

• Product and closure: fill level, cap present, overcap present; the faulty bottle goes to the reject lane.
• Flow: minimum load at the infeed, maximum at the outfeed, a jam at the outfeed; the machine stops.
• The reject path: a sensor confirms the rejected bottle has left, and a full reject conveyor stops the machine.

Two design details deserve a test of their own. The first is the “three in a row” rule. On that line, a bottle with the wrong volume, no cap or no overcap went to the reject lane, and three consecutive rejects stopped the machine. One reject is a random defect. A run of them points to a systematic cause, a blocked nozzle, an empty cap hopper, a sensor out of position, and only a person can fix that. Challenge the counter itself: send three defective units in a row and check that the stop comes on the third.

The second is reject confirmation. Detection is worth nothing if the ejector misses. The sensors on the reject path (reject confirmed, reject lane full) are the detectors of the detector, and they are the ones most often missing from the protocol.

Run the challenges at the top qualified speed, not with the line crawling. Rejects fail on timing: the machine tracks the defective unit from the sensor to the reject gate, and at full speed a few milliseconds decide which bottle gets kicked out. Annex 15 asks for OQ tests at the upper and lower operating limits or under worst-case conditions (§3.11), and for primary packing equipment to be qualified at the minimum and maximum of ranges such as machine speed (§7.2). What happens to a flagged unit when the power fails is a degraded-mode question, covered in When the power and the air go away.

Code readers: every reader, every format

Code readers on labels, leaflets and cartons stop a foreign component from getting into the batch: a leaflet left over from the previous product, a carton for another strength. The 2003 line had three readers, and the protocol challenged each one with six cases:

• a false code, a defective code, and a component with no code at all: the machine stops and the unit is rejected;
• a genuine code: the unit passes;
• a false code stuck onto a genuine component, simulating a mix-up: stop and reject;
• a genuine code stuck on, as a control: the unit passes.

The whole set was repeated on every format, and that is not bureaucracy. On a format change the reader, the component and the code all move. Guides are reset, the leaflet folds differently, the code sits somewhere else on the carton. A reader that works on the smallest carton can miss on the largest.

A reader gives a verdict, read or not read. It does not grade print quality, which needs a verifier grading to ISO/IEC 15415 and is a separate test. The same approach works for a camera checking the printed batch number and expiry date: prepare labels printed wrong on purpose, off the line, and mark them as test material.

Metal detectors: known test pieces, real containers

The 2003 line had one metal detector, on the conveyor feeding the cartoner, to catch a washer or a screw shed by the filling nozzles. Its challenge had three parts, each run twenty times: a test piece, a staple in an empty bottle, a staple in a full bottle. Every pass had to stop the cartoner.

Testing inside a full bottle was the right instinct, because liquid changes what the detector sees and an empty container proves less. Today we would add three things:

• Test pieces of known metal and size, chosen for what the line can actually shed, rather than a staple of unknown size and alloy. The US FDA’s seafood hazards guidance is a food-sector text, but a clear model here: it bases sensitivity standards on whether the hazard is ferrous, non-ferrous or stainless steel.
• Reject and confirm, rather than only stopping the next machine: the unit goes into a locked bin, and a missed reject stops the line.
• A routine frequency with a hold rule. The same guidance challenges detectors at the start of production, every four hours, when processing factors change and at the end, and holds all product made since the detector was last shown to work.

Twenty out of twenty is convincing, but it does not prove perfection. At 95 % confidence, twenty successes still allow a detector that misses about one time in seven (the “rule of three”: roughly 3 divided by the number of trials). Qualification shows the detection is reliable. Routine checks keep it that way.

The qualification is the first challenge, not the last

Sensors drift. They get knocked, wiped with a wet cloth, moved during a format change. So the challenges carry on in routine production. The 2003 line checked its 17 detection points, from “empty bottle” to “grouping label present on the case”, at start-up, after every operator break and after any stop longer than two hours. The operator ticked each one on the packaging record, signed and wrote the time.

Those moments were well chosen: breaks and long stops are when people clear jams, clean and adjust things. We would add three more: after a format change, after any repair or adjustment of a sensor or its bracket, and after a bypass.

The interval between two checks is also the amount of product you will have to re-inspect when a check fails.

That trade-off is the honest way to set the frequency. At 200 bottles a minute, the top rate of the 2003 line, a four-hour interval puts up to 48,000 units in doubt when a check fails. An end-of-batch sample will not save you: a clean sample can find a critical defect but never prove it is absent, see Performance qualification of a packaging line.

What the regulations actually ask

The texts are short and leave the detail to your protocols. Beyond Annex 15 (§3.11 and §7.2, above):

• EU GMP Chapter 5: “Checks should be made to ensure that any electronic code readers, label counters or similar devices are operating correctly” (§5.57). On-line control during packaging must check at least the correct functioning of line monitors (§5.59), and containers for filling are kept free of contaminants “such as glass fragments and metal particles” (§5.53).
• 21 CFR 211.68(a): automatic and electronic equipment “shall be routinely calibrated, inspected, or checked according to a written program designed to assure proper performance”, with written records of those checks.
• 21 CFR 211.122(g)(2), 211.125(c) and 211.122(h): with cut labels, one permitted control is a 100 % electronic examination for correct labelling, and where it is performed, label reconciliation is waived for cut or roll labels. The challenge test then carries the weight of a control you no longer perform. Printing devices are monitored so that every imprint matches the batch production record.

The table turns this into a challenge matrix. The repeats and frequencies are examples; your sensor list and risk analysis decide.

The mistakes we see most often

• Checking the alarm, not the reject. The panel shows the fault, the box gets ticked, and the test unit stays in the flow.
• Forcing the input instead of passing a defect. Forcing a sensor signal from the panel tests the logic and the reject path, not the detector. Use a real defective unit where it matters, and record which method you used.
• Turning sensitivity down to stop false rejects. A detector that rejects too much is a nuisance; one quietly desensitised is a danger. Detector settings go through change control, see When does a repair become a requalification?
• Leaving a bypass on. Maintenance inhibits a detector for a test and forgets to restore it. The bypass state should show on the panel, be logged, and be limited to named people.
• Ticking the boxes at the end of the shift. A challenge recorded after the fact is not evidence. The sheet is filled in at the time, by the person who did the check.

A detector earns trust one way only: by failing on purpose, in front of someone, at the moments when anything could have disturbed it.

References

• EudraLex Volume 4, EU GMP Chapter 5, Production, §5.53, §5.57, §5.59: health.ec.europa.eu.
• EudraLex Volume 4, EU GMP Annex 15, Qualification and Validation (in operation since 1 October 2015), §2.4, §2.8, §3.11, §7.2: health.ec.europa.eu.
• US 21 CFR Part 211: §211.68(a), §211.122(g) and (h), §211.125(c): ecfr.gov.
• US FDA, Fish and Fishery Products Hazards and Controls Guidance, Chapter 20 Metal Inclusion, food-sector guidance used here as a model: fda.gov.
• J. A. Hanley and A. Lippman-Hand, “If nothing goes wrong, is everything all right? Interpreting zero numerators”, JAMA 249 (1983), 1743–1745 (the rule of three).