How to properly use Quality Sampling for Process Validation?
When it comes to sampling of medical devices, we have seen numerous times that the production manager or quality engineer truly wants to do the right thing, but the standards (that are used to write internal operating procedures and work instructions) are not well written, interpreted or simply understood.
In this article, we’d like to explain briefly, how to use AQL Sampling plans in a REAL production line.
There are two main concepts here:
AQL (Acceptable Quality Limit): any batch below this quality level should be rejected.
LTPD (Lot Tolerance Percent Defective): any batch with less defects than this should be accepted.
These two sampling methods are used for accepting/rejecting a lot under inspection, both internally in production, and also externally – agreed upon by the suppliers of parts and the purchasing company. Hence, it is critical to understand how the system works, and at least be at a ‘working knowledge’ level.
AQL method answers:
“How many parts in a batch can fail inspection before we reject the whole batch?”
LTPD method answers:
“How many parts in a batch must pass inspection before we accept the whole batch?”
A corollary is that using the AQL method “if we reject a batch, inevitably there will be GOOD parts that are marked QC REJECT”. This is a risk for the producer/manufacturer (named a risk: risk of rejecting a good part).
On the other hand, LTPD means “if we accept a batch, unfortunately there will be BAD parts that are marked QC PASS”. This is a risk for the consumer/user (named b risk: risk of accepting a bad part)
Is it starting to make sense? Let’s talk about two examples:
When you use LTPD=2% you’re asking: how many items I should test to be “95% confident” that a lot with >2% defective product will be rejected?
When you are willing to accept a batch with 2% nonconforming parts (AQL=2.0) you’d like to know: how many items I should test to be “95% confident” that a lot with >98% good product will be acceptable?
How the heck do I know which method to choose?
The answer is “it does not matter” as long as you know how to calculate LTPD based on AQL or vice versa. See, AQL and LTPD are both calculated using a “confidence level” referred to as c and “number of samples” shown as n. The formula to do that can be found below:
AQL and LTPD are two sides of the same coin; AQL is lower limit of how many good parts are acceptable, LTPD is the highest number of bad parts that is still acceptable.
We will focus on the AQL method since this is used by majority of medical device manufacturers.
How do I choose the AQL number?
This is where it gets more interesting…! First, you’ll need to conduct a risk assessment or consult your FMEA. Choose lower AQL for critical defects (e.g. loss of sterility, not meeting regulatory requirements), medium for major defects (e.g. silicone discoloration), and higher AQL for minor defects (e.g. cosmetic defects).
Critical defects are ones that may harm the user such as loss of sterility. Major defects are ones that user usually doesn’t accept the device. Minor defects are slight departures from the specifications, but the product can still be safely used and accepted by the user. Note: a cosmetic defect for a device may be acceptable for a device (e.g. for surgical forceps) and unacceptable for another (e.g. for hip Implant stem, or an acetabular cup).
Here’s how our auditors normally expect manufacturers to write the sampling procedures and conduct sampling (step-by-step):
- Purchase ISO 2859
- Find the below Table 1:
3.Select General (GIL) or Special (SIL) based on type of devices and acceptable risk
* SILs are to be selected only when inspection is unreasonably expensive (e.g. destructive testing)
* SILs require LESS samples than GIL –> More risk of non-conforming product when choosing SIL
* By default, GIL II must be selected, otherwise I’d expect a justification to be recorded.
4. Based on your Lot Size find the Inspection Letter (A, B, C, D, …)
5. Find the below Table 2:
6 .What is your AQL? This must be determined based on the risk to the patient/user/regulatory requirements – as discussed previously. Reminder: the AQL numbers are in fact PERCENTAGES: as an example, AQL=1.5 means we (manufacturer and user) are willing to accept a batch 1.5% of the which is nonconforming parts.
7.Based on your inspection alphabet and AQL find out “how many rejects is still acceptable?”
NOTE: If sample size ends up being larger than Lot size you must inspect 100% of the lot.
8. If the number of defective parts is less than “RE” (but not equal to it) we accept the lot. Otherwise, reject the lot.
Notes on selecting samples:
- Samples must be RANDOMLY selected
- Samples can be selected either DURING production or AFTER, based on the procedure.
What are “Reduced”, Normal, and “Tightened” sampling?
Based on the whether you undertake the reduced, normal, or tightened method the answer to “how many defective parts is still acceptable” changes. Regardless of the method, same sample size applies! BUT the criteria for lot acceptance is more demanding (less number of rejects is acceptable) or less demanding (more number of rejects is acceptable).
Should we change method depending on how the process goes?
Yes. As per the below:
During Normal Sampling: If 2 out of 5 lots were rejected, you MUST Switch to Tightened Sampling.
During Tightened Sampling: If 5 continuous lots were accepted, you CAN Switch to Normal Sampling.
During Reduced Sampling: If any lot is rejected switch back to normal sampling.
NOTE: If a lot is to be re-inspected after all parts tested (and rejects replaced with new parts): the batch must be re-inspected with “Tightened” inspection, not “Normal”.
What are “Single”, “Double”, and “Multiple” sampling?
Single sampling is normally used. It’s our observation that double or multiple are used when rejecting a batch is very costly so we really try hard to see if the batch is still acceptable. Here’s how each work:
- In “Single” sampling, if number of defective parts is more than “AC” we reject the lot, period.
- In double sampling, if the number of defective parts is more than “AC” but still less than “RE” we go to the second line called “SECOND” and inspect more from remainder of the lot (as stated in SECOND line). If the cumulative number of defective parts is less than the “RE” in SECOND line we accept, otherwise reject batch. Multiple sampling follows the same method as “double” but with more iterations.
Important NOTE: your company procedures must allow for double or multiple sampling, otherwise you cannot take those approaches. You CANNOT switch to, say double, during the inspection.
That’s really all there is to it. Now, your quality team must ensure that your staff have ‘at least’ a working knowledge of sampling methods and follow their procedures line by line. Bear in mind, your auditors CAN and WILL have a look at how you sample, there’s no way around it.
In a future article, we will elaborate on various methods of selecting samples (e.g. random, structured, judgement-based, etc).
Hope you’ve enjoyed the article, please share if you did.