Common performance criteria at a Factory Acceptance Test is a False Positive of better than 0.1% or 1/1000. But, when the question comes up about the False Negative Rate, the room gets really quiet while everyone has a dark flashback to Stats 101.
Let’s assume that the people (or machine) assembling the part produces 3% rejects (true negatives). How many samples need to be observed to guarantee the system has a 1/10000 chance of passing a Reject product (False Negative)? You need to use the following formula:
n = ln(1 - desired false negative rate) / ln(1 - true negative rate)
Where:
• n is the needed sample size.
• ln is the natural logarithm function.
• desired false negative rate is the probability of a false negative that you want to guarantee, in this case 1/10000 or 0.0001.
• true negative rate is the probability of a true negative, in this case 3% or 0.03.
Using the formula, we get:
n = ln(1 - 0.0001) / ln(1 - 0.03)
n = 13049.15
Rounding up to the nearest whole number, you would need a sample size of at least 13050 to guarantee a false negative rate of 1/10000, assuming a true negative rate of 3%. With an image archiving system, all these samples can be reviewed offline at your desk.
Ensuring a vision system’s false negative rate can be a time-consuming task. However, using the formula provided and understanding the assumptions behind it, you can help to show the client you are not pulling the number out of thin air. Nevertheless, obtaining a sufficient sample size is critical to ensure the performance of the vision system meets the necessary standards.
NOTE – this calculation assumes a simple random sample and that the distribution of the characteristic of interest in the population is approximately normal. If the distribution is not normal, or if other factors may affect the variability or bias of the sample, different sample size calculations may be needed.
