Sample Size Required to Guarantee a Vision System’s False Negative Rate

Evolution of Statistics

Sample Size Required to Guarantee a Vision System’s False Negative Rate

  • Rob Couture

Common performance criteria at a Factory Acceptance Test is a False Positive of better than 0.1% or 1/1000.  But, when the question comes up about the False Negative Rate, the room gets really quiet while everyone has a dark flashback to Stats 101.

Let’s assume that the people (or machine) assembling the part produces 3% rejects (true negatives).  How many samples need to be observed to guarantee the system has a 1/10000 chance of passing a Reject product (False Negative)?  You need to use the following formula:

    n = ln(1 - desired false negative rate) / ln(1 - true negative rate)

    •  n is the needed sample size.
    •  ln is the natural logarithm function. 
    •  desired false negative rate is the probability of a false negative that you want to guarantee, in this case 1/10000 or 0.0001. 
    •  true negative rate is the probability of a true negative, in this case 3% or 0.03. 

Using the formula, we get: 

    n = ln(1 - 0.0001) / ln(1 - 0.03)
    n = 13049.15

Rounding up to the nearest whole number, you would need a sample size of at least 13050 to guarantee a false negative rate of 1/10000, assuming a true negative rate of 3%.  With an image archiving system, all these samples can be reviewed offline at your desk. 

Ensuring a vision system’s false negative rate can be a time-consuming task.  However, using the formula provided and understanding the assumptions behind it, you can help to show the client you are not pulling the number out of thin air.  Nevertheless, obtaining a sufficient sample size is critical to ensure the performance of the vision system meets the necessary standards.

NOTE – this calculation assumes a simple random sample and that the distribution of the characteristic of interest in the population is approximately normal.  If the distribution is not normal, or if other factors may affect the variability or bias of the sample, different sample size calculations may be needed. 

Recommended Posts

How Can We Help?

Industrial Machine Vision systems help to ensure the quality of your product and can be designed so that they are maintenance friendly.

From retrofits to turnkey solutions, extracting meaningful data or making an existing system more reliable; we are here to help you meet The Standard.

Fill Out a Project Brief