What criteria should be used to select the number of images used in image processing for detection the drawbacks of any surfaces?

What are the criteria that must be considered to select the number of images in image processing techniques to make sure the proposed method is working well, and to trust the results of that model used based on image processing in detecting any drawbacks of the surfaces?

Answer 1

[Image Analyst]

It depends on your situation. So I'm presuming you have some ground truth that tells you whether the algorithm gave the correct answer or not. So you have four situations:

1. True positive
2. True negative
3. False positive
4. False negative

So which of the 4 is most important depends on the context. Let's say it's a cancer diagnosis. Is the risk of a false negative (you have cancer but we didn't detect it) worse than a false positive (we told you that you have cancer but you really don't)?

You might want to look up ROC Curve : https://en.wikipedia.org/wiki/Receiver_operating_characteristic

Maybe you look at multiple images before making a decision so each point on the ROC curve could be for a different number of images used in the final decision.

but where on the curve you say is the "optimal" number of images to use is a judgment call. Maybe you just simply want to say that if the true positive ("correct") rate is more than 90% you're happy with that. Or maybe if the risk of being wrong is really high you want the true positive rate to be >99%.

[yasmin ismail]

@Image Analyst I read about ROC but I didnt get what i want, let me explian

If I have model to detect crcak on concrete surface , this model using type of filters then find threshold, then using morophological operation and so on...

I measured the performance of this model based on the past four situations TP, TN,FP,and FN and calculated the precision, recall, accuracy. How many image does I need to calculate them? 100, 200, 300 or else? how can I say any number f the them is enough and based on what?

[Image Analyst]

Plot your true positive rate as a function of the number of images used to create the model. It may rise up and start to flatten off after a certain number of images. Once it rises above your minimum required accuracy, you can say that that number of images is sufficient. It may flatten off below your minimum required value. In that case youi might try to improve your algorithm so that it is correct more often. Usually the more images to train with the better, but because people just have a certain number of images total in a scenario, a common tactic is to take 80% to train the model with, and 20% to test the trained model. If the testing shows that the model is good, then go ahead and train on all the images and then go ahead and start using the model on newly acquired images that you don't know the value/class of. You can assume it's good enough on those because you now trust the model you built.

[yasmin ismail]

@Image Analyst How To put TPR as a function of image number? TPR= TP/TP+FN So what I understood assume that I want to TPR reach 90% for example, so I have to assume the total of (TP+FN)let’s say = 300 then I can find TP which will be equalled 270 then I can get the number of FN Is it correct?

[Image Analyst]

FN is the number of true positives that your algorithm mistakenly called negative. But what you can do is simply count P, the number of positive cases. If you have numImages, and you know the actual, true results for them all, then you know the number that are Positive and the number that are negative. So numImages = P + N. So just get the true positive rate by the fraction of images that were correctly identified. TPR = TP/P.

[yasmin ismail]

@Image Analyst I confused little bit. If I want to use TPR as a function of the number of images used to create the model. My aim is to find the enough number of images to my model. Let we assume I want TPR=90%, so I have to try 100 image then find how many TP I got from all positive cases, then increased it till reached 90% or what?