Computer systems today are trained using a method called maximum likelihood estimation (MLE).
This has the consequence that the model becomes overconfident based on the presence of just a few key features in the image, not the collection of all the features.
To see an explanation of this, check out our blog post on the need for uncertainty in medical imaging
. It shows how a simple model that is trained to classify handwritten digits, mistakes the letter X
for an 8
simply because there are two crossing lines in the middle of an 8
that match the middle of an X.
Today's approach to getting around this issue is training the models with images that are distorted in various ways, hoping that exposure to this will make the model more robust. This works to some extent, but it relies on guessing the correct distortions the AI will be exposed to, it does not solve the fundamental problem.