Past research has attempted to automate these classification tasks—a usually laborious and time-consuming manual process—with deep learning (DL) methods. Several studies show significant promise, but few have focused on the uncertainty of the methods’ classifications.
“Uncertainty estimation is crucial to avoid misclassifications that could overlook rare and ecologically significant species, says Iver Martinsen, Ph.D. Candidate at UiT The Arctic University of Norway and SFI Visual Intelligence (VI). “It is important to develop DL methods that accurately calculate how uncertain their predictions are.”
In a recently published study in Artificial Intelligence in Geosciences, Martinsen and researchers at UiT, VI, Nofima, and NSE show how deep learning can achieve human-level performance in estimating uncertainty when classifying forams. “Using 260 images of forams and sediment grains, we trained the DL methods to detect and classify these microscopic organisms,” said Martinsen.
Evaluating the performance of such methods remains challenging. To address this, the researchers created a human-derived set of uncertainty estimations based on classification task responses from four senior geoscientists.
“The geoscientists were given the same 260 images and were tasked to classify each of them, as well as state their confidence level. This formed a comparative baseline which allowed us to assess the models’ estimations against those of human experts,” Martinsen explained.
The study also demonstrates how human uncertainty estimations may provide a relevant and valuable baseline for comparison, he adds. Results show that the DL methods’ estimations can match—and at times be better than—expert geoscientists.
“We gain valuable insights on how these methods’ estimations compare to each other and human experts. We believe this research is a leap towards making these automated tools more reliable, trustworthy, and applicable in real-world settings,” Martinsen said.
