AI Insight
This study presents a computationally efficient method for estimating uncertainty in automated seafloor image classification using "last-layer committee machines," which achieves over 95% reduction in network parameters compared to traditional approaches. The researchers tested their framework on the BenthicNet dataset and found it performed nearly identically to more expensive methods like Bayesian neural networks and Monte Carlo dropout while being far more efficient. The approach enables identification of ambiguous, mislabeled, or challenging images that require human expert review.
Why it matters
This method provides a practical tool for marine scientists to improve the reliability of automated seafloor monitoring systems, which are essential for tracking rapidly changing ocean ecosystems. By efficiently flagging uncertain predictions for human review, it can enhance benthic habitat mapping and marine spatial planning while reducing computational costs.
arXiv:2504.16952v2 Announce Type: replace
Abstract: Automating the annotation of benthic imagery (i.e., images of the seafloor and its associated organisms, habitats, and geological features) is critical for monitoring rapidly changing ocean ecosystems. Deep learning approaches have succeeded in this purpose; however, consistent annotation remains challenging due to ambiguous seafloor images, potential inter-user annotation disagreements, and out-of-distribution samples. Marine scientists implementing deep learning models often obtain predictions based on one-hot representations trained using a cross-entropy loss objective with softmax normalization, resulting with a single set of model parameters. While efficient, this approach may lead to overconfident predictions for context-challenging datasets, raising reliability concerns that present risks for downstream tasks such as benthic habitat mapping and marine spatial planning. In this study, we investigated classification uncertainty as a tool to improve the labeling of benthic habitat imagery. We developed a framework for two challenging sub-datasets of the recently publicly available BenthicNet dataset using Bayesian neural networks, Monte Carlo dropout inference sampling, and a proposed single last-layer committee machine. This approach resulted with a > 95% reduction of network parameters to obtain per-sample uncertainties while obtaining near-identical performance compared to computationally more expensive strategies such as Bayesian neural networks, Monte Carlo dropout, and deep ensembles. The method proposed in this research provides a strategy for obtaining prioritized lists of uncertain samples for human-in-the-loop interventions to identify ambiguous, mislabeled, out-of-distribution, and/or difficult images for enhancing existing annotation tools for benthic mapping and other applications.
Source: Last-layer committee machines for uncertainty estimations of benthic imagery