AI Insight
Researchers developed a method to detect when automated liver segmentation from CT scans fails without requiring expert labeling of failures. By fitting a statistical distribution to quality scores from 400 validation scans, they created a system that categorizes new segmentations into risk groups based on how unusual the scores appear. Testing on 500 scans showed the method could identify all actual failures with 79% specificity when using medium and high-risk categories, or catch 78% of failures with 96% specificity using only the high-risk category.
Why it matters
This approach could make AI-based medical imaging tools safer for clinical use by automatically flagging cases that need human review, without requiring physicians to manually label large numbers of failure cases during system calibration. The label-free nature significantly reduces the annotation burden for deploying quality control systems in medical AI applications.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Reliable clinical deployment of automated liver segmentation requires mechanisms for detecting failures in rare and previously unseen scenarios. Achieving this goal requires an appropriately calibrated threshold that converts an out-of-distribution (OOD) score into a failure prediction. However, threshold calibration typically relies on expert-labeled failures, creating a substantial annotation burden when failures are rare. Building upon our prior work, which uses Pairwise Surface DSC scores as indicators of segmentation quality, we propose a label-free framework for calibrating OOD score thresholds. First, we fitted a log-t distribution to Pairwise Surface DSC scores from a validation set of 400 internal scans to approximate an in-distribution score distribution. New segmentations were assigned significance scores based on their extremity under this fitted distribution and categorized into Low, Medium, and High Risk review groups using statistically principled cutoffs of 0.25 and 0.05. The fitted log-t distribution provided a strong fit to the observed scores and remained robust to moderate contamination by OOD cases. On an independent test set of 500 internal and external scans, the combined Medium and High Risk categories achieved 100% sensitivity and 79% specificity, whereas the High Risk category alone achieved 78% sensitivity and 96% specificity. These results indicate that clinically meaningful failure detection can be derived from unlabeled data. Our code is available at https://github.com/marshalln7/Label_Free_OOD_Threshold_Selection.
Source: Label-Free Threshold Selection for Out-of-Distribution Detection in Liver CT Segmentation