AI Insight
Researchers tested 880 different language model configurations to assess depression from clinical interviews and found that the choice of model explained 30% of variance in depression scores, while actual patient differences explained only 10.5%. Even among models with similar accuracy (AUC ≥ 0.70), randomly selected pairs disagreed on screening decisions for approximately 40% of participants on average. While calibration with labeled data improved overall accuracy from 60% to 75% and reduced disagreement, about 20% of individuals still received different classifications depending on which model was used.
Why it matters
This study reveals a critical reliability problem in using language models for mental health screening: different AI raters with comparable performance levels make inconsistent decisions about the same individuals. The findings suggest that deploying these tools clinically could lead to arbitrary outcomes where a patient's diagnosis depends more on which specific model is chosen than on their actual symptoms.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Depression has no diagnostic blood test. Language models promise tireless, consistent assessment, but can accurate raters disagree about individuals? We pre-registered 880 language-model raters, crossing 11 open models with prompting and scoring choices, and applied them to 189 interviews against the eight-item Patient Health Questionnaire. Model choice explained 30.0% of summed-symptom score variance, stable participant differences 10.5%. Two randomly drawn raters with area under the receiver operating characteristic curve (AUC) >= 0.70 disagreed on screening decisions for 40% of participants, on average. Average over-rating governed how many were flagged, yet equal-capacity raters chose differently for about one participant in five. A locked analysis of 86 new interviews reproduced the main pre-registered findings. Exploratory recalibration with 40 labelled participants raised accuracy from about 60% to 75% and halved disagreement, leaving one participant in five decided differently. Calibration repaired much of the rater dependence without securing agreement about individuals.
Source: Language-model ratings of depression reflect the rater more than the patient