AI Insight
This study presents a new framework for evaluating how well large language models reason about healthcare interventions by grounding them in causal knowledge graphs. Testing with a cardiovascular disease example, researchers found that models provided with integrated causal graph information achieved the best performance in identifying causal relationships, adverse effects, and evidence-based claims, while ungrounded models showed higher raw intervention accuracy but lacked reasoning transparency. The framework uses four grounding conditions and automated scoring to systematically measure whether LLMs can properly reason about medical mechanisms, not just provide correct answers.
Why it matters
This addresses a critical gap in healthcare AI evaluation by moving beyond simple accuracy metrics to assess whether models actually understand causal relationships and evidence, which is essential for safe clinical decision support. The reproducible framework could help identify when LLMs are giving right answers for wrong reasons, potentially preventing harmful medical recommendations.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Large language models (LLMs) are increasingly proposed for healthcare decision support, but their evaluations still reward single-answer accuracy rather than reasoning about interventions, mechanisms, harms, evidence, and uncertainty. We propose a reproducible, graph-centered evaluation framework for intervention-oriented LLM behavior in healthcare and stress-test it in a cardiovascular pilot. The framework has four components: (i) a domain causal knowledge graph in which assertions are first-class, provenance-preserving nodes with stable identifiers; (ii) a scenario-conditioned subgraph extraction step that, given any clinical scenario, retrieves the relevant reified-assertion subgraph; (iii) four controlled grounding conditions that vary how the retrieved subgraph is composed into the model’s context (ungrounded C1, knowledge-graph C2, causal-graph C3, integrated C4); and (iv) an automated scoring pipeline, anchored on assertion identifiers, that computes intervention accuracy, and other evaluation measures on a single pass. To test the framework, we built a category-balanced scenario generator across eight reasoning failure modes and instantiated it on a cardiovascular graph. The metric panel discriminates conditions along interpretable, non-redundant axes: C4 obtains the strongest causal edge F1 (0.838), adverse-effect F1 (0.833), evidence accuracy (0.738), and unsupported claim rate (0.114), while C1 obtains the highest raw intervention accuracy (0.948) with no measurable causal or evidential grounding.
Source: Framework for Grounding Healthcare LLMs in a Causal Knowledge Graph: A Cardiovascular Example Pilot