AI Insight
This study analyzed over 450,000 cardiology encounters in the VA healthcare system to evaluate ten medication-based performance measures used to assess individual clinicians treating coronary artery disease, heart failure, and atrial fibrillation. While most measures showed adequate statistical reliability for comparing clinicians, their accuracy was poor to moderate when compared against expert peer review, with the automated measures consistently underestimating actual clinician performance. The findings suggest these measures can identify potential care gaps but should not be used alone to judge clinician quality.
Why it matters
Healthcare systems increasingly use automated performance metrics to evaluate individual physicians, which can affect compensation, reputation, and career advancement. This research demonstrates that commonly used medication prescribing metrics may unfairly penalize clinicians because they miss clinical nuances that peer reviewers can identify, highlighting the need for more sophisticated assessment methods.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Background: Multiple performance measures are used to assess the management of common cardiovascular conditions, such as coronary artery disease (CAD), heart failure with reduced ejection fraction (HFrEF), and atrial fibrillation (AF). These measures are sometimes used to assess individual clinicians, despite lack of evidence of reliability and accuracy. We used national Veteran Affairs (VA) data to test medication-based performance measures for assessment of clinician performance in outpatient cardiology care. Methods: This retrospective study examined VA cardiology clinician/Veteran dyads from 2017-2023 to evaluate 10 medication-based performance measures for CAD, HFrEF, or AF with elevated stroke risk. Achievement of each performance measure was defined as an active prescription of an indicated medication for each eligible patient encounter, summarized at the clinician level. Reliability was calculated using the beta-binomial method. A Bayesian causal model was used to assess the impact of case complexity on performance measure achievement. Finally, we performed peer review of a subset of cases (n=952) to assess the accuracy of three representative performance measures. Results: Of 456,071 unique cardiology encounters, 312,331 included CAD, 224,599 included HFrEF, and 178,678 included AF. These patients were treated by 1,043 cardiology clinicians. Median performance measure achievement ranged from 0.23 (SGLT2i for HFrEF) to 0.99 (beta blocker for HFrEF). Measure reliability varied from 0.63 to 0.90. Case complexity did not significantly affect performance measure achievement. Using peer review as the gold standard, accuracy was poor to moderate (balanced accuracy 0.51-0.73), and measures underestimated performance relative to peer review. Conclusions: Ten medication-based performance measures for CAD, HFrEF, and AF can be efficiently measured using electronic health record data, and most have sufficient reliability to support clinician benchmarking and the identification of potential care gaps. However, suboptimal accuracy compared with a peer review indicates that these measures may be insufficient as a standalone assessment of clinician quality.