AI Insight
Researchers tested whether machine learning models trained to classify vascular disease using gene expression data could reliably transfer across different studies, tissues, and clinical endpoints. While one model achieved excellent performance (AUC 0.947) in internal validation using abdominal aortic aneurysm samples, its performance varied dramatically in external datasets—ranging from perfect classification in one small plaque sample set to near-random performance in others depending on data processing methods. The study demonstrates that strong predictive accuracy within a single dataset does not guarantee robust performance across different vascular disease contexts or patient populations.
Why it matters
This work highlights critical limitations in developing universal blood vessel disease risk prediction tools from gene expression data. The findings emphasize that researchers and clinicians cannot assume that promising diagnostic or prognostic models will work reliably when applied to different patient groups, tissue types, or data collected using different laboratory methods without rigorous cross-study validation.
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Strong discrimination within a vascular transcriptomic study may not transfer across clinical endpoints, tissue compartments or assay processing. We evaluated seven public Gene Expression Omnibus datasets containing 294 deposited observations, including repeated lesion segments. Four conventional classifiers and a prior-only control underwent nested validation in 48 abdominal aortic aneurysm samples, followed by external evaluation under three preprocessing configurations. Source-rank L1 logistic regression achieved an area under the receiver operating characteristic curve of 0.947; external values were 0.753 and 0.667 in two carotid cohorts and 1.000 in 11 macrophage plaque samples. The latter value fell to 0.367 under a separately source-tuned common-gene pipeline. In 126 plaque donors with censored follow-up, Harrell concordance was 0.565 (95% interval 0.444-0.678) and 0.524 after a 30-day landmark. Processing-incompatible blood data served as a stress control; 43 correlated plaque segments were analysed descriptively. A matched permutation test supported source separation without testing generalisation. These results show that internal discrimination and isolated external success do not establish stable panvascular risk prediction. The benchmark supplies code, per-sample outputs and provenance for scrutinising pipeline sensitivity, endpoint mismatch and observation dependence.
Source: Cross study transfer of vascular transcriptomic classifiers in seven public cohorts