AI Insight
This study provides a rigorous benchmark comparison of machine learning models for predicting drug ADMET properties, finding that no single model type consistently outperforms others across different endpoints when evaluation protocols are carefully controlled for fairness. Graph neural networks, pretrained molecular models, and traditional fingerprint methods each excel on different tasks, with performance differences often falling within normal seed-to-seed variation. The research demonstrates that previously reported model superiority claims may reflect unfair comparison protocols rather than genuine model quality differences.
Why it matters
The findings challenge the assumption that more complex deep learning models are universally better for drug property prediction and highlight critical methodological flaws in published comparisons. This work provides practical guidance for pharmaceutical researchers selecting computational tools for early drug screening, emphasizing the need for endpoint-specific model selection rather than relying on a single "best" approach.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Structure-based prediction of ADMET (absorption, distribution, metabolism, excretion, and toxicity) properties promises fast, low-cost screening of drug candidates, yet model selection remains unclear because published comparisons of molecular representations often use unmatched tuning budgets and leakage-prone evaluation protocols, producing performance differences that may not reflect model quality. To address this gap, we benchmark graph neural networks (GNNs), frozen pretrained models (ChemBERTa, MolCLR), and fingerprint baselines across six Therapeutics Data Commons ADMET endpoints under scaffold-split evaluation. To make it really fair we gave every model family a matched 50-trial, validation-selected tuning budget and reported the median over ten seeds; architecture and hyperparameter selection use validation data only, with the held-out test split reserved for final performance estimation. Under this fair comparison protocol, emph{no model family dominates} and the differences between families fall within the seed-to-seed range on most endpoints. The GNN is clearly best only on Caco-2 permeability (R$^2 approx 0.48$), and a controlled ablation shows this advantage comes entirely from auxiliary physicochemical descriptors rather than the graph representation; Morgan fingerprints are competitive on hERG (AUC $approx 0.77$) and give the best median on both clearance endpoints, while MolCLR is best on Tox21. For hepatocyte clearance and half-life, all models yield near-zero or negative R$^2$. No single hyperparameter-optimization strategy, among seven evaluated, consistently outperforms random search at this budget, and we show that selecting the optimizer using test-set performance or reporting only single best-trial runs can substantially inflate a favored model’s apparent advantage. Reliable ADMET modeling therefore requires endpoint-specific model selection together with fair, leakage-controlled, seed-averaged evaluation, offering more trustworthy decision support for early-stage drug candidate prioritization.
Source: Task-Dependent Performance of GNNs and Pretrained Models in ADMET Prediction