AI Insight
CaliPPer is a new computational framework designed to assess and improve the reliability of AI models that predict molecular binding interactions, such as antibodies, T-cell receptors, and small molecules binding to their targets. The method uses a distance-based approach to estimate how well models will perform on new datasets without requiring experimental labels, and includes a recalibration step that improved prediction accuracy (AUROC) by up to 0.20 in testing. When applied retrospectively to five published studies, CaliPPer increased the rate of successful experimental validations, for example identifying 3 out of 5 confirmed cancer neoantigens compared to 0 out of 5 with the original predictions.
Why it matters
This tool addresses a critical bottleneck in drug discovery and immunotherapy development, where computational predictions often fail when applied to new targets, wasting significant experimental resources. By providing confidence estimates for individual predictions and improving model calibration, CaliPPer could substantially reduce failed experiments and accelerate the identification of therapeutic candidates.
arXiv:2606.07258v1 Announce Type: cross
Abstract: Binding prediction models accelerate therapeutic antibody and TCR discovery, but their performance on new datasets is unpredictable, often leading to low discovery rates. Density-ratio methods (PAPE, M-CBPE) provide label-free performance estimation for binary classification, but their assumptions and aggregate-only outputs limit binding prediction on neoepitopes, antigen variants and chemical scaffolds. Here we present CaliPPer (Calibration and Prediction of Performance), a post-hoc framework pairing a multi-chain Sample-to-Domain Distance (S2DD) with distance-aware Bayesian recalibration, operating at three resolutions: generalisability score, aggregate performance prediction, and per-sample confidence. Across ten models, eight architectures and two immune-receptor domains, CaliPPer attains distance–performance correlations $|r|=0.80text{–}0.92$, predicts AUROC/AP/F1 with mean absolute errors $0.008text{–}0.070$, and improves AUROC by up to $+0.20$ on unseen epitopes/variants. Applied retrospectively to five published TCR, BCR, MHC–peptide and small-molecule studies, CaliPPer raises true discovery rates in all five (e.g. $0/5 to 3/5$ confirmed neoantigens), providing a triage layer between computational prediction and experimental validation.
Source: CaliPPer: quantifying, predicting and improving AI model performance for binding prediction