AI Insight
Researchers developed BRIDGE, a framework that predicts how long humans take to complete tasks by analyzing AI model performance across benchmarks. Using Item Response Theory, the system estimates task difficulty from model responses and found a linear relationship between latent difficulty and logarithmic human completion time. This method eliminates the need for expensive direct human time measurements and successfully reproduced findings showing AI capabilities double approximately every 6 months.
Why it matters
This approach provides a scalable, cost-effective way to evaluate AI progress in human-meaningful terms without requiring extensive human testing for each new benchmark. The ability to forecast when AI systems will handle tasks of specific human-equivalent difficulty has important implications for AI safety planning and workforce preparation.
Understand the Science
arXiv:2602.07267v2 Announce Type: replace
Abstract: Evaluating the real-world capabilities of AI systems requires grounding benchmark performance in human-interpretable measures of task difficulty. Existing approaches that rely on direct human task completion time annotations are costly, noisy, and difficult to scale across benchmarks. In this work, we propose BRIDGE, a unified psychometric framework that learns a latent difficulty scale from model responses and anchors it to human task completion time. Using a two-parameter logistic Item Response Theory model, we jointly estimate latent task difficulty and model capability from model performance data across multiple benchmarks. We demonstrate that latent task difficulty varies linearly with the logarithm of human completion time, allowing human task completion time to be inferred for new benchmarks from model performance alone. Leveraging this alignment, we forecast frontier model capabilities in terms of human task length and independently reproduce METR’s exponential scaling results, with the 50% solvable task horizon doubling approximately every 6 months.
Source: BRIDGE: Predicting Human Task Completion Time From Model Performance