AI Insight
This study evaluates how recently developed machine learning models for atmospheric simulation respond to uniform sea surface temperature warming, comparing their performance against a traditional physics-based climate model. The ML models (ACE2-ERA5, NeuralGCM, and cBottle) successfully reproduce some key climate responses like precipitation patterns, but show significant departures from physical models in radiative responses and land warming patterns. The findings reveal that while ML atmospheric models show promise for climate applications, they still struggle with generalization beyond their training data distribution.
Why it matters
As ML models become increasingly capable of long-term climate simulations, understanding their limitations in predicting climate change scenarios is critical for their reliable use in climate science and policy applications. These results indicate that current ML models require further development before they can be trusted for robust climate change projections.
arXiv:2510.02415v3 Announce Type: replace
Abstract: Machine learning models for the global atmosphere that are capable of producing stable, multi-year simulations of Earth’s climate have recently been developed. However, the ability of these ML models to generalize beyond the training distribution remains an open question. In this study, we evaluate the climate response of several state-of-the-art ML models (ACE2-ERA5, NeuralGCM, and cBottle) to a uniform sea surface temperature warming, a widely used benchmark for evaluating climate change. We assess each ML model’s performance relative to a physics-based general circulation model (NOAA’s Geophysical Fluid Dynamics Laboratory AM4) across key diagnostics, including surface air temperature, precipitation, temperature and wind profiles, and top-of-atmosphere radiation. While the ML models reproduce key aspects of the physical model response, particularly the response of precipitation, some exhibit notable departures from robust physical responses, including radiative responses and land region warming. Our results highlight the promise and current limitations of ML models for climate change applications and suggest that further improvements are needed for robust out-of-sample generalization.