AI Insight
This study analyzes environmental impacts of training machine learning models over the past decade using the Epoch AI database, finding that energy use and environmental impacts have increased exponentially despite optimization efforts. Hardware improvements, algorithmic efficiencies, and use of cleaner energy sources have not mitigated overall impacts, suggesting a rebound effect where efficiency gains are offset by scaling up compute demands. The authors argue that life cycle impacts of hardware must be considered alongside energy use during training to avoid merely shifting environmental burdens.
Why it matters
The findings challenge the assumption that efficiency improvements alone can make AI development sustainable, highlighting the need for systemic changes in how the field evaluates and accounts for environmental costs. This has important implications for AI research planning, policy development, and corporate responsibility as language models continue to grow in size and computational requirements.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Recent Machine Learning (ML) approaches have shown increased performance on benchmarks at the cost of escalating compute demands. Hardware, algorithmic and carbon optimizations have been proposed to curb energy use and environmental impacts. We estimate the environmental impacts associated with training models documented in the Epoch AI database over the last decade, with a particular focus on impacts associated with Large Language Models and the hardware used to train them. We find that energy use and environmental impacts associated with training ML models have increased exponentially, even when considering impact reduction strategies such as using less carbon intensive electricity mixes or more efficient hardware. Optimization strategies do not mitigate the impacts induced by model training, suggesting rebound effect. We show that the impacts of hardware must be considered over the entire life cycle rather than the sole use phase in order to avoid impact shifting. Our study demonstrates that increasing efficiency alone does not ensure sustainability. There is an urgent need to systematically integrate environmental impacts in NLP evaluation practices to better inform the community and support the use of impact as a feature in research planning and decision making.