AI Insight
This study evaluates large language model (LLM) agents augmented with SageMath, a computer algebra system, for solving research-level mathematical problems. The researchers used a ReAct-style framework that combines LLM reasoning with verifiable computational feedback and tested it on the RealMath benchmark across multiple frontier models. Results showed that SageMath access improved performance by an average of 9.7 percentage points across all models, with GPT-5.5 achieving the highest solve rate of 75.2%.
Why it matters
This work demonstrates that integrating computer algebra systems with AI reasoning agents can significantly enhance mathematical problem-solving capabilities, potentially accelerating computational mathematics research and moving toward automated conjecture discovery. The approach offers a practical tool for assisting mathematicians in computational exploration and narrows the performance gap between open-weight and closed AI models.
Understand the Science
arXiv:2607.06820v1 Announce Type: new
Abstract: Recent advances in AI for Mathematics have focused largely on autoformalization and theorem proving, leaving the role of Computer Algebra Systems (CAS) in agentic LLM workflows underexplored. We propose a ReAct-style agentic setup that combines LLM reasoning with verifiable feedback from SageMath, together with Context7 for the up-to-date documentation. We evaluate this agentic setup across frontier models for solving research-level mathematical problems from the RealMath benchmark in a setting that emulates a computational-mathematics research loop. We also propose a refinement to the RealMath benchmark by introducing a multi-step post-processing procedure and a multi-stage validation pipeline, both of which improve the quality and reliability of the extracted problem set. Our experiments reveal substantial performance gains from SageMath access across all evaluated models on +9.7~pp on average, the gains range from 1.5~pp to 27.8~pp and narrow the gap between open-weight and closed models. Qwen~3.7-Max benefits from SageMath the most, while GPT-5.5 achieves the highest solve rate of $75.2%$ and the lowest token usage among tool-enabled configurations. Our findings suggest that CAS-augmented agents represent a promising direction for assisting mathematicians in computational exploration, and we believe that this work is a step towards automated conjecture discovery. The project repository is available online.
Source: Evaluating SageMath-Augmented LLM Agents for Computational and Experimental Mathematics