AI Insight
Researchers developed Gym-Anything for Robotics, a framework that uses AI coding agents to automatically create realistic robotic simulations at scale. The system generated RoboWorld, a benchmark of 25 simulations with 200 reinforcement learning environments based on economically valuable physical tasks identified through U.S. GDP data. Current state-of-the-art AI models performed poorly on these tasks, achieving only 0.06 average reward, but performance improved substantially through techniques like in-context learning, privileged teacher-student learning, and code-based policy generation for training vision-language-action models.
Why it matters
This work addresses a fundamental bottleneck in robotics AI development by automating the creation of training simulations that traditionally require significant expert effort. By focusing on economically valuable tasks and providing a systematic pipeline for simulation generation, this could accelerate the development of AI systems capable of performing useful physical work in real-world settings.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model’s own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as $pi_{0.5}$. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics
Source: Gym-Anything for Robotics: Scaling physical simulations with digital agents