AI Insight
Researchers developed a framework for training AI agents that can perform tasks in multiple styles, allowing users to control how the task is executed after training is complete. Using universal value function approximators combined with specific training scenarios and data augmentation, they demonstrated that agents can respond to different style requests while maintaining task performance across three diverse domains: car racing in Gran Turismo, combat in Horizon Forbidden West, and humanoid walking simulations. The approach enables flexible, run-time control over agent behavior without requiring retraining.
Why it matters
This work addresses a significant limitation in current reinforcement learning systems, which typically learn only one fixed behavior. The ability to adjust AI behavior post-training has practical applications in video games, robotics, and human-AI interaction, where different situations may require different approaches to solving the same core task while maintaining user preferences and control.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Reinforcement learning has proven to be a valuable tool in the creation of advanced AI and robotic systems, contributing to everything from game playing (Silver et al., 2016; 2017; Wurman et al., 2022) to robotics (Levine et al., 2016; Andrychowicz et al., 2020; Luo et al., 2025; D”urr et al., 2026) to foundation models (Ouyang et al., 2022; Noukhovitch et al., 2024; Sun et al., 2024; Guo et al., 2025). Through trial and error, these AI systems typically learn a single, near-optimal behavior to solve their tasks. However, there are many use cases in which one would like to assert some level of control, after training is completed, over how the task is solved. We refer to these modifications of a core task as styles. We combine universal value function approximators (UVFAs) with carefully selected training scenarios, learning algorithms, and data augmentation to create a framework for coaching agents that exhibit styles in complex domains. We demonstrate the framework’s application in the AAA video games Horizon Forbidden West and Gran Turismo, and in an open-source humanoid test domain. Despite the different nature of the domains–car racing, stylized game combat, and humanoid walking–each agent shows strong coherence to the style requests while still satisfying the main task in its domain. Importantly, the techniques outlined in this paper allow an end user to choose the final behavior at run time, giving them flexible control over the final executed performance.
Source: Coachable Agents for Interactive Gameplay