AI Insight
This research demonstrates that humans improve goal-directed learning by compressing complex reward functions into simplified rules that can be stored in long-term memory rather than relying solely on working memory. Across six experiments, the study shows that learning efficiency decreases with larger goal spaces but improves when goals can be compressed into simpler structures, and that faster reward processing correlates with better learning outcomes. The findings suggest that efficient reinforcement learning depends on transforming flexible but cognitively costly goal evaluation into more automatic processes.
Why it matters
Understanding how humans optimize goal-directed learning through reward function compression could inform the design of educational interventions, training programs, and behavioral change strategies that align with natural cognitive processes. The research also has implications for developing more human-like artificial intelligence systems and supporting individuals struggling with goal achievement.
Understand the Science
arXiv:2509.06810v3 Announce Type: replace
Abstract: Humans can uniquely assign value to novel, abstract outcomes to support reinforcement learning. However, this flexibility is cognitively costly and reduces learning efficiency. We propose that goal-dependent learning initially relies on capacity-limited working memory. With consistent experience, learners create a “compressed” reward function – a simplified goal rule — that transfers to long-term memory for a more automatic evaluation upon receiving feedback. This automaticity frees working memory resources, thereby boosting learning efficiency. Across six experiments, we demonstrate that learning is impaired by the size of the goal space but improves when this space allows for compression. Additionally, faster reward processing correlates with better learning. Although the algorithmic details remain to be established, our behavioral results and computational models suggest that efficient goal-directed learning relies on compressing complex goal information into a stable reward function. These findings illuminate the cognitive mechanisms of intrinsic motivation and can inform behavioral interventions supporting human goal achievement.
Source: Reward function compression facilitates goal-dependent reinforcement learning