AI Insight
Researchers developed a new artificial intelligence approach that combines symbolic logic planning with neural network learning to help AI agents learn complex tasks more efficiently. The system, called neurosymbolic Hierarchical Reinforcement Learning with Incremental Knowledge (InK), uses high-level symbolic planning that updates as the AI explores its environment, while low-level neural modules learn specific movement skills. Tests on navigation tasks showed this approach significantly improved sample efficiency compared to traditional methods that use fixed, unchangeable knowledge structures.
Why it matters
This research could accelerate AI training in scenarios where feedback is rare or delayed, such as robotics, autonomous vehicles, or complex decision-making systems. By reducing the amount of trial-and-error needed for learning, the approach could make AI development more practical and cost-effective for real-world applications.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: (Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning. A compelling approach to improve sample efficiency is to incorporate knowledge into learning and decision-making. In standard Hierarchical RL (HRL), knowledge is encoded in a fixed, non-updatable form, such as architectural choices, and remains unchanged throughout learning. With fixed HRL, reasoning with incremental knowledge learned during exploration is impractical before sufficient environmental knowledge is acquired, leading to poor sample efficiency. In this work, we propose neurosymbolic HRL with {em Incremental Knowledge (InK)}: symbolic high-level components perform {em symbolic planning} (e.g. using $D^*$) on an updatable representation of current InK, while low-level goal-conditioned neural modules learn motion primitives through experience using reward shaping. Experiments on navigation tasks demonstrate that incorporating InK substantially improves sample efficiency. Additionally, to perform {em optimal} symbolic planning given {em prior} knowledge about the world, we develop Belief World Tree Search. The code is available at https://github.com/CPS-research-group/ink_bwts.