AI Insight
D-JEPA is a new latent world model that improves decision-making in robotic systems by learning which predicted future states will lead to successful action execution, rather than just predicting future states accurately. The model addresses a key problem where actions predicted to be closer to a goal may actually perform worse than alternatives by learning decision-relevant relationships from actual executed outcomes. Testing across robotic manipulation tasks, autonomous driving, and physical robots showed substantial improvements, including 87.89% success on the PushT benchmark and significant performance gains on real robot tasks.
Why it matters
This approach bridges the gap between prediction and control in AI systems, potentially improving autonomous robots, self-driving vehicles, and other systems that must select effective actions in real-world environments. The method's ability to work with pretrained models and deploy across different robotic platforms suggests practical scalability for existing robotics applications.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Latent world models predict the consequences of actions, but accurate prediction does not guarantee that latent distance reflects which candidate will execute successfully. We identify a decision-local prediction gap: among the few futures competing for execution, a candidate predicted closer to the goal can produce a worse realized outcome than an available alternative. We introduce D-JEPA, a decision-aligned latent world model that learns decision-relevant relations among candidate futures from executed outcomes. A bounded, permutation-equivariant operator jointly reasons over goal-relative predictive features and ordinal evidence, refining pretrained predictive geometry where action choices are most consequential. Restricted predictor adaptation and a shared ordinal interface extend this alignment across complementary predictive geometries. D-JEPA further realizes the learned decision structure in JEPA-compatible future representations, enabling deployment through native latent-distance planning. Evaluations across latent control, manipulation, pretrained action-producing models, physical robots and autonomous driving demonstrate improved action selection, including 87.89% success on PushT, a 15.04-point average gain on RoboTwin, and a 17-point gain on physical robot tasks. These results establish decision-relevant relational structure as a direct bridge between predictive world modeling and effective control.