AI Insight
Researchers discovered that goal-conditioned AI world models can achieve high accuracy on spatial reasoning tasks through "instruction leakage" rather than genuine perception. When the model is given instructions like "put the red block left of the blue block," it transcribes the instruction rather than understanding the actual spatial relationships, as evidenced by accuracy collapsing from 0.90 to 0.27 when the goal is withheld and the model following false instructions 94.5% of the time. The study proposes a solution by separating goal information from the dynamics model and supervising only the perception pathway, recovering genuine spatial grounding that maintains 0.88 accuracy regardless of goal presence.
Why it matters
This finding reveals a fundamental flaw in how AI systems appear to understand spatial relationships and language grounding, which has implications for robotics, autonomous systems, and any AI application requiring genuine scene understanding rather than pattern matching. The proposed fix provides a methodological framework for building more robust world models that truly perceive spatial relations.
Understand the Science
arXiv:2607.06925v1 Announce Type: new
Abstract: Compact world models that condition on a language goal promise to ground relations such as “put the red block left of the blue block” using a sparse set of explicit emph{reference anchors}. We ask when such references actually ground a relation, and identify a trap: a goal-conditioned predictor reaches a striking $0.90$ relation-readout accuracy, yet this is emph{instruction transcription}, not perception. Withholding the goal collapses it to chance ($0.90!to!0.27$, three seeds) and a counterfactual instruction makes the predicted anchors follow the emph{false} instruction $94.5%$ of the time (true scene $2.3%$; $N{=}256$). Tested across three settings and a within-task ablation, our central claim characterizes the confound: textbf{instruction leakage occurs when the scored quantity is transcribable from the instruction (when the instruction names the answer) and is essentially independent of how predictive the non-instruction inputs are.} Our tabletop and the external BabyAI benchmark leak, whereas a Language-Table forward-dynamics world model whose instruction names emph{referents} does not, until the instruction is augmented to name the direction; and degrading the action never increases leakage, the opposite of what predictor-competition predicts. The diagnosis prescribes the fix: keep the goal out of the dynamics (it belongs to the planner’s cost) and supervise the emph{read} path, recovering genuine, instruction-independent grounding ($0.88$, identical with and without the goal). The detection protocol and remedy apply to any goal-conditioned world model whose instruction names the scored quantity.