AI Insight
This study identifies two critical flaws in how researchers test whether AI world models respond to actions. First, standard diagnostic tests can falsely detect action effects when actions are autocorrelated, because future actions serve as proxies for unobserved historical information. Second, when many data points share the same action sequence (common in world models), pooled statistical tests amplify small model errors into false evidence, with this effect worsening dramatically as the number of pooled observations increases. The authors propose using tempered e-value statistics with mixture approaches over fixed grids to avoid these pitfalls, validating their method on biological optogenetic experiments and multiple world model architectures.
Why it matters
These findings reveal fundamental problems in how action-conditioned world models are currently validated, potentially affecting the reliability of AI systems used in robotics, autonomous systems, and scientific discovery. The proposed statistical framework provides researchers with tools to properly test whether their models genuinely learn causal action effects rather than spurious correlations.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
-cross
Abstract: Action-conditioned world models are increasingly trained on data from randomized experiments, and whether their predictions respond to actions is usually checked by replacing or removing the actions and measuring the loss in accuracy. Taking biological experiments as the guiding case, we show that such diagnostics report action effects where there are none: under autocorrelated designs, future actions stand in for history the model does not see. When the design law is known, a conditional randomization test written as an e-value is valid in finite samples whatever the model, provided the resampler conditions on the full state the design used. A second failure arises whenever many sites share one action sequence, as in world models: the pooled score turns small model errors along the action contrast into overwhelming evidence against a true effect. In a pixel world, its median log e-value fell from -17.3 to -6832 as frames grew from 64 to 4096 pixels, while the oracle’s rose. Tempering the score changes the bet, not the validity. Our own preregistered analysis of optogenetic ERK experiments failed in exactly this way; a tempered statistic, registered before the data were obtained, then rejected the null on a never-opened field (E = 1.1e10). Every single tempering factor we tried, estimated from residuals or tuned on validation data, collapsed in some setting; mixtures over a fixed grid of factors never did. In registered tests on three world-model families trained by others, the pooled test rejected in 0 of 96 units of a diffusion model in which actions mattered, and reached the registered power bar in 10 of 26 Atari games under IRIS’s own token likelihood and 1 of 6 control tasks under TD-MPC2; mixtures fixed in advance reached it in all 96 units, 24 of 26 games and 6 of 6 tasks. Under exact nulls the statistics made 4 rejections in 22,848 tests.