acceptodds
Under review as a conference paper at ICLR 2027

How World Models Misjudge Actions: A Case Study of LeWM

Abstract

World models guide planning by comparing predicted action consequences, but the distinctions they make need not match those that matter for execution. We study this mismatch in LeWM across Reacher, TwoRoom, Cube, and PushT through matched candidate executions and interventions on physical states, commands, scoring, and output formation. Beyond an average ranking advantage of realized over predicted representations at matched endpoints in all four tasks, these interventions reveal mismatches in both directions. Cube action encoding nearly ignores differences that change object motion, whereas TwoRoom predictions respond to command differences discarded during execution. In Reacher, single-frame inputs conceal velocities that change future motion. In PushT pools admitting valid orientation-edited references for every candidate, scoring interactions involving the edit and prediction error can reinforce or offset incorrect action preferences. Averaging can carry nearly ignored action differences into execution or break equivalence between individually interchangeable commands, changing success without changing selected members. In two additional TwoRoom weight variants, removing execution-irrelevant command differences improves fixed-pool rankings, while separate closed-loop tests yield no observed gains in success. These findings motivate testing both sensitivity to consequential differences and respect for execution equivalence, together with how scoring and output formation turn model distinctions into decisions.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.