acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing and Repairing World-Model Errors for Planning

Abstract

World models guide planning by predicting action consequences, yet lower average prediction error need not improve action selection. Relative errors in predicted returns can reverse candidate rankings even when average prediction error is small. We develop a support gate that couples penalties on positive predicted rewards to state and action support. A complementary pairwise action-effect loss supplements prediction training by matching differences between real next-state outcomes of alternative actions from the same state. Our analysis connects selection loss to relative score errors and explains how the two designs modify planning scores and relative outcome predictions. The gate reduces CartPole mean normalized regret from 0.676 to 0.077 without retraining, and improves return in 20 of 27 models across nine public-data settings. Component comparisons show further gains from pairwise training with the gate on DoorKey and CartPole, with the full combination performing best among the evaluated configurations. An independent acquisition study under matched interaction budgets finds that additional data can improve control beyond continued training.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.