- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies whether planning can be improved by training world models to be less sensitive to their own prediction errors. The authors introduce two auxiliary losses that compare simulated futures starting from the real next state and the model's predicted next state, under identical subsequent actions. They fine-tune an existing DINO-WM predictor using these losses and evaluate whether it ranks candidate action sequences more accurately. Although the losses alter how prediction errors propagate, the resulting planning improvements are inconsistent and depend on the training budget.
Critical strengths
-
The paper directly tests whether making a world model less sensitive to its own prediction errors leads to better action selection, rather than assuming that it does. The authors state that the test was specified in advance (Pages 2 and 6), and they clearly report the negative result, noting that “Every FG-AS regret interval includes zero” (Page 7). This makes the finding more credible, and it shows why it is important to test whether this measure is useful before using it as a training objective.
-
The comparison is carefully controlled. All methods start from the same DINO-WM predictor and evaluate the same candidate action sequences (Table 1, Page 6), and the selected sequences are scored with the true simulator (Page 7). The authors also report results both at an equal number of training updates and at equal wall-clock time, along with the cost of each loss (Table 6, Page 15). This is useful because it shows how the conclusions change with the training budget, which would be easy to miss if only one comparison were reported.
Critical weaknesses
-
The better results at equal wall-clock time may simply come from FG-AS training for fewer steps, rather than from the auxiliary losses. Table 2 (Page 7) shows that Native performs worse as training continues: from 1,200 updates to 1,800 s, its regret rises from 3.124 to 3.309 on PointMaze and from 11.570 to 13.088 on PushT, while FG-AS changes much less (3.228 to 3.158 and 12.485 to 12.168). Native after 1,200 updates also has the lowest regret of all six method and budget entries in Table 2, on both tasks. Within 1,800 s, Native runs about 7,748 updates and FG-AS about 1,289 (Table 6, Page 15). This makes the main positive result less convincing, since FG-AS may only look better because it avoids the drop in performance that comes with longer training. The one paired interval that excludes zero, a 5.00-point success gain on PointMaze at 1,800 s (Page 7), follows the same pattern, as Native's success falls from 66.46 to 62.92 while FG-AS goes from 67.50 to 67.92. This matters because the result reverses at equal updates, and none of the regret differences is statistically clear: every FG-AS regret interval includes zero (Page 7).
-
The paper does not explain why reducing the model's sensitivity to its own prediction errors does not reliably improve planning. Across 40 comparisons with Native, the response is reduced in 23 to 25 cases, but regret improves in only 18, and both happen in only 10 to 12 (Section 5.2, Page 8). The authors acknowledge that their gradient diagnostics establish “scale mismatch and occasional local opposition, not a cause of planning outcomes” (Page 8), that the mechanism analysis “remains descriptive”, and that their comparison cannot identify routing as the cause of a planning difference (Page 9). The broader idea that better internal model measures do not always lead to better decisions is already known, as the introduction itself notes (Page 1). With two tasks, one architecture and five seeds (Section 7, Page 9), it is also hard to tell whether the method has a small benefit or none at all. Since the negative result is the paper's main contribution, I would have liked a clearer explanation of why it happens and what new insight it provides.
Questions for authors
-
(Clarification) Could you show how Native's regret changes throughout training? In Table 2, Native performs better after the same number of updates, but FG-AS performs better after the same training time, mainly because Native's regret increases with further training while FG-AS improves slightly. The training curve would show whether the improvement comes from the auxiliary losses or simply from differences in training behaviour. The paper notes that DRF shows the same directional split between the two budgets (Page 7).
-
(Clarification) Could you show an example from Figure 4 (Page 8) where the response is reduced but regret is higher than Native's? Figure 3 (Page 8) shows representative roots with their true regret and success, but not the predicted costs of the candidates or how each model ranks them. Showing these for one root from such a contrast might help explain why a lower response does not lead to better choices.