- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to train world models to preserve information relevant to planning on downstream tasks. Specifically, it introduces a quantity called Decision-Regime Trajectory Sufficiency (DRTS) to measure whether a model preserves the action-conditioned distribution of future decision-regime sequences. The proposed method, BoundaryWorld, adds a regime-prediction loss to a latent world model loss and propagates its gradients through the encoder and dynamics. The paper reports improved regret and return on Safety-Gymnasium and MetaDrive.
Critical strengths
-
The paper addresses the meaningful problem that low state-prediction error does not necessarily imply good planning decisions. The distinction between predicting the regime sequence accurately and choosing a regime abstraction that preserves utility is useful.
-
The controlled experiments include several relevant comparisons, particularly marginal regime prediction, direct sequence prediction without latent dynamics, and variants that remove gradients through the dynamics. Ablations on sharing data, counterfactual branches, and CEM settings across methods help reveal the effects of the training objectives.
Critical weaknesses
-
The main motivation is to predict the joint regime sequence, but the planning score in Eq. (16) only needs per-step regime probabilities. It is therefore unclear whether modeling correlations across time explains the gains.
-
The regime construction is still difficult to follow. The paper should explain exactly how states receive regime labels and how each regime is assigned a utility and cost.
-
Several baselines use a regime-prediction head fitted after training. This is a useful controlled comparison, but it would also help to compare against baselines that plan directly with their learned value or cost predictions.
Questions for authors
-
Why is joint sequence prediction needed for the additive planning score in Eq. (16)?
-
How exactly are regime labels and their utility/cost scores constructed?
-
How do the baselines perform when planning with their own value or cost predictions?