- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a world-model architecture whose transitions exactly preserve a standard Gaussian latent reference under a specified behavior policy, making its long-horizon observation distribution accessible through static sampling. The method combines conditional action transformations, joint state–action mixing, and Gaussian noise while retaining a tractable conditional density. The authors derive finite-initialization convergence bounds and bounds on marginal departure under policy changes. Validation includes controlled dynamical systems and 216 recurrent pixel-model checkpoints across twelve control tasks. The primary model maintains complete numerical survival at 100,000 steps, with marginal statistics matching static-reference predictions. The experiments also demonstrate that these guarantees do not necessarily yield better short-horizon prediction or planning performance.
Critical strengths
- Controlled preserving architecture: The joint state–action construction preserves the Gaussian reference under the behavior policy while allowing action-dependent dynamics and a tractable conditional density.
- Computable marginal guarantees: The finite-initialization bounds accommodate empirical starting-state banks, while the policy-deviation analysis relates departure from the reference to changes in action distributions.
Critical weaknesses
- Limited demonstrated practical value: The guarantees characterize convergence to the model’s own reference distribution, rather than accuracy relative to the environment. The occupancy model has higher ten-step MSE than the Gaussian baseline (0.1271 versus 0.0708) and fails in 19 of 36 CEM planning settings. These results leave unclear when the proposed guarantees provide a useful benefit for dynamics modeling or control.
- Incomplete positioning against closely related methods: Although the paper cites DySLIM and PFNN, it provides limited substantive comparison with their approaches to invariant-measure learning and long-term dynamics. Generative Markov state models with explicit stationary distributions also warrant discussion.
Questions for authors
Q1: What is the underlying cause of the nineteen CEM planning failures, and how does this relate to the assumptions or entropy contraction required by your guarantees?
Q2: Which modeling capabilities or theoretical guarantees are newly enabled by your construction compared to PFNN and generative Markov state models with explicit stationary distributions?
Dear Reviewer iMap,
As the discussion deadline approaches, we would be very grateful for any further thoughts or feedback on our rebuttal and additional experiments. We sincerely appreciate the time and effort you have devoted to reviewing our work and would be happy to address any remaining concerns or questions.
Best regards,
Authors