- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper shows that even when a representation fully preserves state information, additional dimensions may be necessary to support regular prediction under bounded distance expansion across all actions. Theoretical lower bounds and synthetic experiments show that state identifiability alone does not guarantee a sufficiently regular predictive geometry.
Critical strengths
The paper shows that representing state information and regular prediction impose distinct requirements. Therefore, successful state recovery alone does not guarantee that a representation captures the predictive structure needed for the task.
Critical weaknesses
-
The paper does not provide a formal definition of state recovery in general. How can we determine whether a representation fully specifies the underlying state, and how robust is state recovery to noise in observations or measurements?
-
How this finiding can be expanded to recurrent models, e.g., in state-space models where state recovery is more challenging?
Questions for authors
The significance of the work and its impact on real-world applications seem limited. There is only one set of synthetic experimental results, which would be great if it could be elaborated/extended to real-world use cases.
We sincerely thank the reviewer for the constructive feedback and for recognizing the distinction between state information preservation and predictive regularity. Your comments have helped us better clarify our theoretical contributions and their scope.
Our central question and contributions
Our work is motivated by a fundamental question: Why do world models, which aim to learn compact state representations, often use latent dimensions substantially larger than the intrinsic degrees of freedom of the underlying system? What roles might these seemingly redundant dimensions play?
Previous studies have explored explanations including overparameterization, feature expansion, Koopman representations, and Neural ODE expressivity.
We identify and rigorously establish a different mechanism: even when all state information is preserved, realizing action-conditioned transitions under a prescribed Lipschitz gain may require additional dimensions because all actions share the same latent representation.
Our theoretical contributions are:
1. State recovery and noise robustness
We define exact state recovery through injectivity, equivalently, the existence of an inverse decoder on the represented state space (clarified in Section 3.1).
For example, a continuous two-dimensional state space (\mathcal X=[0,1]^2) cannot be losslessly encoded into (\mathbb R) by a continuous encoder. This illustrates the role of state-space structure; our present theorems require injectivity without assuming continuity.
We agree that robust recovery from noisy observations is important, but it is outside our scope. Our contribution concerns the additional dimensions that regular prediction may require even when state recovery is perfect.
2. Extension to recurrent models
Our framework is architecture-independent and also applies to deterministic recurrent hidden-state updates treated as input-conditioned transitions.
Recurrence alone does not remove the dimensional constraints imposed by gain-bounded dynamics. Recovering sufficient states from partially observable histories is a separate problem; when the hidden state is not sufficient, our analysis concerns its update geometry rather than recovery of the underlying physical state.
3. Practical significance
We have added experiments using trained LeWorldModel representations on PushT and Cube across five latent dimensions.
We observe that larger representations generally exhibit lower upper-tail transition expansion and prediction errors, accompanied by improved task performance. These findings extend our analysis beyond synthetic systems and support the practical relevance of predictive geometry, although they do not establish a causal dimensional requirement.