- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Cross-View World Models (XVWM), which train an action-conditioned diffusion world model to predict a future frame from either the same or a different camera view. The experiments use 4,186 synchronized Aimlabs gameplay episodes with egocentric, bird's-eye, over-the-shoulder, and front views. The paper compares Single-View, Two-View, and Four-View models, evaluates cross-view generation and bird's-eye marker localization, and probes hidden states for a shared linear code of allocentric position and heading. The main interpretability result is that linear heading and position probes transfer between egocentric and bird's-eye contexts in middle transformer blocks, and that injecting a yaw action rotates the decoded heading in this interval.
Critical strengths
1.The paper proposes a simple and well-motivated cross-view prediction objective and evaluates it beyond qualitative samples. In particular, Table 1 measures ego-to-BEV localization on 1,257 test samples, and Figure 6 evaluates ego-target perceptual quality with LPIPS and DreamSim, including bootstrap 95% confidence intervals. The Two-View model obtains a 2.0-pixel median BEV marker error, while the marker is approximately 17 pixels on its long side. These results substantiate that the model can associate egocentric context with global position in the evaluated setting.
2.The interpretability analysis is unusually careful for this type of model. The authors use episode-grouped cross-validation, test zero-shot transfer of a probe between ego and BEV contexts, include shuffled-label controls, and apply frozen probes to disjoint episodes under yaw interventions. Figures 3–5 give convergent evidence that, in the reported middle-layer interval, pose is linearly decodable across views and the decoded heading changes approximately proportionally to the injected yaw. This is a meaningful characterization of the model's internal representation.
Critical weaknesses
-
The evidence is limited to an episode-level 90:10 split within a known Aimlabs environment. The paper does not report a held-out environment, map, or spatial-region split, and it explicitly identifies transfer to unseen environments as an open question. This leaves an important alternative explanation for the strong localization and BEV-to-ego results: the model may learn a representation specialized to the training environment and its fixed map rather than a more general cross-view spatial representation. The current claims should therefore be limited to spatial grounding in a known environment unless stronger generalization experiments are provided.
-
The intervention supports action-equivariance of a probed heading readout, but it does not yet establish that this code causally mediates image generation or that it is target-view independent. In Section 4.1, the target view is always egocentric during probing; therefore, the shared code observed for ego and BEV contexts may be partly specific to producing an egocentric target. The paper itself notes that changing the action input and reading the representation does not establish that the model uses this code to produce images. This limits the strength of the mechanistic claim that the model applies actions through a general view-invariant cognitive-map code.
Questions for authors
1.Can the authors evaluate a stronger generalization split, such as held-out Aimlabs environments/maps or spatially disjoint regions of a map? At minimum, please clarify how the 90:10 episode split controls overlap in locations and near-duplicate trajectories between training and test data.
2.Can the probing and yaw-intervention analyses be repeated with bird's-eye view as the target view, and ideally with multiple output views? In addition, can the authors test causal mediation more directly, for example by editing the identified heading-code direction at an intermediate layer and measuring the resulting change in generated images?
3.In Figure 6(b,c), the exposure-matched Single-View results are intermediate checkpoints, whereas the Two-View model is fully trained. Please report the matched total update count and total view-frame exposure, and clarify whether a control can hold total optimization compute fixed while isolating the effect of cross-view targets. This is needed to support the claim that the reported improvement is specifically positive transfer from cross-view training.