- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DELTA, an action-conditioned world model built on frozen V-JEPA 2.1 visual features. The method combines dual-stream latent encoding, action-conditioned state transitions, and residual feature prediction. Specifically, two temporal observation windows are encoded into a persistent latent state using cross-attention, which is then updated through FiLM modulation and a Transformer conditioned on robot actions. A residual head predicts future feature changes instead of reconstructing complete visual representations. Experiments on LIBERO Object evaluate the method against several baselines, including DINO-WM and STORM, using open-loop prediction, replanning, and CEM-based action selection. The results demonstrate competitive multi-step feature prediction accuracy, while ablation studies highlight the importance of residual prediction and action conditioning.
Critical strengths
Clear organization and completeness of presentation:The paper is well-structured and presents a coherent narrative, progressing logically from the research background and motivation to the proposed methodology and conclusions. The key components of the proposed framework are described with mathematical formulations and an overall architecture diagram, making the methodology relatively easy to follow. Overall, the manuscript is reasonably complete in its organization and technical presentation.
Critical weaknesses
Insufficient justification and validation of the semantic-dynamics stream distinction: The distinction between the semantic and dynamics streams is primarily based on the lengths of their temporal input windows (two versus four frames), while both streams use the same frozen V-JEPA 2.1 encoder. The paper does not provide explicit supervision or empirical evidence demonstrating that the two streams actually capture distinct semantic and motion-related information. Since both temporal windows may contain overlapping semantic and dynamic information, the motivation for assigning these specific roles remains unclear. Furthermore, the single-stream ablation only demonstrates the benefit of adding another stream, rather than establishing the necessity of semantic-dynamics specialization. Additional controlled experiments using different temporal window combinations or feature-level analyses would be needed to substantiate this central architectural assumption. Insufficient justification for the additional cross-stream attention: Before concatenating the two streams, DELTA introduces an additional cross-attention module that allows the dynamics stream to attend to the semantic stream. However, the subsequent action-conditioned Transformer already employs self-attention over all concatenated tokens, enabling interactions between the two streams. Therefore, the necessity of this additional cross-attention and the choice of its asymmetric direction remain unclear. The paper lacks a direct-concatenation baseline or alternative fusion strategies to demonstrate that this design provides benefits beyond the subsequent self-attention module.
Questions for authors
1.What evidence supports the distinction between the semantic and dynamics streams?
The two streams primarily differ in their temporal input windows (two versus four frames), while both use the same frozen V-JEPA 2.1 encoder. What evidence demonstrates that they learn distinct semantic and motion-related representations rather than largely overlapping information? Could the authors provide controlled comparisons with alternative temporal window combinations (e.g., 2+2 or 4+4 frames), or feature-level analyses to justify this distinction? This is important for assessing whether the proposed dual-stream design provides meaningful specialization beyond additional temporal context and model capacity.
2.Why is an additional asymmetric cross-attention module necessary before concatenating the two streams?
The dynamics stream attends to the semantic stream before concatenation, but the subsequent action-conditioned Transformer already performs self-attention over all concatenated tokens. What additional benefit does this asymmetric interaction provide? Could the authors compare it with direct concatenation without the extra cross-attention module? This comparison would help establish whether the proposed fusion mechanism is necessary for the reported performance improvements.
3.Could the authors clarify the evaluation protocol and its consistency with the supplementary implementation?
The main paper reports results on 20 test episodes, whereas the supplementary README describes a fixed eight-episode evaluation subset. Which episodes were used to generate the main results in Tables 1–3? Additionally, the provided evaluation scripts call predict_features() without supplying the base features required when residual prediction is enabled. Could the authors clarify which evaluation implementation and model configuration produced the reported results? Resolving these discrepancies is important for assessing the reliability and reproducibility of the experimental comparisons.
Response to Reviewer KVje
We thank the reviewer for a constructive and specific report. All numbers below are the metric used throughout the paper (pooled semantic-token MSE, 10^2), on the same episodes, at the same checkpoints.
Q1 -- Evidence for the semantic/dynamics distinction.
We do not claim that the two streams learn different content from the same encoder. We claim that they read different temporal extents and that both are needed. Three controls, now added as Table 4 of the revision:
Shortening the long window costs 27% at h=40, lengthening the short window costs 6%, and cutting to one stream costs 14%. The 4+4 gap sits close to the seed spread (4%, measured across two seeds of the full model), so we do not claim more than this: both windows are needed, and the asymmetric pairing is the best of the three. Short horizons are unaffected in every case (0.34 to 0.35 at h=5), which is consistent with the streams serving the long-horizon carry rather than the one-step term.
Q2 -- Is the cross-stream cross-attention redundant given the later self-attention?
We replaced the cross-stream step with a concat plus a linear projection, keeping both stream readers and matching capacity (the variant has slightly fewer parameters, 5.39M against 5.52M):
The 0.07 gap at h=40 is smaller than the seed spread, so we do not claim that the coupling operator is the source of the gain, and we have removed any phrasing that implied it. What the coupling does buy is that the two streams can carry different token counts (K and 2K) while sharing one conditioning path, which is what the 2- and 4-frame windows require; a concat forces a common token count. We thank the reviewer for pressing on this, since the original wording overclaimed.
Q3 -- Which episodes produced Tables 1-3, and the evaluation script.
The tables use a fixed evaluation subset of the held-out test split, defined in the released split file and applied identically to every method and every table. The paper now names it that way rather than saying "the test split", which is where the discrepancy with the README came from. For completeness, the full 20-episode test split gives 0.21 / 0.35 / 0.46 / 0.97 / 1.76 at h=1/5/10/20/40 and 0.70 under replan at r=10; every ordering in Tables 1-3 is unchanged on it.
On the script: the reviewer is correct that the shipped predict_features() call omitted the base features. That call is valid only when use_feat_residual is disabled; with the residual head enabled it silently drops the anchor the head was trained on, so the shipped script could not reproduce the residual rows. The updated supplement passes the base pair and feeds the predicted pair back between replanning steps, matching the training-time interface, and the README records the change. The numbers in the paper were produced by the corrected path. We are grateful for the catch.
We thank the reviewer again for a report that led to concrete improvements in both the paper and the released code.