- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses the problem that FastWAM-style world action models generalize poorly under visual distribution shifts. The authors argue that FastWAM struggles with unseen scenes and objects for two reasons: its reconstruction-oriented representations emphasize appearance-specific details, and it lacks observation history, which deprives it of temporal evidence.
To address this, they propose the Causal Semantic World Action Model (CSWAM), which adds a causal semantic expert built on a frozen V-JEPA 2.1 encoder to the existing FastWAM architecture. The expert predicts future semantic features from a sparse history of observations, aiming to make the model less dependent on appearance-specific details.
At inference, action denoising is conditioned on both the current video state and the observed semantic history, thereby retaining action-only inference.
The method is evaluated on RoboTwin 2.0 under both a mixed-training protocol and a Clean-to-Randomized transfer protocol, as well as on two real-robot tasks across three OOD difficulty levels, with ablations on the video expert, the observation history, and the semantic encoder.
The authors report that, with embodied pretraining, CSWAM improves Clean-to-Randomized success over FastWAM from 10.16% to 45.18%, and that it raises average real-robot success in the Hard and Extreme settings from 10% to 55%.
Critical strengths
S1. The Clean-to-Randomized evaluation protocol is stricter than the common mixed-training protocol.This setting measures genuine transfer, making it the paper’s most convincing evidence for its OOD generalization claim. Evidence: §4.1 and Table 3.
S2. CSWAM outperforms FastWAM in every real-robot task–difficulty combination.The gains remain consistent under compounded shifts in real-world robot experiments. I therefore find the main results credible and did not assign a low overall score. Evidence: Table 5.
Critical weaknesses
W1: It is not clear whether OOD gains come from the V-JEPA representation, the history frames, or the increased capacity of the additional expert. OOD generalization is the paper’s central claim, but the source of the gains remains unclear, making its explanation of why the method works unconvincing. This is my main reason for assigning a Soundness score of 1.
W2: The efficiency and scale mentioned in the paper are not supported by exact data. There's no number for latency, frequency, FLOPs or parameter count.The paper emphasizes efficiency, but provides insufficient data to verify this claim. If CSWAM is substantially larger, model capacity would further confound the attribution of gains discussed in W1, making that concern more serious.
Questions for authors
- Section 4.1 states that CSWAM uses the embodied-pretrained initialization on LIBERO-Plus, but Table 4 marks CSWAM's "Emb. PT" as "No". Which is correct? In either case, were FastWAM and ST-WAM in Table 4 initialized in the same way as CSWAM?
- Could the authors report the inference latency, control frequency, and parameter count of CSWAM and FastWAM, measured on the same hardware and under the same settings?
- Were the baseline results in Table 2 reproduced by the authors or taken from the original papers?
- In Fig. 2, why do the observed-context queries primarily attend to the action token (a_1) while masking out the semantic token (j_0) during inference? Is this a figure error, or does it reflect the attention pattern implemented in this work? If the latter, how can this be reconciled with the training mask and the reuse of the fixed observation context during denoising?
- I suggest reporting the same aggregate in the abstract and the conclusion. The abstract reports the average improvement over all three difficulty levels (from 27.5% to 70.0%), whereas the conclusion includes only the Hard and Extreme settings, with different numbers (from 10% to 55%).
4 comments
Updated Manuscript and Summary of Revisions
We have uploaded a revised manuscript incorporating additional experiments, clarifications, and corrections in response to the reviewers' feedback.
The main updates include:
- Section 3.3 (Fig. 2): Clarified the attention mask and cross-stream information flow.
- Section 4.2: Expanded the comparison with ST-WAM and the analysis of OOD robustness on LIBERO-Plus.
- Section 4.4 (Tables 6–8): Added ablations on the complementary video and semantic pathways, V-JEPA versus DINOv3, and temporal history length.
- Section 4.5: Added an inference latency comparison with FastWAM.
- Additional corrections: Corrected a typo regarding embodied pretraining in the LIBERO-Plus experiments and aligned the presentation of results in the Conclusion with the Abstract.
We thank the reviewers and Area Chair for their valuable feedback and welcome further questions or discussion.
Dear Reviewer,
We hope this message finds you well. As the discussion period is nearing its end with less than 24 hours remaining, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you’d like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper.
Dear Reviewer duap,
Please respond to the author soon and engage in the discussion.
Best,
AC
Response for Submission20013 review
W1: To clarify the sources of the OOD gains, we extend the ablations in Tables 6–8 from RoboTwin-Full to Clean-to-Randomized transfer. As in CSWAM with Embodied PT., all retained video/action experts use the same pretrained FastWAM weights, while additional expert DiTs are initialized from Wan2.2 video DiT weights via dimension-wise interpolation and rescaling to match their reduced hidden and FFN widths. The results show:
We will include the detailed results, settings, and analyses in the revised manuscript.
W2Q2. We agree that the original manuscript did not provide sufficient quantitative evidence for efficiency and model scale. We will add the following latency, throughput, and parameter measurements, together with the hardware and inference settings. Both models generate 32 actions per inference call.
CSWAM adds 8 ms of latency (+4.2%), while increasing total parameters by 1.114B and trainable parameters by 1.027B. Our efficiency claim therefore concerns retaining inference latency close to FastWAM through action-only denoising.
We do not rule out a contribution from increased capacity. To address this concern, the new C2R ablations discussed in W1. We will report the parameter increase explicitly and clarify the scope of our efficiency claim.
Q1: Thank you for catching this error. Table 4 is correct: CSWAM was evaluated on LIBERO-Plus without embodied pretraining. CSWAM, ST-WAM, and FastWAM all use the same embodied pretraining setup. The sentence in Sec. 4.1 was inadvertently carried over from the RoboTwin/real-robot setup and will be corrected in the newly submitted version.
Q3: Baseline results in Table 2 are taken from the original papers.
Q4: Thank you for catching this error. The inference mask in Fig. 2 contains a drawing mistake: observed-context queries attend to the semantic token j0, while the action token a1 should be masked out. We will correct Fig. 2; the reported results are unaffected.
Q5: Thank you for pointing this out. We will revise the conclusion to match the abstract, reporting average success across both tasks and all three difficulty levels: 27.5% for FastWAM and 70.0% for CSWAM. The Hard-and-Extreme aggregate will remain separately labeled in the results section.