- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Iso-WAM, which trains a world-action model with auxiliary alignment between learnable queries in its action stream and compact latent actions extracted from observation pairs. The frozen Uni-LAM teacher maps semantic (DINOv3) and geometric (DA3) transitions into a shared 32-dimensional representation, trained to reconstruct future features and predict proprioceptive change. The paper evaluates on LIBERO, LIBERO-Plus, RoboTwin 2.0, and two real-world pick-and-place tasks. It reports 75.1% on LIBERO-Plus (7.5 percentage points above Fast-WAM-Joint), improvements in the RoboTwin clean-to-random setting, and higher success under the tested real-world lighting and tabletop changes.
Critical strengths
-
The paper provides a thorough comparison of alternative feature-alignment strategies. Under a common WAM evaluation framework, it compares static semantic and geometric features, single-modality latent actions, optical flow, dual latent-action targets, and the proposed unified target. This systematic comparison makes the empirical trade-offs among alignment choices substantially clearer than a comparison against a single baseline would.
-
The proposed transition representation is evaluated across multiple benchmarks and settings. The policy-level evaluation covers in-distribution LIBERO, the distribution-shifted LIBERO-Plus benchmark, clean-to-random generalization on RoboTwin 2.0 using two WAM backbones, and real-robot manipulation under lighting and tabletop-texture changes. The accompanying representation analyses provide additional evidence that Uni-LAM captures motion-related information. Taken together, these evaluations show that the proposed transition-based approach is not assessed on only one benchmark or one type of shift.
Critical weaknesses
-
The paper's central methodological contribution remains insufficiently clear. Iso-WAM combines transition-based supervision of the WAM action stream with Uni-LAM, which fuses semantic and geometric transitions in a shared latent space and uses auxiliary proprioceptive prediction. The experiments support the effectiveness of the complete design, but the manuscript does not clearly identify which of these elements constitutes the primary contribution, or how that contribution is distinct from prior WAM representation-alignment and latent-action methods. A sharper statement of the methodological novelty is needed for readers to assess what is new beyond the combination of these ingredients.
-
The robustness experiments do not clearly separate the effect of latent-action alignment from that of visual data augmentation, and the tested perturbations are too limited to support claims about robustness. Uni-LAM pretraining uses random resized crops and color jitter, while the augmentation protocol for policy training is not clearly specified and no augmentation-only control is reported. Moreover, the real-world evaluation mainly relies on fixed blue/purple lighting and a small set of patterned tabletop textures. These conditions test robustness to particular appearance changes, but do not sufficiently examine more diverse and stochastic shifts. Stronger randomized perturbations, such as varying illumination color, intensity and direction, camera viewpoint, object and background appearance, texture, occlusion, and distractor placement, would be needed to determine whether the observed gains generalize beyond the selected conditions.
-
The core role of latent actions in the proposed method is not sufficiently demonstrated. In Iso-WAM, latent actions are primarily used as an auxiliary alignment target to improve robustness under the visual distribution shifts considered in the experiments. Although this design provides practical performance gains, the paper does not show that latent actions play a broader or more fundamental role in representing transferable action semantics or enabling capabilities beyond visual robustness. Consequently, the contribution currently appears to be mainly the use of latent actions as an additional supervision signal, and the broader methodological significance and distinctiveness of this design remain insufficiently established.
Questions for authors
-
Could the authors clarify which element constitutes the primary methodological contribution of Iso-WAM and how it differs from prior WAM representation-alignment and latent-action methods? In particular, it would help to distinguish the contribution of transition-based action-stream supervision from that of Uni-LAM's semantic-geometric fusion and training objective.
-
Could the authors clarify the visual augmentations used during policy training and whether Iso-WAM and Fast-WAM-Joint follow the same augmentation protocol? Since no augmentation-only control is reported, it is difficult to determine how much of the robustness improvement comes specifically from latent-action alignment. In addition, could the authors evaluate the method under stronger and more diverse randomized perturbations, such as independently varying lighting, camera viewpoint, background texture, object appearance, occlusion, and distractor configurations? Such an evaluation would better establish whether the reported robustness extends beyond the fixed lighting and texture shifts currently considered.
7 comments
Response to Reviewer YdYf (2/3)
2. Does visual data augmentation introduce a confound, and how does the method perform under stronger randomized perturbations?
Data augmentation settings. We agree that image processing during teacher pretraining and policy training should be distinguished explicitly.
Iso-WAM and Fast-WAM-Joint use exactly the same policy image transformations: images are resized to 224×224, with no color jitter or random cropping during policy training. Therefore, the OOD improvement of Iso-WAM over Fast-WAM-Joint cannot be attributed to additional policy-side data augmentation. Random cropping and color jitter are used only during latent-action teacher training. The teachers used by DINO-LAM, DA3-LAM, and Dual, as well as Uni-LAM, follow the same augmentation settings. Consequently, performance differences among the transition targets are not caused by different augmentation configurations. We will document the settings for the two training stages separately in the revised manuscript to remove the ambiguity in the current description.
Stronger and more diverse randomized perturbations. Thank you for this question. We first clarify that our evaluation includes complementary settings: LIBERO-Plus examines seven categories of distribution shift, RoboTwin 2.0 clean-to-random tests combined environmental randomization, and real-robot experiments assess deployment under physical lighting and background-texture changes. Together, these results support the method's effectiveness across multiple distribution shifts rather than a single visual perturbation. To further address your concerns about perturbation strength and diversity, we conducted additional evaluations in the LIBERO simulator using four types of randomized perturbations—lighting, camera viewpoint, image noise, and occlusion—at greater intensities than in the previous experiments. We compared Iso-WAM with Fast-WAM-Joint under identical evaluation settings. Neither policy used data augmentation during training.
| Randomized perturbation | Fast-WAM-Joint success (%) | Iso-WAM success (%) | Gain (pp) |
|---|---|---|---|
| Clean | 92.1875 (59/64) | 95.3125 (61/64) | +3.1250 |
| Lighting | 79.6875 (51/64) | 92.1875 (59/64) | +12.5000 |
| Camera viewpoint | 75.0000 (48/64) | 90.6250 (58/64) | +15.6250 |
| Image noise | 42.1875 (27/64) | 85.9375 (55/64) | +43.7500 |
| Occlusion | 92.1875 (59/64) | 93.7500 (60/64) | +1.5625 |
In these additional evaluations with strong randomized perturbations, Iso-WAM outperforms Fast-WAM-Joint by 15 and 43 percentage points under camera viewpoint changes and image noise, respectively. These results further support the method's ability to improve action generation under substantial visual changes with identical policy image processing, extending the robustness benefits observed in the existing benchmarks to these additional test conditions.
Response to Reviewer YdYf (3/3)
3. The role of latent actions and the methodological significance of our work
We understand your concern: beyond improving performance, what methodological insight does using latent actions as supervision provide? The significance of our work lies in advancing WAM representation alignment from introducing additional visual features to systematically studying two design questions: what information should supervise action learning, and at which stage should complementary information be integrated?
Regarding supervision content, our work distinguishes scene descriptions from observation transitions. A representation that describes a future scene is not automatically the most suitable representation for supervising action generation. By comparing static features, latent transitions, and optical flow as targets, we investigate how this distinction affects action learning. After training on robot data and compression to 32 dimensions, the additional compact static target achieves 73.5% on LIBERO-Plus, below DINO-LAM's 74.3% and Uni-LAM's 75.1%. Thus, our findings go beyond showing that additional supervision helps: the information expressed by the supervision and the way it is constructed should be designed for action learning.
Regarding information integration, our work shows that complementary representations do not necessarily yield effective joint supervision. As discussed in the first section, directly aligning with individually effective semantic and geometric targets instead reduces performance. Uni-LAM therefore moves fusion into target learning on the teacher side, allowing multiple priors to form a shared transition representation before supervising the action stream. This provides a concrete design approach: information across representations can be integrated by learning a unified target, rather than leaving the downstream policy entirely responsible for reconciling multiple independent supervision spaces.
Using latent actions as supervision is therefore the implementation of this methodological design, rather than a reason to regard the contribution as merely combining existing components. The value of our work lies in identifying the importance of supervision content and information integration through systematic controls, and developing a unified transition target validated through policy experiments. We hope these explanations clarify the methodological significance of our work beyond simply adding an auxiliary loss.
Summary
Thank you for your constructive feedback, which has helped us clarify our methodological contributions and add more targeted experiments. Through these responses, we clarify the relationship between transition-based action-stream supervision and Uni-LAM's target construction, distinguish the augmentation settings used during teacher and policy training, and provide additional randomized-perturbation experiments supporting the method's robustness. We will incorporate these clarifications and experimental details into the revised manuscript to improve its clarity and completeness. We hope these responses address your main concerns, and we would be grateful if you would reconsider your rating in light of the additional evidence. We would be happy to provide any further clarification.
Response to Reviewer YdYf (1/3)
Thank you for recognizing our systematic comparison of alignment targets and our evaluation across simulation and real-world robotic settings. We appreciate your constructive suggestions and address your concerns below.
1. Main methodological contribution
Our contribution consists of two connected aspects: first, systematically reexamining representation alignment in WAMs; and second, using these findings to develop unified transition supervision for action generation.
First, we systematically compare alignment targets within a common WAM framework, revealing the importance of both representation choice and how representations are combined. The comparison covers static semantic and geometric features, single-modality latent transitions, optical flow, dual-target transition alignment, and a unified transition target. The results reveal complementary representational priors: geometric targets perform better under camera viewpoint shifts, whereas semantic targets are more stable under noise. More importantly, two individually effective targets do not necessarily remain beneficial when used jointly. In Table 4, semantic and geometric latent actions achieve 74.3% and 73.0%, respectively, whereas direct dual-target supervision achieves only 70.0%. These findings show that representation alignment in WAMs depends not only on which information is introduced, but also on how that information is organized and provided to the action stream.
Second, motivated by these findings, we fuse semantic and geometric transitions in the teacher to construct a shared observation-transition target for the WAM action stream. Uni-LAM extracts a compact shared code from an observation pair. Conditioned on current-frame features, this code supports the reconstruction of both future semantic and geometric features. Policy queries then interact with action tokens and predict transition codes for the corresponding temporal intervals, connecting scene-change prediction with action generation. Uni-LAM thus determines how the supervision target is constructed, while query alignment in the action stream determines how that target contributes to action learning.
The key design is to integrate information across representations during target learning, rather than requiring the downstream policy to fit two independent latent spaces separately. In Table 4, the unified teacher target achieves 75.1%, improving over direct dual-target supervision by 5.1 percentage points and outperforming both single-modality variants. This shows that simply adding two sources of supervision is insufficient to realize their combined benefits; the organization of the supervision target itself has a substantial effect. Thus, compared with the static-feature alignment or separate-target designs considered in our study, our main methodological innovation is to learn a unified semantic-geometric transition code through conditional reconstruction and use it as shared supervision for the action stream. The systematic comparison motivates this design, and Uni-LAM turns these findings into a concrete method.