acceptodds
Under review as a conference paper at ICLR 2027

Iso-WAM: Rethinking Representation Alignment in World Action Models

Abstract

World-Action Models (WAMs) adapt pretrained video generators for robot manipulation, but pixel-level future prediction entangles action-induced visual changes with scene appearances unrelated to the action, making control brittle under visual distribution shifts. Aligning policies with future static representation, such as semantic or geometric features, offers an alternative source of supervision, but these representations may still retain irrelevant background information. We propose Iso-WAM, which disentangles physical dynamics from visual appearance through dynamic representation alignment. Iso-WAM uses learnable action queries to predict latent action targets provided by a pretrained Unified Latent Action Model (Uni-LAM). Uni-LAM integrates semantic and geometric transitions into a single latent action space for alignment, providing the policy with comprehensive and robust dynamic guidance. Experiments show that Iso-WAM achieves strong in-distribution performance on the standard LIBERO benchmark and robust zero-shot OOD generalization on LIBERO-Plus and real-world robot deployments. In particular, Iso-WAM attains a 75.1% average success rate on LIBERO-Plus, improving over the Fast-WAM-Joint baseline by 7.5%. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world action models, representation learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YdYfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Iso-WAM, which trains a world-action model with auxiliary alignment between learnable queries in its action stream and compact latent actions extracted from observation pairs. The frozen Uni-LAM teacher maps semantic (DINOv3) and geometric (DA3) transitions into a shared 32-dimensional representation, trained to reconstruct future features and predict proprioceptive change. The paper evaluates on LIBERO, LIBERO-Plus, RoboTwin 2.0, and two real-world pick-and-place tasks. It reports 75.1% on LIBERO-Plus (7.5 percentage points above Fast-WAM-Joint), improvements in the RoboTwin clean-to-random setting, and higher success under the tested real-world lighting and tabletop changes.

Critical strengths
  1. The paper provides a thorough comparison of alternative feature-alignment strategies. Under a common WAM evaluation framework, it compares static semantic and geometric features, single-modality latent actions, optical flow, dual latent-action targets, and the proposed unified target. This systematic comparison makes the empirical trade-offs among alignment choices substantially clearer than a comparison against a single baseline would.

  2. The proposed transition representation is evaluated across multiple benchmarks and settings. The policy-level evaluation covers in-distribution LIBERO, the distribution-shifted LIBERO-Plus benchmark, clean-to-random generalization on RoboTwin 2.0 using two WAM backbones, and real-robot manipulation under lighting and tabletop-texture changes. The accompanying representation analyses provide additional evidence that Uni-LAM captures motion-related information. Taken together, these evaluations show that the proposed transition-based approach is not assessed on only one benchmark or one type of shift.

Critical weaknesses
  1. The paper's central methodological contribution remains insufficiently clear. Iso-WAM combines transition-based supervision of the WAM action stream with Uni-LAM, which fuses semantic and geometric transitions in a shared latent space and uses auxiliary proprioceptive prediction. The experiments support the effectiveness of the complete design, but the manuscript does not clearly identify which of these elements constitutes the primary contribution, or how that contribution is distinct from prior WAM representation-alignment and latent-action methods. A sharper statement of the methodological novelty is needed for readers to assess what is new beyond the combination of these ingredients.

  2. The robustness experiments do not clearly separate the effect of latent-action alignment from that of visual data augmentation, and the tested perturbations are too limited to support claims about robustness. Uni-LAM pretraining uses random resized crops and color jitter, while the augmentation protocol for policy training is not clearly specified and no augmentation-only control is reported. Moreover, the real-world evaluation mainly relies on fixed blue/purple lighting and a small set of patterned tabletop textures. These conditions test robustness to particular appearance changes, but do not sufficiently examine more diverse and stochastic shifts. Stronger randomized perturbations, such as varying illumination color, intensity and direction, camera viewpoint, object and background appearance, texture, occlusion, and distractor placement, would be needed to determine whether the observed gains generalize beyond the selected conditions.

  3. The core role of latent actions in the proposed method is not sufficiently demonstrated. In Iso-WAM, latent actions are primarily used as an auxiliary alignment target to improve robustness under the visual distribution shifts considered in the experiments. Although this design provides practical performance gains, the paper does not show that latent actions play a broader or more fundamental role in representing transferable action semantics or enabling capabilities beyond visual robustness. Consequently, the contribution currently appears to be mainly the use of latent actions as an additional supervision signal, and the broader methodological significance and distinctiveness of this design remain insufficiently established.

Questions for authors
  1. Could the authors clarify which element constitutes the primary methodological contribution of Iso-WAM and how it differs from prior WAM representation-alignment and latent-action methods? In particular, it would help to distinguish the contribution of transition-based action-stream supervision from that of Uni-LAM's semantic-geometric fusion and training objective.

  2. Could the authors clarify the visual augmentations used during policy training and whether Iso-WAM and Fast-WAM-Joint follow the same augmentation protocol? Since no augmentation-only control is reported, it is difficult to determine how much of the robustness improvement comes specifically from latent-action alignment. In addition, could the authors evaluate the method under stronger and more diverse randomized perturbations, such as independently varying lighting, camera viewpoint, background texture, object appearance, occlusion, and distractor configurations? Such an evaluation would better establish whether the reported robustness extends beyond the fixed lighting and texture shifts currently considered.

7 comments
AuthorsOpenReview ↗

Response to Reviewer YdYf (1/3)

Thank you for recognizing our systematic comparison of alignment targets and our evaluation across simulation and real-world robotic settings. We appreciate your constructive suggestions and address your concerns below.

1. Main methodological contribution

Our contribution consists of two connected aspects: first, systematically reexamining representation alignment in WAMs; and second, using these findings to develop unified transition supervision for action generation.

First, we systematically compare alignment targets within a common WAM framework, revealing the importance of both representation choice and how representations are combined. The comparison covers static semantic and geometric features, single-modality latent transitions, optical flow, dual-target transition alignment, and a unified transition target. The results reveal complementary representational priors: geometric targets perform better under camera viewpoint shifts, whereas semantic targets are more stable under noise. More importantly, two individually effective targets do not necessarily remain beneficial when used jointly. In Table 4, semantic and geometric latent actions achieve 74.3% and 73.0%, respectively, whereas direct dual-target supervision achieves only 70.0%. These findings show that representation alignment in WAMs depends not only on which information is introduced, but also on how that information is organized and provided to the action stream.

Second, motivated by these findings, we fuse semantic and geometric transitions in the teacher to construct a shared observation-transition target for the WAM action stream. Uni-LAM extracts a compact shared code from an observation pair. Conditioned on current-frame features, this code supports the reconstruction of both future semantic and geometric features. Policy queries then interact with action tokens and predict transition codes for the corresponding temporal intervals, connecting scene-change prediction with action generation. Uni-LAM thus determines how the supervision target is constructed, while query alignment in the action stream determines how that target contributes to action learning.

The key design is to integrate information across representations during target learning, rather than requiring the downstream policy to fit two independent latent spaces separately. In Table 4, the unified teacher target achieves 75.1%, improving over direct dual-target supervision by 5.1 percentage points and outperforming both single-modality variants. This shows that simply adding two sources of supervision is insufficient to realize their combined benefits; the organization of the supervision target itself has a substantial effect. Thus, compared with the static-feature alignment or separate-target designs considered in our study, our main methodological innovation is to learn a unified semantic-geometric transition code through conditional reconstruction and use it as shared supervision for the action stream. The systematic comparison motivates this design, and Uni-LAM turns these findings into a concrete method.

AuthorsOpenReview ↗

Response to Reviewer YdYf (2/3)

2. Does visual data augmentation introduce a confound, and how does the method perform under stronger randomized perturbations?

Data augmentation settings. We agree that image processing during teacher pretraining and policy training should be distinguished explicitly.

Iso-WAM and Fast-WAM-Joint use exactly the same policy image transformations: images are resized to 224×224, with no color jitter or random cropping during policy training. Therefore, the OOD improvement of Iso-WAM over Fast-WAM-Joint cannot be attributed to additional policy-side data augmentation. Random cropping and color jitter are used only during latent-action teacher training. The teachers used by DINO-LAM, DA3-LAM, and Dual, as well as Uni-LAM, follow the same augmentation settings. Consequently, performance differences among the transition targets are not caused by different augmentation configurations. We will document the settings for the two training stages separately in the revised manuscript to remove the ambiguity in the current description.

Stronger and more diverse randomized perturbations. Thank you for this question. We first clarify that our evaluation includes complementary settings: LIBERO-Plus examines seven categories of distribution shift, RoboTwin 2.0 clean-to-random tests combined environmental randomization, and real-robot experiments assess deployment under physical lighting and background-texture changes. Together, these results support the method's effectiveness across multiple distribution shifts rather than a single visual perturbation. To further address your concerns about perturbation strength and diversity, we conducted additional evaluations in the LIBERO simulator using four types of randomized perturbations—lighting, camera viewpoint, image noise, and occlusion—at greater intensities than in the previous experiments. We compared Iso-WAM with Fast-WAM-Joint under identical evaluation settings. Neither policy used data augmentation during training.

Randomized perturbationFast-WAM-Joint success (%)Iso-WAM success (%)Gain (pp)
Clean92.1875 (59/64)95.3125 (61/64)+3.1250
Lighting79.6875 (51/64)92.1875 (59/64)+12.5000
Camera viewpoint75.0000 (48/64)90.6250 (58/64)+15.6250
Image noise42.1875 (27/64)85.9375 (55/64)+43.7500
Occlusion92.1875 (59/64)93.7500 (60/64)+1.5625

In these additional evaluations with strong randomized perturbations, Iso-WAM outperforms Fast-WAM-Joint by 15 and 43 percentage points under camera viewpoint changes and image noise, respectively. These results further support the method's ability to improve action generation under substantial visual changes with identical policy image processing, extending the robustness benefits observed in the existing benchmarks to these additional test conditions.

AuthorsOpenReview ↗

Response to Reviewer YdYf (3/3)

3. The role of latent actions and the methodological significance of our work

We understand your concern: beyond improving performance, what methodological insight does using latent actions as supervision provide? The significance of our work lies in advancing WAM representation alignment from introducing additional visual features to systematically studying two design questions: what information should supervise action learning, and at which stage should complementary information be integrated?

Regarding supervision content, our work distinguishes scene descriptions from observation transitions. A representation that describes a future scene is not automatically the most suitable representation for supervising action generation. By comparing static features, latent transitions, and optical flow as targets, we investigate how this distinction affects action learning. After training on robot data and compression to 32 dimensions, the additional compact static target achieves 73.5% on LIBERO-Plus, below DINO-LAM's 74.3% and Uni-LAM's 75.1%. Thus, our findings go beyond showing that additional supervision helps: the information expressed by the supervision and the way it is constructed should be designed for action learning.

Regarding information integration, our work shows that complementary representations do not necessarily yield effective joint supervision. As discussed in the first section, directly aligning with individually effective semantic and geometric targets instead reduces performance. Uni-LAM therefore moves fusion into target learning on the teacher side, allowing multiple priors to form a shared transition representation before supervising the action stream. This provides a concrete design approach: information across representations can be integrated by learning a unified target, rather than leaving the downstream policy entirely responsible for reconciling multiple independent supervision spaces.

Using latent actions as supervision is therefore the implementation of this methodological design, rather than a reason to regard the contribution as merely combining existing components. The value of our work lies in identifying the importance of supervision content and information integration through systematic controls, and developing a unified transition target validated through policy experiments. We hope these explanations clarify the methodological significance of our work beyond simply adding an auxiliary loss.

Summary

Thank you for your constructive feedback, which has helped us clarify our methodological contributions and add more targeted experiments. Through these responses, we clarify the relationship between transition-based action-stream supervision and Uni-LAM's target construction, distinguish the augmentation settings used during teacher and policy training, and provide additional randomized-perturbation experiments supporting the method's robustness. We will incorporate these clarifications and experimental details into the revised manuscript to improve its clarity and completeness. We hope these responses address your main concerns, and we would be grateful if you would reconsider your rating in light of the additional evidence. We would be happy to provide any further clarification.

Reviewer_2YLbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Iso-WAM adds transition-prediction queries to a Fast-WAM policy with Wan2.2-derived video and action experts. A separate Uni-LAM teacher combines frozen semantic and geometric features into 32-dimensional transition targets, learned through future-feature reconstruction and motion supervision on four robot datasets. Policy training aligns queries with these frozen targets alongside video/action generation. Evaluation covers LIBERO, LIBERO-Plus, UR5 and RoboTwin.

Critical strengths
  1. The internal comparison improves LIBERO-Plus from 67.6% to 75.1%, while retaining 98.7% on LIBERO. The action-only variant also improves from 51.4% to 63.6% OOD (Tables 1–2), supporting the combined method's practical value.
  2. UR5 success improves substantially under the tested appearance shifts, including 45.0% to 82.5% under blue lighting and 32.5% to 75.0% on Cloth A (Table 3). The teacher's motion probes also outperform either single-modality variant under blue/dim inputs (Tables 7–8).
Critical weaknesses
  1. The methodological novelty appears limited. Representation alignment and learning latent actions from observation transitions are established ideas. Uni-LAM combines pretrained semantic/geometric features with query aggregation, a variational bottleneck and conditional reconstruction; the policy predicts these targets through queries and cosine alignment. The main advance is their particular integration into a shared transition target. Although useful, the paper does not make a convincing case that this integration offers a substantial methodological advance over existing alignment and latent-action approaches.
  2. The ablations leave the distinctive design's contribution uncertain. Static DINO alignment already reaches 73.3%; DINO-LAM reaches 74.3%, but also changes target compression and introduces robot-data teacher training. Uni-LAM adds 0.8 point over DINO-LAM, with some categories favoring a single modality (Table 4). The shared LAM training data and stronger motion probes are positive evidence, but do not separate the policy benefit of transition information from target construction, or establish where geometric fusion matters most.
Questions for authors
  1. What distinguishes the proposed target construction and policy alignment from existing representation-alignment and latent-action approaches? Which design provides the main methodological advance?
  2. Can existing controls separate transition information from compression and teacher pretraining in the 73.3% to 74.3% improvement? A compact static target with comparable training would help.
  3. Where does geometry help beyond DINO-LAM? Can task-level results or existing probes explain the additional 0.8 point and the categories where fusion underperforms a single modality?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 2YLb (1/2)

Thank you for recognizing the practical value of our approach and the real-world experiments, and for your careful analysis of the method and experimental attribution. Below, we clarify the core contribution, the compact static-target control, and the specific benefits of geometric information.

1. Distinction from existing methods and the main methodological advance

Iso-WAM reexamines representation alignment in WAMs and develops a method based on the resulting findings. Our starting point is that, although prior work has shown that visual representation supervision can improve action learning, which information to align with and how to integrate information from different representations still warrant systematic comparison within a common framework. Our contribution therefore consists of two connected aspects.

First, within the same WAM framework, we systematically compare static semantic and geometric features, single-modality latent transitions, optical flow, dual-target transition alignment, and a unified transition target. Beyond comparing final performance, this study reveals two important trade-offs in target selection. Different representational priors have different strengths: geometric targets perform better under camera viewpoint shifts, whereas semantic targets are more stable under noise. Moreover, two targets that individually improve a policy do not necessarily remain beneficial when used jointly. In Table 4, semantic and geometric latent actions achieve 74.3% and 73.0%, respectively, whereas direct dual-target supervision reduces performance to 70.0%.

Second, motivated by these observations, we learn a shared dynamic representation in the teacher and use it as a unified alignment target for the action stream. Different representation spaces organize information differently; aligning with two independent targets requires the policy to accommodate both organizations simultaneously. Uni-LAM moves this integration into teacher learning: a single compact code supports the reconstruction of both future semantic and geometric features, conditioned on current-observation features. The two sources of information thus form a shared transition representation before policy training. The policy predicts one unified target rather than separately fitting two independent latent spaces. This design achieves 75.1% on LIBERO-Plus, outperforming both the dual-target variant and the two single-modality variants.

Our methodological contribution is therefore to systematically identify the complementary strengths of WAM alignment targets and the difficulties of joint supervision, and use these findings to construct a shared transition target that brings multiple visual priors into action generation through unified supervision.

2. Does the static-feature versus transition-target comparison separate the effects of compression and teacher training?

Thank you for identifying these confounding factors. The original static DINO features and DINO-LAM differ not only in whether they model transitions, but also in target dimensionality and teacher training. To address this question more directly, we introduce a compact static teacher, DINO-Static-AE. It keeps the DINO encoder frozen and adds a small projector that compresses single-frame features into a 32-dimensional code, from which a decoder reconstructs the features of that frame. We train this teacher on the same robot data as DINO-LAM, matching data augmentation, training budget, and model capacity. After teacher training, we freeze its parameters and use the resulting 32-dimensional static code as the supervision target, training the WAM through the same policy-alignment interface as DINO-LAM. This control gives the static target both robot-data training and low-dimensional compression while retaining its single-frame representational objective.

Alignment targetCode dimensionTeacher trained on robot dataLIBERO (%)LIBERO-Plus (%)
Original static DINO featuresOriginalNo98.473.3
DINO-Static-AE32Yes98.073.5
Full DINO-LAM32Yes98.774.3
Uni-LAM32Yes98.775.1

Even after training on the same data and low-dimensional compression, the compact static target remains below the models aligned with dynamic targets. This indicates that, in this setting, the gains are not merely attributable to compression and robot data; explicitly modeling observation transitions provides additional value.

AuthorsOpenReview ↗

Response to Reviewer 2YLb (2/2)

3. Where does geometry help, and why is the unified target not best in every category?

We agree that the benefits of the unified target should be interpreted by condition rather than solely through the overall average. In Table 4, Uni-LAM improves over DINO-LAM by 6.0, 4.0, and 1.0 percentage points under camera viewpoint, noise, and lighting shifts, respectively. It trails DINO-LAM by 0.5, 1.3, 2.2, and 1.6 points under robot initial-state, language, background, and layout shifts, respectively.

The clearest complementarity appears between camera viewpoint and noise conditions. DA3-LAM achieves 49.2% under camera viewpoint shifts, compared with 41.5% for DINO-LAM, but only 58.7% under noise, compared with 72.5% for DINO-LAM. Uni-LAM achieves 47.5% and 76.5%, respectively, retaining strong viewpoint robustness while avoiding the pronounced degradation of the geometry-only target under noise. For categories such as background, language, and layout, we acknowledge that the unified target does not consistently outperform the semantic target. A shared code must support the prediction of multiple types of information, which may introduce trade-offs in information retention and optimization.

To further examine whether the unified representation preserves control-relevant spatial information, we conduct an additional controlled spatial-readout experiment. We hold the physical states of the robot and objects fixed and vary only the camera viewpoint or add image noise, comparing how accurately different teacher codes predict the object–gripper relative position. All probes are trained only on clean observations from the original viewpoint and evaluated on held-out layouts and perturbed observations. Paired proprioception in the table refers to the robot’s proprioceptive states at the current and future endpoints. The experiment uses 360 observation pairs from 30 tasks, with 126 pairs for training, 54 for validation, and 180 for testing. The following results report 3D relative-position error in millimeters; lower is better.

Probe inputOriginal viewCamera −15°Camera +15°Noise σ=0.04
DINO-LAM + paired proprioception176.19182.34187.29181.29
DA3-LAM + paired proprioception164.19165.12166.06174.71
Uni-LAM + paired proprioception167.87169.07170.41171.86

These results further reveal the complementary strengths of semantic and geometric transition representations. DA3-LAM achieves the lowest spatial-readout error in the original view and both shifted views, demonstrating the advantage of the geometric target in this spatial-prediction task. Uni-LAM achieves errors of 169.07 and 170.41 mm in the two shifted views, below DINO-LAM's 182.34 and 187.29 mm and close to DA3-LAM's performance. Under noise, DA3-LAM's error increases by 10.52 mm, whereas Uni-LAM's increases by only 3.99 mm, yielding the lowest absolute error among the three representations. This indicates that the unified representation mitigates the noise sensitivity of the geometry-only representation while maintaining good spatial-readout accuracy and viewpoint stability. The experiment therefore supports a better balance between spatial information and stability under appearance perturbations, rather than uniform superiority over single-modality targets in every condition. This finding is consistent with the trade-off observed at the policy level under camera viewpoint shifts and noise.

Summary

We sincerely appreciate your constructive feedback. Your suggestions have helped us distinguish more clearly between the effectiveness of the complete method, the contributions of individual design choices, and the conclusions supported by the available evidence. Building on this discussion, we will incorporate a clearer contribution statement, the compact static-target control, and the controlled spatial-readout analysis into the revised manuscript to improve its clarity and completeness. We hope these responses address your main concerns, and we would be grateful if you would reconsider your rating in light of the additional evidence. We would be happy to provide any further clarification.

AuthorsOpenReview ↗

Follow-up to Reviewer 2YLb

Dear Reviewer 2YLb,

Thank you again for your constructive review. We understand this is a busy period. As the discussion phase is drawing to a close, we wanted to check whether our responses address your three questions:

Q1: Clarified the contribution of the shared semantic–geometric transition target and its distinction from static alignment and dual-target supervision.

Q2: Added DINO-Static-AE to examine transition-target gains with matched compression dimensionality and teacher training.

Q3: Added condition-wise analysis and spatial-readout experiments to examine where geometry helps and the trade-offs of fusion.

If anything remains unclear, we would be glad to clarify or run further checks in the remaining time. Thank you for your time and consideration.

Reviewer_kvopOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Iso-WAM enhances the visual robustness of World-Action Models by aligning action-sequence queries with latent observation transitions. Its Uni-LAM teacher model integrates semantic and geometric information into a shared representation. Experiments demonstrate that this approach yields performance improvements on the LIBERO-Plus benchmark, as well as in real-world tasks involving variations in lighting and background.

Critical strengths

The method addresses a practical problem, and the overall robustness improvements over Fast-WAM are substantial.

The experiments compare several representation choices and include both simulation and real-world evaluation.

The appendix provides useful motion probes and discusses limitations, supporting the value of the learned representation.

Critical weaknesses

The additional gain over the semantic-only DINO-LAM is modest: 75.1% versus 74.3% on LIBERO-Plus. Reporting variation across runs would make this comparison more convincing.

The claim of disentangling physical dynamics from appearance is stronger than the evidence supports. The results more directly demonstrate improved robustness under the tested visual shifts.

Questions for authors

Are the two reported seeds independent training runs or evaluation seeds? How consistent is the improvement over DINO-LAM?

2 comments
AuthorsOpenReview ↗

Response to Reviewer kvop (1/2)

Thank you for recognizing the improvements in practical robustness and the value of the learned representations. We sincerely appreciate your constructive suggestions and address your concerns below. We clarify the evaluation-seed protocol and present an additional controlled experiment that directly measures the effect of appearance changes on action generation.

1. Evaluation seeds and the improvement over DINO-LAM.

The two LIBERO-Plus seeds reported in the paper are evaluation seeds applied to the same trained policy, with 10,030 episodes per seed, rather than two independent training runs. We will make this explicit in Section 4.1.

The average difference of 0.8 percentage points reflects a condition-dependent performance trade-off. The detailed results from Table 4 are as follows:

LIBERO-Plus categoryDINO-LAM (%)Uni-LAM (%)Difference (pp)
Camera viewpoint41.547.5+6.0
Robot initial state71.771.2-0.5
Language94.893.5-1.3
Lighting95.396.3+1.0
Background65.162.9-2.2
Noise72.576.5+4.0
Layout79.477.8-1.6
Reported overall average74.375.1+0.8

Uni-LAM integrates information from semantic and geometric foundation models within a shared transition space, enabling stronger camera-viewpoint robustness while maintaining robustness to appearance perturbations. Compared with DINO-LAM, which uses only semantic transition information from DINOv3, Uni-LAM improves performance under camera viewpoint shifts by 6.0 percentage points and achieves broadly comparable performance under lighting, noise, and background changes, while substantially outperforming the Fast-WAM-Joint baseline in all these conditions. The unified target therefore combines the complementary strengths of both sources: improving viewpoint robustness while preserving the benefits of semantic representations under appearance changes, ultimately achieving a higher overall success rate (75.1% vs. 74.3%).

To further assess the consistency of the improvement over DINO-LAM, we added a third evaluation seed to the two reported in the original submission. For each method, we kept the trained checkpoint fixed across all three evaluations. The LIBERO-Plus results are shown below:

Evaluation seedDINO-LAM (%)Uni-LAM (%)Improvement (pp)
Seed 174.074.9+0.9
Seed 274.675.3+0.7
Seed 374.175.4+1.3
Mean ± sample standard deviation74.23 ± 0.3275.20 ± 0.26+0.97

Uni-LAM outperforms DINO-LAM under all three evaluation seeds, with improvements ranging from 0.7 to 1.3 percentage points and an average improvement of 0.97 points. These results demonstrate a consistent advantage across the tested evaluation seeds.

We conducted an additional paired closed-loop evaluation on eight LIBERO tasks, using two initial states and four action-sampling seeds per task, for a total of 64 episodes per condition. The results are as follows:

Randomized perturbationBaseline success (%)DINO-LAM success (%)Uni-LAM success (%)
Clean92.1875 (59/64)95.3125 (61/64)95.3125 (61/64)
Lighting79.6875 (51/64)93.7500 (60/64)92.1875 (59/64)
Camera viewpoint75.0000 (48/64)79.6875 (51/64)90.6250 (58/64)
Image noise42.1875 (27/64)87.5000 (56/64)85.9375 (55/64)
Occlusion92.1875 (59/64)93.7500 (60/64)93.7500 (60/64)

Compared with DINO-LAM, Uni-LAM improves success under camera viewpoint shifts by 10.9375 percentage points, while recording only one fewer success under each of the lighting and noise conditions and matching performance under clean and occlusion conditions. These results indicate that Uni-LAM improves viewpoint robustness while retaining appearance-perturbation robustness close to that of the semantic target, consistent with our motivation for incorporating geometric information. We hope these responses address your concern.

AuthorsOpenReview ↗

Response to Reviewer kvop (2/2)

2. Additional controlled experiment: the effect of appearance changes on action generation.

To further examine how appearance factors affect action generation, we conducted a controlled experiment: we fixed the physical states of the robot and objects, the task instruction, and the action-sampling seed, and changed only image brightness or noise to compare predicted actions under clean and perturbed inputs.

The experiment covers 24 shared states from eight LIBERO tasks, with two action-sampling seeds per state. We compare the first five steps of each predicted action chunk, matching the replanning interval used during execution. The table below reports the mean change in translation commands, measured in the controller's native units after reversing dataset normalization. Smaller values indicate less influence of appearance perturbations on the action output.

Appearance changeFast-WAM-JointIso-WAM
Brightness ×0.750.03970.0144
Brightness ×0.550.06510.0347
Gaussian noise σ=0.020.17200.0340
Gaussian noise σ=0.040.29490.0528

Iso-WAM exhibits smaller action changes under all four conditions. Under noise with σ=0.04, the translation-command change decreases from 0.2949 for the baseline to 0.0528 for Iso-WAM, a reduction of approximately 82.1%.

Compared with closed-loop success rates, this experiment directly examines the effect of appearance changes on action outputs: with the physical state and task held fixed, Iso-WAM's action generation is less affected by image perturbations. This provides complementary action-level evidence supporting our design motivation of reducing the influence of task-irrelevant appearance on control. At the same time, we agree that the available evidence supports reduced action sensitivity to the tested appearance perturbations, rather than establishing full disentanglement of physical dynamics and appearance. We will revise the corresponding claims in the manuscript accordingly.

Summary

These responses clarify the meaning of the evaluation seeds and, through paired closed-loop evaluations and action-sensitivity experiments with fixed physical states, further illustrate Uni-LAM's performance trade-offs and Iso-WAM's stability under appearance perturbations. We will incorporate these results and experimental details into the revised manuscript to improve its clarity and completeness. We hope these responses address your main concerns, and we would be grateful if you would reconsider your rating in light of the additional evidence. We would be happy to provide any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.