- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
WALA learns latent actions from semantic and depth changes, then uses both encoder-derived distillation targets and decoder-based future prediction to supervise the policy tokens that directly generate robot actions. Robot action labels supervise labeled examples, while action-free videos provide auxiliary supervision. The LAM and future observations are unnecessary at inference. Evaluation covers RoboTwin, RoboCasa, and real-robot manipulation, including human-video scaling and target-task zero-shot/few-shot transfer. The central contribution is the combination of semantic-geometric latent learning and complementary supervision of the action-generating representation.
Critical strengths
-
The controlled policy ablations provide convincing evidence of practical benefit. With the same policy backbone and robot data, WALA improves RoboCasa success from 54.2% to 75.2% (Table 3), exceeding world-model supervision alone (71.0%) and distillation alone (67.6%). This supports the usefulness of combining the two signals, rather than relying only on comparisons between independently developed systems. The fixed-label experiment also improves success from 53.9% to 67.8% as additional action-free robot videos are included during policy training (Figure 3).
-
The real-robot experiments directly test the intended use of human videos. At 200 robot demonstrations per base task, adding 400 human videos per task raises average success from 75.0% to 83.3% (Table 4). With no target-task robot demonstrations, adding 150 target-task human videos improves bread pick-place from 9/30 to 24/30 and cabinet opening from 4/30 to 15/30 (Table 5). These are meaningful transfer results under the stated setting of a policy also trained on related robot tasks. The inference-time removal of auxiliary prediction machinery makes the approach practically attractive.
Critical weaknesses
-
The evidence isolates the overall training recipe more clearly than its proposed representation-level explanation. There is no matched learned-LAM ablation replacing semantic/depth deltas with absolute features while holding architecture and pretraining fixed. Table 13 compares raw visual representations with linear probes, which does not establish the effect of delta modeling on the learned policy. Likewise, the 2.2-point geometry comparison uses separately pretrained LAMs with different architectures (Appendix A.3). Attention maps and retrieval examples are suggestive, but do not resolve these attribution questions. This limits how strongly the paper establishes why its particular representation is preferable.
-
The main policy results lack reported variability across independent training runs; the three-seed representation probes do not address this uncertainty. This matters especially for the 2.2-point geometry gain, the 1.9-point readout comparison, and the 0.09-point RoboTwin lead of WALA-Wan over LingBot-VA. Moreover, WALA's 25,224-hour LAM pretraining is not matched across competing methods. Appendix A.3 usefully matches target-task training/evaluation budgets for Moto and UniVLA, but does not establish matched pretraining or backbone capacity. Thus, the strongest evidence is the internal ablation and video-scaling evidence, rather than a clean attribution of leaderboard differences to the proposed method.
Overall, I lean toward weak acceptance: the substantial controlled improvements and real human-video transfer outweigh these limitations, although the mechanistic and fine-grained comparative claims need more cautious support.
Questions for authors
-
Representation attribution: Is there a matched experiment comparing learned delta-based and absolute-feature LAMs with the same architecture, data, and optimization? Table 13 does not appear to be such a control. Please also clarify how much of the semantic-only versus semantic-geometric comparison could be attributable to architecture differences. Existing controlled evidence would strengthen the representation claim; otherwise, please narrow the causal interpretation.
-
Robustness of the reported differences: How many independently trained policies underlie the main tables, and what uncertainty can you report for the key Table 3 comparisons and the smaller readout/leaderboard differences? Please distinguish rollout variability from training-seed variability. This would affect whether the smaller gains support component-level conclusions, even though the larger overall gains are persuasive.
3 comments
Response to Reviewer iPpk (2/2)
W2 & Q2: Repeated Runs and Method Comparisons
Variation across downstream training runs. The baseline results cited in our main benchmark tables are generally published as single success rates without standard deviations across independent training runs. We follow this format. WALA's original results use one training run and one evaluation per configuration.
We have now repeated downstream training for the semantic-only and semantic–geometric world-model-supervision variants in Table 3. For each configuration, we reuse its existing pretrained LAM checkpoint and train two additional policies with seeds 43 and 44. Together with the original seed-42 run, this gives three independently trained policies evaluated using the same benchmark protocol. The table reports the mean and sample standard deviation of their success rates.
| Configuration | Success rate (%, mean ± SD) |
|---|---|
| Semantic only | 68.9 ± 0.2 |
| Semantic + geometry | 71.4 ± 0.3 |
Adding the geometric branch improves performance with every training seed and yields an average gain of 2.5 percentage points. These runs assess downstream training variability with the pretrained LAM held fixed within each configuration.
Action readout. This experiment tests whether a small set of latent-action tokens provides sufficient information for control. Matching the performance of a head that reads all backbone tokens, or achieving a modest improvement, already supports this design objective. Latent-only readout achieves 75.2%, compared with 73.3% using all backbone tokens (Appendix B.5). The attention analysis in the same appendix shows that, even when all backbone tokens are available, the action head primarily attends to latent-action tokens. The performance and attention results thus support using these compact tokens as the direct input to the action head.
Performance across benchmarks. A central goal of WALA is to learn transferable interaction knowledge from videos without action annotations and use it to improve robot control. LingBot-VA benefits from large-scale action-labeled pretraining and explicitly predicts future video states in latent space, using inverse dynamics to generate actions conditioned on those predictions. In contrast, WALA pretrains its LAM without action annotations and requires only the policy and action head at inference. Without inference-time future-video generation, WALA-Wan nevertheless achieves 92.33% on RoboTwin, compared with LingBot-VA's 92.24% (Table 1), matching this strong baseline with a slightly higher reported success rate.
To extend this comparison beyond RoboTwin, we trained LingBot-VA on the full RoboCasa training data, obtaining 56.8%, compared with WALA's 75.2% reported in Table 2, a difference of 18.4 percentage points. The table also reports efficiency measured on the same H100 platform at a per-GPU batch size of 1, using each method's own training and inference configuration.
| Method | RoboCasa success (%) | Training (s/step) | Training memory (GB) | Inference (s/call) |
|---|---|---|---|---|
| LingBot-VA | 56.8 | 12.994 | 44.01 | 5.068 |
| WALA | 75.2 | 0.638 | 23.69 | 0.070 |
WALA uses 46.2% less training memory, with 20.4× faster training steps and 72.4× faster inference calls. These results show near-equal RoboTwin success, higher RoboCasa success, and lower measured training and inference costs.
Pretraining data and backbone capacity. As described in Appendix A.3, our original Moto and UniVLA comparisons match downstream training and evaluation budgets while using each method's own pretrained model and backbone. Our new comparison fixes the policy architecture and training data to address these differences.
We implement UniVLA-style, villa-X-style, and VLA-JEPA-style within WALA using the same Qwen3-VL-4B backbone, action head, 10% RoboCasa episode subset, and downstream training and evaluation settings. All methods, including WALA, use this same subset for both video pretraining and policy training. Each style follows the corresponding method's pretraining procedure and supervision design within the shared framework.
| Method | Success (%) |
|---|---|
| VLA-JEPA-style | 43.1 |
| villa-X-style | 45.8 |
| UniVLA-style | 46.4 |
| WALA | 51.3 |
WALA improves over VLA-JEPA-style, villa-X-style, and UniVLA-style by 8.2, 5.5, and 4.9 percentage points, respectively. These results demonstrate an advantage for WALA's representation and supervision design under matched data and policy architecture, complementing the comparisons between the original methods.
We will incorporate these additional experiments and discussions into the revised manuscript. We hope that our responses address all of your concerns. Please let us know if you have any further questions or would like clarification on any point. We would be happy to provide further details.
Looking forward to further discussion
Dear Reviewer iPpk,
Thank you again for your thoughtful comments on representation design and experimental reliability. We would appreciate your feedback on the matched delta-versus-absolute comparison and repeated policy-training results provided in our response. We hope these additions address your concerns and would welcome the opportunity to clarify any remaining questions.
Thank you for your time and continued consideration.
Best regards,
Authors
Response to Reviewer iPpk (1/2)
Thank you for carefully reading our paper and providing constructive suggestions on representation design and experimental reliability. We appreciate the time and attention you have given to our experiments and their interpretation. Below, we clarify the comparisons and report additional experiments addressing your concerns.
W1 & Q1: Delta Modeling and Geometric Information
Delta versus absolute representations. We have added the requested comparison between two learned LAMs. Both use the same architecture, pretraining data, future-frame sampling, and optimization settings. The difference is whether the LAM encodes and predicts changes relative to the current observation or absolute features. We then train the corresponding policies with the same backbone, action head, robot data, and downstream training and evaluation settings.
Both stages use the same 10% RoboCasa episode subset. The results are summarized below.
Delta improves success by 3.5 percentage points. This experiment evaluates delta modeling through the performance of policies trained with the learned LAMs. It therefore adds direct downstream evidence to the raw-feature comparison in Table 13 of the original manuscript.
Adding geometric information. The architecture in Section 3.2 explains how geometric information enters the LAM. The semantic-only model encodes and predicts changes in DINOv3 features. Building on this design, the semantic–geometric model adds a depth-encoding branch and a corresponding depth-prediction branch. The encoder combines information from the semantic and geometric branches into shared latent-action tokens, and the decoder uses those tokens to predict both semantic and depth changes.
The depth branches provide the pathway through which geometric information enters the representation and its training objectives. The architectural difference between these variants is therefore part of introducing geometric information into our LAM. With the policy backbone and action head unchanged, the ablation evaluates the benefit of adding this geometric pathway and its depth supervision to semantic latent-action learning.
As described in Appendix A.3, the two variants have separately pretrained LAM checkpoints. We use the existing checkpoint for each configuration in the additional downstream experiments. The original comparison in Table 3 gives 68.8% for semantic-only prediction and 71.0% for the semantic–geometric world model. To assess whether this improvement is consistent across policy-training runs, we have now repeated downstream training for both configurations, as detailed below.