- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates state aliasing in VLA visual representations, where fine-grained state differences may not be sufficiently preserved for accurate robot control. The authors analyze the relationship between state-difference prediction and action prediction across pretrained visual encoders, and propose an auxiliary inverse dynamics loss to improve state-sensitive representations. Experiments on LIBERO, CALVIN, and SimplerEnv demonstrate improvements across several VLA architectures.
Critical strengths
-
The paper identifies an interesting and important problem: whether pretrained VLM representations preserve sufficiently fine-grained information for robot control. The motivation is clear and worth investigating.
-
The proposed method is simple and practical, requiring no additional annotations or inference-time overhead. The substantial improvement on LIBERO is encouraging, and the representation-level analyses provide additional supporting evidence.
Critical weaknesses
-
Limited probing methodology. Using a simple MLP on frozen feature differences may underestimate Qwen3-VL's capabilities. The authors should evaluate stronger probes, including the full VLM with its LLM decoder, before concluding that the visual encoder lacks fine-grained state information.
-
Weak baseline. The LIBERO baseline achieves only 53.9%, far below recent VLAs such as StarVLA, VLAct, and is trained for only one epoch per stage. The authors should validate their method on a stronger, well-trained baseline to demonstrate its effectiveness.
Questions for authors
-
Have the authors considered evaluating state discrimination using the full Qwen3-VL model with its LLM decoder? Could the relatively poor probing performance reflect limitations of the probe architecture rather than missing information in the visual representation? I believe VLM models can show the difference between the two images.
-
Why is the LIBERO baseline substantially weaker than recent VLA methods? Can the authors provide training curves and evaluate the proposed method on a stronger baseline, such as StarVLA, VLAct, under matched training conditions?
-
Why should inverse dynamics supervision be particularly effective at reducing state aliasing? How does it compare with direct state-difference prediction or other auxiliary representation-learning objectives?
6 comments
Additional results: inverse dynamics vs. state-difference supervision
Q3. Why inverse dynamics and comparison with alternative objectives
Thank you for this suggestion. You ask, “How does it compare with direct state-difference prediction or other auxiliary representation-learning objectives?” Direct state-difference supervision yields a lower state-difference probe loss, while inverse dynamics achieves better policy performance.
On CALVIN ABC→D, we compare vanilla FLOWER with three auxiliary objectives: behavior cloning (BC)—actions from current visual features; state-difference prediction (SD)—state differences from current and future visual features; and Ours—intervening actions from current and future visual features.
| Model | 1 task ↑ | 2 tasks ↑ | 3 tasks ↑ | 4 tasks ↑ | 5 tasks ↑ | Avg. successful sequence length ↑ |
|---|---|---|---|---|---|---|
| Vanilla | 96.9% | 91.4% | 85.5% | 78.9% | 73.1% | 4.258 |
| +BC | 96.6% (−0.3) | 91.9% (+0.5) | 86.2% (+0.7) | 80.0% (+1.1) | 73.0% (−0.1) | 4.277 (+0.019) |
| +SD | 99.2% (+2.3) | 94.9% (+3.5) | 88.0% (+2.5) | 80.8% (+1.9) | 73.2% (+0.1) | 4.361 (+0.103) |
| +Ours | 99.5% (+2.6) | 96.6% (+5.2) | 91.2% (+5.7) | 86.9% (+8.0) | 81.6% (+8.5) | 4.558 (+0.300) |
We also freeze the trained vision encoders and apply the same state-difference probe to features from the CALVIN validation set.
| Model | State-difference probe loss ↓ |
|---|---|
| Vanilla | 0.033 |
| +BC | 0.024 |
| +SD | 0.007 |
| +Ours | 0.021 |
SD yields a lower probe loss, but Ours achieves better policy performance. One possible explanation is that predicting intervening actions encourages sensitivity to state changes useful for the policy's own action prediction, beyond supervising state differences alone.
Dear Reviewer KayM,
With one day remaining in the discussion period, we would appreciate your feedback on our responses above, including the probing of representations after the LLM decoder (W1/Q1), the stronger FLOWER baseline on LIBERO-10 (W2/Q2), and the comparison with direct state-difference prediction and auxiliary BC (Q3), as well as our clarification of the training curves (W2/Q2). Could you let us know which concerns, if any, remain unresolved? We would be happy to provide further clarification.
Thank you for your time and consideration.
Response to Reviewer KayM
Thank you for your thoughtful comments and suggestions. We provide clarifications and additional results addressing your concerns below.
W1 / Q1. Probe limitations and full-VLM evaluation
You ask, “Have the authors considered evaluating state discrimination using the full Qwen3-VL model with its LLM decoder?” Following your suggestion, we have now extended our state-difference analysis to representations obtained after the LLM decoder. We appreciate this suggestion. All results below use Qwen3-VL 2B, with the vision encoder and LLM frozen and only the probe trained.
We compare vision-encoder features with image-token hidden states after the LLM decoder. We additionally present both images in temporal order within a single prompt, asking “What is the difference in the robot's state between these two images?”, and probe an equal number of final-token hidden states.
Vision-encoder features yield the lowest reconstruction loss in this comparison. Under our probing protocol, neither LLM-processed image-token features nor the joint-prompt representations improve the recovery of fine-grained robot-state differences over vision-encoder features.
W2 / Q2. Baseline strength and training curves
You ask, “Why is the LIBERO baseline substantially weaker than recent VLA methods?” Our baseline is VLM4VLA, a VLA built with minimal changes to a pretrained VLM, trained jointly on all four LIBERO suites, a more demanding setting than single-suite training.
You also request “training curves” and evaluation on “a stronger baseline” under matched training conditions. We provide the training curves in Appendix D.3, Figure 3, and a stronger-baseline comparison below. The curves show that the decline in loss had substantially slowed by the end of training.
We evaluate FLOWER [1] on LIBERO-10, following the original paper’s LIBERO training and evaluation setup and comparing vanilla and +Ours under the same conditions.
The improvement from 88.4% to 94.4% shows that our method remains effective with a stronger baseline.
The experiments for your remaining points are still running; we will post the results in a follow-up comment as soon as they are ready.
[1] FLOWER: Democratizing Generalist Robot Policies with Efficient Vision-Language-Action Flow Policies, CoRL 2025.