- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces "Retrieve to Act" (R2A), a method designed to enhance the robustness of Vision-Language-Action (VLA) models under OOD execution conditions. The authors organize successful demonstration segments into a spatiotemporal graph and retrieves suitable continuations using the robot's current predicted actions and recent motion history. It keeps the original VLA parameters fixed, corrects action outputs through a small fusion module, and adjusts the robot's initial posture before execution to improve performance under perturbations. Experiments across several models and benchmarks evaluate the resulting gains.
Critical strengths
- The problem is practically important and the method is clearly specified. By combining motion history, graph-based retrieval, and action blending, the paper offers a workable way to exploit demonstration experience at inference time.
- The experiments encompass various backbones, simulation platforms, and real-world robot tasks, demonstrating the effectiveness of the method.
Critical weaknesses
- On RoboTwin, R2A also changes the control frequency and action-processing rules: Appendix C.1, the baseline executes full 50-step chunks, while R2A requeries the policy every 10 steps after the gripper closes, and adds displacement compensation triggered by stagnation detection. Since chunk size is known to materially affect policy performance [1][2], such discrepancies inevitably lead to unfair comparisons.
- The contributions of initial-state correction and trajectory retrieval are not disentangled. Tab.1 and 5 suggest that most of the gain comes from initial-state correction alone: removing it drops the success rate from 87.5% to 49.9%. Without a "base VLA + identical initial-state correction" baseline, the independent contribution of trajectory retrieval cannot be isolated. The method should also be compared against simpler recovery strategies (a fixed initial state, or the initial state of the nearest demonstration) to establish that diffusion-based correction is necessary.
- The method depends on motions covered by existing demonstrations and lacks explicit scene geometry, limiting its handling of new obstacles or recovery requiring novel action sequences. Real-robot tests cover only two tasks and initial-state perturbations; broader real-world evaluation and cost analysis with larger demonstration libraries remain useful directions.
- There are unexplained discrepancies in configurations and results between the main text and the appendices (for instance, regarding connection configurations, training workflows, and success rates for RoboTwin/LIBERO-LONG) which require careful rectification.
[1] PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking arXiv:2606.00537
[2] Mixture of Horizons in Action Chunking arXiv.2511.19433
Questions for authors
As noted above, the presence of several inconsistencies and some unfair comparisons within the paper directly affects my overall assessment. The value of the work could be enhanced if the author were to provide clarifications, make necessary corrections, and conduct more objective ablation studies.
7 comments
Response to Reviewer uKCE (2/2)
W3. Dependence on demonstrated motions, breadth of the real-world evaluation, and cost
We agree that R2A can retrieve only motions represented in the demonstration library and does not use explicit scene geometry. Our experiments do not establish that it can handle novel obstacles; we now state this limitation in Section 6. Our target setting is robustness to contextual perturbations while the task remains unchanged. In this setting, VLS, which builds reward functions from scene geometry, improves neither base policy, whereas R2A improves both.
R2A and VLS, which uses scene geometry, on LIBERO-10 (success rate, %).
| Method | Camera | Robot init | Language | Light | Background | Noise | Layout | Avg. |
|---|---|---|---|---|---|---|---|---|
| π₀.₅ | 50.6 | 73.3 | 88.8 | 95.6 | 93.4 | 75.5 | 89.1 | 80.9 |
| + VLS | 45.7 | 44.3 | 79.4 | 80.3 | 72.3 | 56.8 | 75.6 | 64.9 |
| + R2A | 55.1 | 96.9 | 91.9 | 95.6 | 95.2 | 78.0 | 91.3 | 86.3 |
| GR00T N1.7 | 38.2 | 32.6 | 77.3 | 84.2 | 82.4 | 55.9 | 69.6 | 62.9 |
| + VLS | 38.1 | 30.9 | 66.8 | 77.9 | 73.2 | 50.2 | 64.1 | 57.3 |
| + R2A | 58.9 | 87.5 | 84.6 | 94.5 | 88.9 | 78.6 | 81.4 | 82.1 |
Hardware constraints limited the breadth of our real-world evaluation, so we prioritized perturbations that most strongly degrade the base policies in simulation. We also added a camera-viewpoint perturbation for Banana2Pot and evaluated a second platform, SO-101, with π₀.₅.
Real-world success rates (%), 20 trials per entry.
| Platform, task | Policy | Condition | Base | + R2A |
|---|---|---|---|---|
| ALOHA, Banana2Pot | GR00T N1.7 | No perturbation | 60 | 80 |
| ALOHA, Banana2Pot | GR00T N1.7 | Robot initial state | 30 | 80 |
| ALOHA, Banana2Pot | GR00T N1.7 | Camera viewpoint | 10 | 60 |
| ALOHA, Cube2Drawer | GR00T N1.7 | No perturbation | 55 | 85 |
| ALOHA, Cube2Drawer | GR00T N1.7 | Robot initial state | 10 | 45 |
| SO-101, Drawer Stack | π₀.₅ | No perturbation | 65 | 80 |
| SO-101, Drawer Stack | π₀.₅ | Robot initial state | 25 | 80 |
To assess computational cost, we measured storage requirements and query overhead as the demonstration library grows.
Cost of the motion primitive graph as the demonstration library grows.
| Library | Frames | Nodes | Temporal edges | Spatial edges | Storage (MB) | Overhead per query (ms) |
|---|---|---|---|---|---|---|
| LIBERO-10 | 138,090 | 28,304 | 252,040 | 516,536 | 19.5 | 16.0 |
| LIBERO Goal | 63,728 | 13,235 | 101,350 | 242,128 | 9.0 | 12.3 |
| LIBERO Object | 74,507 | 15,444 | 123,440 | 274,320 | 10.4 | 12.8 |
| LIBERO Spatial | 62,250 | 12,948 | 98,480 | 226,378 | 8.8 | 12.1 |
| LIBERO (four suites) | 338,575 | 69,931 | 575,310 | 1,221,710 | 47.5 | 25.2 |
| RoboTwin 2.0 | 552,287 | 88,964 | 655,344 | 1,540,352 | 72.0 | 38.8 |
These results are reported in Section 6 and Table 3, with the camera-viewpoint and SO-101 experiments in Appendix B and the cost analysis in Appendix D.6 (Table 19).
W4. Discrepancies between the main text and the appendices
We reviewed the manuscript for consistency in three areas. Method: the temporal-edge description is now consistent between the main text and appendix (each node connects to nodes steps before and after it in the same demonstration), and the ISR inference procedure matches the implementation. Settings: all simulation experiments use the released checkpoints in Table 6 without fine-tuning. The real-world policies are fine-tuned on task demonstrations, and R2A is applied to the same fine-tuned policy as the baseline. Numbers: we replaced outdated results in the introduction and checked the remaining values against the tables. These revisions are reflected in Section 1, Sections 3.3 and 3.4, and Appendix C.1.
We hope these revisions address your concerns.
Follow-up to Reviewer uKCE
Dear Reviewer uKCE,
Thank you again for your detailed review. As the discussion period closes soon, we would like to briefly check whether our response addresses your two main concerns.
(1) RoboTwin comparison (W1): with one execution schedule for both methods, R2A improves π₀.₅ from 56.8% to 65.5% with 50-step chunks and from 59.3% to 67.1% with 10-step chunks (Tables 2 and 17).
(2) ISR vs. retrieval (W2): with ISR removed, retrieval and harmonization raise GR00T N1.7 from 62.9% to 77.2% on LIBERO-10. When every method is given the same fixed reset, R2A reaches 89.3%, SDN 83.0% and the base policy 81.4% (Tables 5 and 16).
If any concern remains, we would be happy to clarify. Thank you for your time.
Response to Reviewer uKCE (1/2)
Thank you for the thorough review, especially your comments on the fairness of the RoboTwin comparison. We address each point below.
W1. RoboTwin comparison: chunk size and re-querying
Chunk size affects performance, and the original comparison confounded this effect with the contribution of R2A. We therefore reran RoboTwin2.0-Plus under two matched schedules. Both methods use the same chunk size throughout each rollout, and R2A no longer re-queries the policy after gripper changes; query frequency is therefore identical. R2A raises average success from 56.8% to 65.5% (+8.7 pp) with 50-step chunks, outperforming the baseline on six of seven axes, and from 59.3% to 67.1% (+7.8 pp) with 10-step chunks, outperforming it on all seven. Thus, the gains do not result from more frequent queries.
RoboTwin2.0-Plus success rates (%), same execution schedule for both methods.
Displacement compensation is one component of R2A. Even without it, R2A improves the base policy from 57.4% to 63.1% with 50-step chunks and from 59.1% to 64.7% with 10-step chunks.
Effect of displacement compensation on RoboTwin2.0-Plus (average success rate, %). The first two rows come from one separate run, so the base differs slightly from the table above.
We updated Table 2, added Tables 17 and 18 to Appendix D.5, and clarified the protocol in Appendix C.1.
W2. Disentangling ISR from retrieval, and simpler recovery strategies
Our original experiments did not isolate the two contributions. We added a "base VLA + ISR" variant (R2A with ISR only). On the Robot Initial States axis, ISR accounts for most of the gain, raising GR00T N1.7 from 32.6% to 82.7%; the full method reaches 87.5%. Across all seven axes, retrieval and harmonization without ISR reach 77.2%, ISR alone reaches 70.1%, and the full method reaches 82.1%. These results indicate that the components are complementary.
Contribution of each component, GR00T N1.7 on LIBERO-10 (success rate, %).
We also compared ISR with simpler recovery strategies. On LIBERO-10, fixed reset (81.4%), nearest-demonstration start (79.4%), ISR-MLP (82.4%), and ISR (82.7%) are within 3.3 pp; we therefore do not claim that diffusion is necessary in this setting. In Meta-World, which has two start regions, ISR reaches 90.3%, compared with 73.8% for the MLP. To isolate retrieval under the same initial-state correction, we applied an identical fixed reset across methods: GR00T N1.7 reaches 81.4%, SDN reaches 83.0%, and R2A's retrieval and harmonization reach 89.3%.
Start-state recovery (success rate, %). The two "Fixed reset +" rows apply the same fixed reset to SDN and to R2A's retrieval and harmonization.
These results appear in Table 5, Section 5.2(b), and Appendix D.4 (Table 16).