- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Amplitude-Horizon Refinement (AHR), a framework that jointly learns residual action corrections and their execution horizons for frozen VLA policies. The method introduces horizon-conditioned residual sequences and uses off-policy actor-critic learning with duration-aware value estimation. Experiments on RoboMimic-Pixel, DexMimicGen, and two real-world manipulation tasks demonstrate improvements over existing refinement methods.
Critical strengths
-
Jointly optimizing action corrections and execution horizons addresses a practical challenge in robotic manipulation.
-
Horizon-conditioned residual learning and duration-aware value estimation provide a coherent framework for adaptive action refinement.
-
The method achieves strong simulation and real-world performance. The ablation studies also provide evidence supporting the proposed design.
Critical weaknesses
-
Incremental methodological contribution. AHR mainly combines established residual reinforcement learning and adaptive horizon optimization techniques. Although horizon-conditioned residual generation provides a meaningful distinction, the paper does not sufficiently establish the broader methodological significance of this combination.
-
Limited experimental generalization. The evaluation relies on a single VLA backbone, with the key component ablations concentrated on one task. This limits the evidence that the proposed method generalizes across different policy architectures and manipulation settings.
Questions for authors
-
Limited methodological novelty. The proposed approach appears closely related to existing residual RL methods, such as DICE-RL [1], combined with adaptive action horizon selection. Could the authors clarify the key methodological contributions beyond integrating these established techniques?
-
Limited evaluation across VLA architectures. All experiments use GR00T N1.6 as the base policy. Given the claim of general VLA refinement, could the authors evaluate AHR on other architectures, such as π0, OpenVLA-OFT, or Diffusion Policy?
-
Insufficient ablation coverage. The main correction-unit ablations are primarily conducted on Threading. Could the authors extend these experiments to other tasks to demonstrate that horizon-conditioned residual corrections consistently outperform shared residuals and adaptive horizon selection alone?
-
Generalization and robustness. The experiments mainly evaluate task-specific refinement. How does AHR perform under unseen object configurations, execution disturbances, or different control frequencies? Such evaluations would better establish its practical generalizability.
[1] From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning. ICML 2026.
6 comments
Response to Reviewer dmeu (2 / 3)
W2/Q2: Effectiveness Across Policy Architectures
-
Existing evidence: across seven tasks. Our submitted paper already evaluates AHR with both GR00T N1.6 and across seven manipulation tasks in Appendix B.7/Figure 6, comparing each refined policy with its own frozen base. The results support AHR across a second backbone. We now explicitly state this existing evaluation in Section 4.1 and point readers to Appendix B.7, making the cross-backbone coverage clear in the main text.
-
Additional experiments: three policy families across seven tasks. We evaluate ACT, OpenVLA-OFT (L1) and StarVLA (flow matching), covering visual action-chunking policies and additional VLAs. Each backbone-task setting uses a shared frozen Base. Refinement methods use five independent training seeds and 100 evaluation trials per final model; Base uses five matched 100-trial evaluation batches. We report mean ± sample SD across training runs and evaluation batches, respectively. T1 shows Base → Base+AHR.
T1. Seven-task refinement: success mean ± sample SD (%).
| Task | ACT | OpenVLA-OFT | StarVLA | |
|---|---|---|---|---|
| Square | 85.2±3.6 → 99.0±0.7 | 76.0±2.8 → 98.6±0.5 | 82.4±2.5 → 99.0±0.7 | 84.8±2.4 → 99.2±0.4 |
| Transport | 48.6±3.0 → 94.2±1.1 | 43.0±3.5 → 93.2±1.6 | 49.0±3.4 → 94.4±0.9 | 43.6±3.4 → 93.8±0.8 |
| Tool Hang | 33.4±5.3 → 93.4±1.7 | 28.0±3.7 → 93.0±1.6 | 34.6±3.9 → 93.6±1.5 | 26.6±3.6 → 92.8±1.5 |
| Coffee | 89.6±2.6 → 99.4±0.9 | 78.0±3.5 → 98.8±1.3 | 90.2±2.5 → 99.4±0.9 | 91.6±2.3 → 99.4±0.9 |
| Piece Assembly | 80.4±2.9 → 96.4±2.3 | 77.4±3.7 → 96.2±2.4 | 78.0±2.5 → 96.4±2.1 | 77.8±3.7 → 96.2±2.4 |
| Threading | 69.4±2.7 → 95.4±2.5 | 68.0±3.9 → 95.2±2.6 | 67.6±4.0 → 95.2±2.6 | 63.6±4.1 → 94.6±2.7 |
| Drawer Cleanup | 55.4±2.6 → 94.8±2.0 | 51.0±3.3 → 94.0±2.1 | 58.8±3.2 → 95.0±1.9 | 62.2±3.2 → 95.4±1.8 |
| Macro mean | 66.0 → 96.1 | 60.2 → 95.6 | 65.8 → 96.1 | 64.3 → 95.9 |
Results. AHR improves mean success on every task for every policy. Seven-task mean gains are 30.1/35.4/30.3/31.6 pp for /ACT/OpenVLA-OFT/StarVLA, respectively. These gains support refinement across policy architectures and action heads. Appendix F.2-F.3 documents the matched interfaces and settings.
W2/Q3: Cross-Task Correction-Unit Ablations
-
Existing evidence: four-task component ablations in the submitted paper. Main-text Table 2 focuses on Threading. We have already evaluated the same correction-unit variants on Square, Transport and Tool Hang in Appendix C.1.2 (Tables 9, 11 and 13); Table 14 summarizes all four tasks. Across these tasks, AHR improves mean success by 1.2–4.2 pp over Shared Residual and 8.6–43.2 pp over Horizon-only. We now explicitly state this existing four-task coverage in Section 4.2.1 and point readers to Appendix C.1.2/Table 14, so readers can locate the additional task-wise evidence directly from the main-text ablation discussion.
-
Additional experiments: four-variant ablations on two additional backbones. We compare Best Fixed, Horizon-only, Shared Residual and AHR with and OpenVLA-OFT (L1) on Threading, Square, Transport and Tool Hang, matching the GR00T N1.6 variants in Table 2. Methods share a frozen checkpoint within each backbone-task setting and use matched training protocols and budgets. Full results are reported in Appendix F.3.
T2. Additional-backbone correction-unit ablations: mean ± sample SD (%).
(a)
| Variant | Threading | Square | Transport | Tool Hang |
|---|---|---|---|---|
| Best Fixed | 89.2±2.9 | 97.4±1.1 | 89.6±2.7 | 88.8±2.5 |
| Horizon-only | 82.2±3.4 | 92.4±2.2 | 54.0±4.2 | 47.0±4.7 |
| Shared Residual | 90.8±2.0 | 98.0±1.0 | 91.6±2.1 | 90.6±2.4 |
| AHR | 95.4±2.5 | 99.0±0.7 | 94.2±1.1 | 93.4±1.7 |
(b) OpenVLA-OFT (L1)
| Variant | Threading | Square | Transport | Tool Hang |
|---|---|---|---|---|
| Best Fixed | 88.6±3.0 | 97.2±0.8 | 90.0±2.4 | 89.0±2.2 |
| Horizon-only | 81.2±3.5 | 91.2±2.0 | 54.4±3.9 | 48.2±4.2 |
| Shared Residual | 90.4±2.1 | 97.8±0.8 | 91.8±1.9 | 91.0±2.1 |
| AHR | 95.2±2.6 | 99.0±0.7 | 94.4±0.9 | 93.6±1.5 |
AHR has the highest mean success in all eight settings, exceeding Shared Residual by 1.0-4.8 pp, Horizon-only by 6.6-46.4 pp, and Best Fixed by 1.6-6.6 pp. Since Shared Residual also jointly learns corrections and duration, the further gains support horizon conditioning. AHR versus Best Fixed assesses the complete adaptive, horizon-conditioned design. The 40.0-46.4 pp gains over Horizon-only on Transport and Tool Hang show the value of corrective motion. Together with GR00T, these ablations cover three backbones and four tasks.
(2 / 3)
Response to Reviewer dmeu (3 / 3)
Q4: Generalization and Robustness
We thank the reviewer for highlighting the importance of generalization and robustness beyond task-specific refinement. We extend our real-world evaluation to test whether AHR remains effective under unseen configurations and changes in execution conditions. Following Q4, we examine (i) object configurations and visual conditions, (ii) execution disturbances, and (iii) control frequencies. We reuse the GR00T N1.6+AHR checkpoints from the main-paper real-world experiments.
Evaluation protocol. We run 50 ID trials per task and 50 trials per OOD condition. The ID and OOD conditions are defined separately for each experiment below.
1. Unseen object configurations and visual conditions
- Unseen cup colors (T3). ID: red/blue cups from the training distribution. OOD: unseen green/gray cups. Object geometry and the stacking task remain within the training distribution; only the cup colors change.
| Task | ID (AHR) | OOD (AHR) |
|---|---|---|
| Cup stacking | 49/50 | 49/50 |
- Unseen positions (T4). ID: predominantly front–back source–target layouts, as in training. OOD: held-out left–right layouts in the same workspace. Objects and tasks remain unchanged; see Appendix F.7.
| Task | ID (AHR) | OOD (AHR) |
|---|---|---|
| Cup stacking | 49/50 | 49/50 |
| Test-tube insertion | 47/50 | 47/50 |
- Changed lighting (T5). ID: training illumination. OOD: warm lighting from the nearby panel, with objects, task and policy observation setup unchanged (Appendix F.7).
| Task | ID (AHR) | OOD (AHR) |
|---|---|---|
| Test-tube insertion | 47/50 | 46/50 |
- Object occlusion (T6). ID: the target region is visible, without added occluders. OOD: a kettle and box partially obscure this region during the same stacking task.
| Task | ID (AHR) | OOD (AHR) |
|---|---|---|
| Cup stacking | 49/50 | 45/50 |
2. Execution disturbances: manual target displacement (T7)
ID: stationary targets. OOD: an experimenter moves the target cup or translates/rotates the rack after grasping and before placement/insertion. The robot then continues autonomously. Appendix F.7 describes the intervention.
| Task | ID (AHR) | OOD (AHR) |
|---|---|---|
| Cup stacking | 49/50 | 46/50 |
| Test-tube insertion | 47/50 | 43/50 |
3. Different task-command frequencies: test-tube insertion (T8)
ID: 20 Hz (nominal training command rate). OOD: 10, 25 and 30 Hz. The checkpoint, candidate horizons in action steps, servo settings and task deadline are fixed (Appendix F.8).
| Task | 10 Hz (OOD) | 20 Hz (ID) | 25 Hz (OOD) | 30 Hz (OOD) |
|---|---|---|---|---|
| Test-tube insertion | 47/50 | 47/50 | 46/50 | 46/50 |
Results and interpretation. AHR achieves 45-49 successes out of 50 trials across the five object/visual settings, with unchanged success counts for unseen cup colors and positions. Following manual target displacement, it succeeds in 46/50 stacking and 43/50 insertion trials. Insertion success is 47/50 at both 10 and 20 Hz, and 46/50 at both 25 and 30 Hz. These results support robustness to the tested visual and spatial changes, recovery after target displacement, and stable success across the tested control frequencies.
Concluding Remarks
We thank the reviewer again for the constructive feedback. Overall, our responses clarify and strengthen the work in three areas:
-
Methodological novelty and significance (W1/Q1): clarified AHR's semi-Markov reformulation of residual refinement and its correction-unit architecture, providing a new refinement paradigm that preserves pretrained VLA priors while enabling precise trajectory correction for real-world deployment.
-
Architectures and ablations (W2/Q2-Q3): added ACT, OpenVLA-OFT and StarVLA evaluations on seven tasks, and extended correction-unit ablations to three backbones and four tasks.
-
Practical robustness (Q4): added real-world tests of unseen visual and spatial conditions, target disturbances, and different control frequencies.
The additional protocols, results and execution visualizations are collected in Appendix F.2-F.9: cross-policy studies in F.2-F.3, robustness tests in F.7-F.8, and execution excerpts in F.9.
We hope these clarifications and additional comparisons make AHR's contribution and the supporting evidence clearer. We would welcome further discussion of AHR's methodological contribution and the expanded experimental evaluation. If you find our responses and additional results persuasive, we would be grateful if you would consider a more favorable overall assessment of our work.
(3 / 3)
Response to Reviewer dmeu (1 / 3)
We thank the reviewer for the careful reading and constructive feedback, and for recognizing the practical importance of jointly optimizing corrections and execution horizons. Your questions have helped us clarify the methodological contribution and strengthen the evaluation across backbones, tasks, and deployment conditions.
W1/Q1: Methodological Novelty and Significance
We thank the reviewer for recognizing the distinction of horizon-conditioned residuals and for encouraging us to clarify their broader methodological significance. AHR introduces a correction-unit formulation for frozen VLA refinement that jointly represents, evaluates, and optimizes a horizon-conditioned corrective trajectory and its execution duration as a single semi-Markov decision. Its central insight is that execution duration should shape the correction itself, rather than merely determine when to truncate it.
Key insight: Correction content depends on temporal commitment. The timing of future feedback can change the appropriate correction from its very first command. For example, a short insertion correction may prioritize alignment before re-observation, whereas a longer correction must coordinate alignment and advancement. These behaviors may require different initial residuals, which cannot generally be represented by truncating a shared, horizon-independent correction sequence. Thus, adapting execution duration alone does not capture the dependence of corrective motion on the intended duration.
Methodological contribution: Joint learning over correction units. Existing residual refiners typically optimize corrections at a prescribed temporal granularity, such as stepwise residual adjustments in ResFiT or fixed-horizon chunk refinement in DICE-RL. AHR instead makes the horizon-conditioned corrective trajectory and its duration the two components of a unified refinement action. A semi-Markov actor–critic framework evaluates complete correction units using duration-aware value targets and jointly optimizes residual generation and horizon selection through shared unit-level values (Eqs. 4–5). The intended horizon shapes the correction, while the correction’s expected return guides horizon selection. This allows online interaction to learn both corrective motion and its temporal organization, balancing timely feedback against coordinated multi-step execution.
Evidence: Comparisons with published methods and internal ablations. Under matched frozen GR00T N1.6 checkpoints, demonstrations, and online interaction budgets, AHR improves benchmark-average success over DICE-RL by 4.4 and 5.0 percentage points on RoboMimic-Pixel and DexMimicGen, respectively; the corresponding gains over ResFiT are 19.7 and 9.7 percentage points (Table 1). These comparisons support the effectiveness of our complete refinement framework relative to existing methods.
To examine the contribution of horizon conditioning within this framework, we introduce Shared Residual as a restricted ablation of AHR. It retains our proposed semi-Markov correction-unit learning framework, jointly learns corrections and durations, and uses the same selector, critic, objective, and execution interface. It removes horizon conditioning by restricting candidate corrections to prefixes of a single horizon-independent residual sequence. AHR improves success over this ablation by 1.2–4.2 percentage points across four controlled tasks (Table 14) and 1.0–4.8 percentage points on two additional backbones (Appendix F.3). Appendix C.1.3 further characterizes the representational limitation of shared-prefix corrections. These internal comparisons isolate the additional value of allowing the intended execution duration to shape the correction itself.
In summary, AHR contributes a unified correction-unit formulation and joint learning framework for corrective motion and execution duration. Within this framework, horizon conditioning allows the intended duration to shape the correction from its first command, rather than merely determining how much of a shared residual sequence to execute.
(1 / 3)