acceptodds
Under review as a conference paper at ICLR 2027

AHR: Amplitude and Horizon Refinement for Vision-Language-Action Manipulation

Abstract

Vision-language-action (VLA) policies provide strong semantic grounding and coherent motion proposals for robotic manipulation. However, small execution errors can prevent these capabilities from translating into reliable task completion. We introduce Amplitude–Horizon Refinement (AHR), which improves the execution of frozen VLA policies by jointly learning how to correct their action proposals and how long to execute each correction before re-observation. The key insight is that the intended correction horizon can change the appropriate correction from the first command. AHR therefore represents each refinement decision as a correction unit: a horizon-conditioned residual sequence paired with its execution length. Given RGB observations, proprioception, and a base proposal, a lightweight actor generates bounded residual candidates and selects a complete unit, allowing different corrections to overlapping commands across horizons. An off-policy actor–critic objective jointly optimizes residual generation and horizon selection using shared unit-level value estimates, duration-aware semi-Markov targets, and mixed demonstration and online replay. With a frozen GR00T N1.6 base policy, AHR achieves average success rates of 96.1% on RoboMimic-Pixel and 96.8% on DexMimicGen, outperforming the compared methods on both benchmarks. Across two real-world tasks, AHR achieves a 95.6% average success rate,compared with 65.0% for the frozen base policy. AHR thus provides an effective framework for refining frozen VLA policies by jointly learning action corrections and correction horizons.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Contact-Rich Manipulation, Policy Refinement, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dmeuOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Amplitude-Horizon Refinement (AHR), a framework that jointly learns residual action corrections and their execution horizons for frozen VLA policies. The method introduces horizon-conditioned residual sequences and uses off-policy actor-critic learning with duration-aware value estimation. Experiments on RoboMimic-Pixel, DexMimicGen, and two real-world manipulation tasks demonstrate improvements over existing refinement methods.

Critical strengths
  1. Jointly optimizing action corrections and execution horizons addresses a practical challenge in robotic manipulation.

  2. Horizon-conditioned residual learning and duration-aware value estimation provide a coherent framework for adaptive action refinement.

  3. The method achieves strong simulation and real-world performance. The ablation studies also provide evidence supporting the proposed design.

Critical weaknesses
  1. Incremental methodological contribution. AHR mainly combines established residual reinforcement learning and adaptive horizon optimization techniques. Although horizon-conditioned residual generation provides a meaningful distinction, the paper does not sufficiently establish the broader methodological significance of this combination.

  2. Limited experimental generalization. The evaluation relies on a single VLA backbone, with the key component ablations concentrated on one task. This limits the evidence that the proposed method generalizes across different policy architectures and manipulation settings.

Questions for authors
  1. Limited methodological novelty. The proposed approach appears closely related to existing residual RL methods, such as DICE-RL [1], combined with adaptive action horizon selection. Could the authors clarify the key methodological contributions beyond integrating these established techniques?

  2. Limited evaluation across VLA architectures. All experiments use GR00T N1.6 as the base policy. Given the claim of general VLA refinement, could the authors evaluate AHR on other architectures, such as π0, OpenVLA-OFT, or Diffusion Policy?

  3. Insufficient ablation coverage. The main correction-unit ablations are primarily conducted on Threading. Could the authors extend these experiments to other tasks to demonstrate that horizon-conditioned residual corrections consistently outperform shared residuals and adaptive horizon selection alone?

  4. Generalization and robustness. The experiments mainly evaluate task-specific refinement. How does AHR perform under unseen object configurations, execution disturbances, or different control frequencies? Such evaluations would better establish its practical generalizability.

[1] From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning. ICML 2026.

6 comments
AuthorsOpenReview ↗

Response to Reviewer dmeu (1 / 3)


We thank the reviewer for the careful reading and constructive feedback, and for recognizing the practical importance of jointly optimizing corrections and execution horizons. Your questions have helped us clarify the methodological contribution and strengthen the evaluation across backbones, tasks, and deployment conditions.


W1/Q1: Methodological Novelty and Significance

We thank the reviewer for recognizing the distinction of horizon-conditioned residuals and for encouraging us to clarify their broader methodological significance. AHR introduces a correction-unit formulation for frozen VLA refinement that jointly represents, evaluates, and optimizes a horizon-conditioned corrective trajectory and its execution duration as a single semi-Markov decision. Its central insight is that execution duration should shape the correction itself, rather than merely determine when to truncate it.

  • Key insight: Correction content depends on temporal commitment. The timing of future feedback can change the appropriate correction from its very first command. For example, a short insertion correction may prioritize alignment before re-observation, whereas a longer correction must coordinate alignment and advancement. These behaviors may require different initial residuals, which cannot generally be represented by truncating a shared, horizon-independent correction sequence. Thus, adapting execution duration alone does not capture the dependence of corrective motion on the intended duration.

  • Methodological contribution: Joint learning over correction units. Existing residual refiners typically optimize corrections at a prescribed temporal granularity, such as stepwise residual adjustments in ResFiT or fixed-horizon chunk refinement in DICE-RL. AHR instead makes the horizon-conditioned corrective trajectory and its duration the two components of a unified refinement action. A semi-Markov actor–critic framework evaluates complete correction units using duration-aware value targets and jointly optimizes residual generation and horizon selection through shared unit-level values (Eqs. 4–5). The intended horizon shapes the correction, while the correction’s expected return guides horizon selection. This allows online interaction to learn both corrective motion and its temporal organization, balancing timely feedback against coordinated multi-step execution.

  • Evidence: Comparisons with published methods and internal ablations. Under matched frozen GR00T N1.6 checkpoints, demonstrations, and online interaction budgets, AHR improves benchmark-average success over DICE-RL by 4.4 and 5.0 percentage points on RoboMimic-Pixel and DexMimicGen, respectively; the corresponding gains over ResFiT are 19.7 and 9.7 percentage points (Table 1). These comparisons support the effectiveness of our complete refinement framework relative to existing methods.

    To examine the contribution of horizon conditioning within this framework, we introduce Shared Residual as a restricted ablation of AHR. It retains our proposed semi-Markov correction-unit learning framework, jointly learns corrections and durations, and uses the same selector, critic, objective, and execution interface. It removes horizon conditioning by restricting candidate corrections to prefixes of a single horizon-independent residual sequence. AHR improves success over this ablation by 1.2–4.2 percentage points across four controlled tasks (Table 14) and 1.0–4.8 percentage points on two additional backbones (Appendix F.3). Appendix C.1.3 further characterizes the representational limitation of shared-prefix corrections. These internal comparisons isolate the additional value of allowing the intended execution duration to shape the correction itself.

In summary, AHR contributes a unified correction-unit formulation and joint learning framework for corrective motion and execution duration. Within this framework, horizon conditioning allows the intended duration to shape the correction from its first command, rather than merely determining how much of a shared residual sequence to execute.


(1 / 3)

AuthorsOpenReview ↗

Response to Reviewer dmeu (2 / 3)


W2/Q2: Effectiveness Across Policy Architectures

  • Existing evidence: π0.5\pi_{0.5} across seven tasks. Our submitted paper already evaluates AHR with both GR00T N1.6 and π0.5\pi_{0.5} across seven manipulation tasks in Appendix B.7/Figure 6, comparing each refined policy with its own frozen base. The π0.5\pi_{0.5} results support AHR across a second backbone. We now explicitly state this existing π0.5\pi_{0.5} evaluation in Section 4.1 and point readers to Appendix B.7, making the cross-backbone coverage clear in the main text.

  • Additional experiments: three policy families across seven tasks. We evaluate ACT, OpenVLA-OFT (L1) and StarVLA (flow matching), covering visual action-chunking policies and additional VLAs. Each backbone-task setting uses a shared frozen Base. Refinement methods use five independent training seeds and 100 evaluation trials per final model; Base uses five matched 100-trial evaluation batches. We report mean ± sample SD across training runs and evaluation batches, respectively. T1 shows Base → Base+AHR.

T1. Seven-task refinement: success mean ± sample SD (%).

Taskπ0.5\pi_{0.5}ACTOpenVLA-OFTStarVLA
Square85.2±3.6 → 99.0±0.776.0±2.8 → 98.6±0.582.4±2.5 → 99.0±0.784.8±2.4 → 99.2±0.4
Transport48.6±3.0 → 94.2±1.143.0±3.5 → 93.2±1.649.0±3.4 → 94.4±0.943.6±3.4 → 93.8±0.8
Tool Hang33.4±5.3 → 93.4±1.728.0±3.7 → 93.0±1.634.6±3.9 → 93.6±1.526.6±3.6 → 92.8±1.5
Coffee89.6±2.6 → 99.4±0.978.0±3.5 → 98.8±1.390.2±2.5 → 99.4±0.991.6±2.3 → 99.4±0.9
Piece Assembly80.4±2.9 → 96.4±2.377.4±3.7 → 96.2±2.478.0±2.5 → 96.4±2.177.8±3.7 → 96.2±2.4
Threading69.4±2.7 → 95.4±2.568.0±3.9 → 95.2±2.667.6±4.0 → 95.2±2.663.6±4.1 → 94.6±2.7
Drawer Cleanup55.4±2.6 → 94.8±2.051.0±3.3 → 94.0±2.158.8±3.2 → 95.0±1.962.2±3.2 → 95.4±1.8
Macro mean66.0 → 96.160.2 → 95.665.8 → 96.164.3 → 95.9

Results. AHR improves mean success on every task for every policy. Seven-task mean gains are 30.1/35.4/30.3/31.6 pp for π0.5\pi_{0.5}/ACT/OpenVLA-OFT/StarVLA, respectively. These gains support refinement across policy architectures and action heads. Appendix F.2-F.3 documents the matched interfaces and settings.


W2/Q3: Cross-Task Correction-Unit Ablations

  • Existing evidence: four-task component ablations in the submitted paper. Main-text Table 2 focuses on Threading. We have already evaluated the same correction-unit variants on Square, Transport and Tool Hang in Appendix C.1.2 (Tables 9, 11 and 13); Table 14 summarizes all four tasks. Across these tasks, AHR improves mean success by 1.2–4.2 pp over Shared Residual and 8.6–43.2 pp over Horizon-only. We now explicitly state this existing four-task coverage in Section 4.2.1 and point readers to Appendix C.1.2/Table 14, so readers can locate the additional task-wise evidence directly from the main-text ablation discussion.

  • Additional experiments: four-variant ablations on two additional backbones. We compare Best Fixed, Horizon-only, Shared Residual and AHR with π0.5\pi_{0.5} and OpenVLA-OFT (L1) on Threading, Square, Transport and Tool Hang, matching the GR00T N1.6 variants in Table 2. Methods share a frozen checkpoint within each backbone-task setting and use matched training protocols and budgets. Full results are reported in Appendix F.3.

T2. Additional-backbone correction-unit ablations: mean ± sample SD (%).

(a) π0.5\pi_{0.5}

VariantThreadingSquareTransportTool Hang
Best Fixed89.2±2.997.4±1.189.6±2.788.8±2.5
Horizon-only82.2±3.492.4±2.254.0±4.247.0±4.7
Shared Residual90.8±2.098.0±1.091.6±2.190.6±2.4
AHR95.4±2.599.0±0.794.2±1.193.4±1.7

(b) OpenVLA-OFT (L1)

VariantThreadingSquareTransportTool Hang
Best Fixed88.6±3.097.2±0.890.0±2.489.0±2.2
Horizon-only81.2±3.591.2±2.054.4±3.948.2±4.2
Shared Residual90.4±2.197.8±0.891.8±1.991.0±2.1
AHR95.2±2.699.0±0.794.4±0.993.6±1.5

AHR has the highest mean success in all eight settings, exceeding Shared Residual by 1.0-4.8 pp, Horizon-only by 6.6-46.4 pp, and Best Fixed by 1.6-6.6 pp. Since Shared Residual also jointly learns corrections and duration, the further gains support horizon conditioning. AHR versus Best Fixed assesses the complete adaptive, horizon-conditioned design. The 40.0-46.4 pp gains over Horizon-only on Transport and Tool Hang show the value of corrective motion. Together with GR00T, these ablations cover three backbones and four tasks.


(2 / 3)

AuthorsOpenReview ↗

Response to Reviewer dmeu (3 / 3)


Q4: Generalization and Robustness

We thank the reviewer for highlighting the importance of generalization and robustness beyond task-specific refinement. We extend our real-world evaluation to test whether AHR remains effective under unseen configurations and changes in execution conditions. Following Q4, we examine (i) object configurations and visual conditions, (ii) execution disturbances, and (iii) control frequencies. We reuse the GR00T N1.6+AHR checkpoints from the main-paper real-world experiments.

Evaluation protocol. We run 50 ID trials per task and 50 trials per OOD condition. The ID and OOD conditions are defined separately for each experiment below.

1. Unseen object configurations and visual conditions

  • Unseen cup colors (T3). ID: red/blue cups from the training distribution. OOD: unseen green/gray cups. Object geometry and the stacking task remain within the training distribution; only the cup colors change.
TaskID (AHR)OOD (AHR)
Cup stacking49/5049/50
  • Unseen positions (T4). ID: predominantly front–back source–target layouts, as in training. OOD: held-out left–right layouts in the same workspace. Objects and tasks remain unchanged; see Appendix F.7.
TaskID (AHR)OOD (AHR)
Cup stacking49/5049/50
Test-tube insertion47/5047/50
  • Changed lighting (T5). ID: training illumination. OOD: warm lighting from the nearby panel, with objects, task and policy observation setup unchanged (Appendix F.7).
TaskID (AHR)OOD (AHR)
Test-tube insertion47/5046/50
  • Object occlusion (T6). ID: the target region is visible, without added occluders. OOD: a kettle and box partially obscure this region during the same stacking task.
TaskID (AHR)OOD (AHR)
Cup stacking49/5045/50

2. Execution disturbances: manual target displacement (T7)

ID: stationary targets. OOD: an experimenter moves the target cup or translates/rotates the rack after grasping and before placement/insertion. The robot then continues autonomously. Appendix F.7 describes the intervention.

TaskID (AHR)OOD (AHR)
Cup stacking49/5046/50
Test-tube insertion47/5043/50

3. Different task-command frequencies: test-tube insertion (T8)

ID: 20 Hz (nominal training command rate). OOD: 10, 25 and 30 Hz. The checkpoint, candidate horizons in action steps, servo settings and task deadline are fixed (Appendix F.8).

Task10 Hz (OOD)20 Hz (ID)25 Hz (OOD)30 Hz (OOD)
Test-tube insertion47/5047/5046/5046/50

Results and interpretation. AHR achieves 45-49 successes out of 50 trials across the five object/visual settings, with unchanged success counts for unseen cup colors and positions. Following manual target displacement, it succeeds in 46/50 stacking and 43/50 insertion trials. Insertion success is 47/50 at both 10 and 20 Hz, and 46/50 at both 25 and 30 Hz. These results support robustness to the tested visual and spatial changes, recovery after target displacement, and stable success across the tested control frequencies.


Concluding Remarks

We thank the reviewer again for the constructive feedback. Overall, our responses clarify and strengthen the work in three areas:

  • Methodological novelty and significance (W1/Q1): clarified AHR's semi-Markov reformulation of residual refinement and its correction-unit architecture, providing a new refinement paradigm that preserves pretrained VLA priors while enabling precise trajectory correction for real-world deployment.

  • Architectures and ablations (W2/Q2-Q3): added ACT, OpenVLA-OFT and StarVLA evaluations on seven tasks, and extended correction-unit ablations to three backbones and four tasks.

  • Practical robustness (Q4): added real-world tests of unseen visual and spatial conditions, target disturbances, and different control frequencies.

The additional protocols, results and execution visualizations are collected in Appendix F.2-F.9: cross-policy studies in F.2-F.3, robustness tests in F.7-F.8, and execution excerpts in F.9.

We hope these clarifications and additional comparisons make AHR's contribution and the supporting evidence clearer. We would welcome further discussion of AHR's methodological contribution and the expanded experimental evaluation. If you find our responses and additional results persuasive, we would be grateful if you would consider a more favorable overall assessment of our work.


(3 / 3)

Reviewer_hvtQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Amplitude–Horizon Refinement (AHR), a framework that jointly learns residual action corrections and their execution horizons for frozen VLA policies. By introducing horizon-conditioned residual corrections and joint off-policy optimization, AHR aims to improve manipulation accuracy and execution efficiency. Experiments on RoboMimic-Pixel, DexMimicGen, and two real-world manipulation tasks demonstrate improvements over existing baselines.

Critical strengths
  1. Well-motivated and practically meaningful problem. The paper highlights that the execution horizon can affect the appropriate residual action corrections. Jointly optimizing correction amplitudes and execution horizons is a meaningful approach to improving VLA execution reliability.

  2. Promising experimental results and detailed evaluation protocols. The method demonstrates consistent improvements over existing baselines across multiple simulation tasks and two real-world manipulation tasks. The appendix also provides detailed experimental protocols and implementation details, supporting the reproducibility of the results.

Critical weaknesses
  1. Limited generalization evaluation. The experiments mainly focus on task-specific settings, without sufficient evaluation on unseen tasks or out-of-distribution (OOD) conditions. This raises concerns about the robustness and generalizability of AHR.

  2. Insufficient validation across VLA backbones. The main experiments primarily use GR00T N1.6. Although additional results with π0.5 are provided in the appendix, corresponding baseline comparisons and mechanism-specific ablations are missing, making it difficult to assess whether the improvements consistently generalize across different VLA architectures.

  3. Incomplete ablation of the core mechanism. The comparison between AHR and Shared Residual, which is essential for validating the benefit of horizon-conditioned corrections, covers only a subset of simulation tasks and does not extend to real-world experiments. This limits the evidence supporting the paper's central contribution.

Questions for authors
  1. Core mechanism validation. How much improvement does AHR achieve over Shared Residual when using π0.5 and on the two real-world tasks? These comparisons are important for assessing the effectiveness of horizon-conditioned corrections across different backbones and physical settings.

  2. OOD generalization and robustness. How does AHR perform under out-of-distribution conditions, such as object positions outside the training range, lighting variations, occlusions, or disturbances during action execution? How does its performance compare with the frozen base policy and Shared Residual under these conditions?

5 comments
AuthorsOpenReview ↗

Response to Reviewer hvtQ (1 / 2)


We thank the reviewer for the careful assessment and concrete suggestions. Your questions helped us focus the additional comparisons on mechanism validation across backbones and robot platforms, and robustness under distribution shifts. We address these points with controlled comparisons and the corresponding evaluation protocols.


W2, W3 & Q1: Mechanism Validation Across Backbones and Robot Embodiments

1. Cross-Backbone Correction-Unit Ablations

We extend the correction-unit comparison to π0.5\pi_{0.5} and OpenVLA-OFT (L1) on four tasks. Methods share one task-adapted frozen Base checkpoint within each setting. Shared Residual and AHR use matched training protocols and budgets. Each refinement method uses five independent training seeds and 100 evaluation trials per final model; Base uses five matched 100-trial evaluation batches. We report mean ± sample SD across training runs for refined methods and evaluation batches for Base. Full four-variant ablations are provided in Appendix F.3.

Table 1. Correction-unit results across two VLA backbones (success mean ± sample SD, %).

BackboneMethodThreadingSquareTransportTool Hang
π0.5\pi_{0.5}Base69.4±2.785.2±3.648.6±3.033.4±5.3
π0.5\pi_{0.5}Shared Residual90.8±2.098.0±1.091.6±2.190.6±2.4
π0.5\pi_{0.5}AHR95.4±2.599.0±0.794.2±1.193.4±1.7
OpenVLA-OFT (L1)Base67.6±4.082.4±2.549.0±3.434.6±3.9
OpenVLA-OFT (L1)Shared Residual90.4±2.197.8±0.891.8±1.991.0±2.1
OpenVLA-OFT (L1)AHR95.2±2.699.0±0.794.4±0.993.6±1.5

Across all eight settings, AHR improves mean success over Base by 13.8-60.0 percentage points and over Shared Residual by 1.0-4.8 points. With the original GR00T N1.6 ablations, this supports the horizon-conditioned design across three backbones and four tasks.

2. Cross-Backbone Real-World Evaluation on Dobot CR5

On the same Dobot CR5 platform, we compare Base, Shared Residual, and AHR with GR00T N1.6 and π0.5\pi_{0.5} on Cup Stacking and Test-Tube Insertion. In all real-world comparisons, methods share the same task-adapted frozen checkpoint within each backbone-platform-task setting. The refinement variants use matched training protocols and online interaction budgets, with 50 evaluation trials per method and task under matched conditions.

Table 2. Cross-backbone results on Dobot CR5 (successes/50).

BackboneTaskBaseShared ResidualAHR
GR00T N1.6Cup Stacking36/5045/5049/50
GR00T N1.6Test-Tube Insertion29/5042/5047/50
π0.5\pi_{0.5}Cup Stacking38/5046/5049/50
π0.5\pi_{0.5}Test-Tube Insertion30/5043/5048/50

3. Cross-Embodiment Evaluation Using the GR00T N1.6 Architecture

We use the GR00T N1.6 architecture on Dobot CR5 with a parallel gripper and Unitree G1 with two Dex3-1 three-fingered dexterous hands. Each platform-task setting uses its own task-adapted Base checkpoint. Within that setting, Base, Shared Residual, and AHR share this frozen checkpoint. This extends coverage from single-arm to bimanual dexterous manipulation.

Toy Handover and Basket Placement: The left hand grasps a toy and transfers it to the right hand, which places it in a basket. This requires coordinating both hands' relative poses and grasp/release timing.

Table 3. Cross-embodiment results with GR00T N1.6 (successes/50).

PlatformTaskBaseShared ResidualAHR
Dobot CR5Cup Stacking36/5045/5049/50
Dobot CR5Test-Tube Insertion29/5042/5047/50
Unitree G1Toy Handover and Basket Placement25/5041/5048/50

Analysis. Across both backbones on Dobot CR5 (Table 2), AHR improves over Base by 22-36 percentage points and over Shared Residual by 6-10 points. On Unitree G1 (Table 3), the corresponding gains are 46 and 14 points, respectively. Shared Residual uses the same adaptive horizon selection and proposal-refresh mechanism. These gains support AHR's horizon-conditioned correction-unit architecture, which matches correction content to its intended execution duration.

Appendix F.4-F.6 provides platform configurations and evaluation protocols; F.9 provides execution visualizations.


(1 / 2)

AuthorsOpenReview ↗

Response to Reviewer hvtQ (2 / 2)


W1 & Q2: OOD Generalization and Robustness

We thank the reviewer for requesting OOD comparisons with Base and Shared Residual. We consider unseen positions, lighting changes, partial occlusion, and execution disturbances.

Evaluation protocol. Each Dobot CR5 task uses its own task-adapted GR00T N1.6 checkpoint, frozen and shared by Base, Shared Residual, and AHR within that task. Shared Residual and AHR use matched training protocols and online interaction budgets. We reuse the main-paper AHR checkpoints and evaluate each method over 50 trials per condition, with matched initial-state distributions and success criteria. Tables 4 and 5 summarize the ID reference and OOD comparisons; Appendix F.7 provides the complete protocol.

Table 4. ID reference results (successes/50).

TaskBaseShared ResidualAHR
Cup Stacking36/5045/5049/50
Test-Tube Insertion29/5042/5047/50

Table 5. OOD comparison under the four requested condition shifts (successes/50).

ConditionTaskBaseShared ResidualAHR
Unseen positionsCup Stacking34/5044/5049/50
Unseen positionsTest-Tube Insertion26/5041/5047/50
Warm lightingTest-Tube Insertion28/5041/5046/50
Partial occlusionCup Stacking32/5042/5045/50
Target displacementCup Stacking30/5041/5046/50
Target displacementTest-Tube Insertion23/5037/5043/50

Across these six settings, AHR exceeds Base by 26-42 percentage points and Shared Residual by 6-12.

1. Unseen object positions. ID: placements from the predominantly front–back source–target arrangements used in training. OOD: held-out left–right arrangements within the same workspace, with object identities and task objectives unchanged.

AHR retains its ID counts of 49/50 stacking and 47/50 insertion successes, exceeding Shared by 10 and 12 percentage points. Residual refinement can compensate for shifted approach and alignment errors. AHR further allows short alignment and longer placement or insertion segments to use different corrections, whereas Shared constrains them to a common residual prefix.

2. Lighting variations. ID: training illumination. OOD: warm lighting from the panel beside the workspace, with the objects, task and policy observation setup unchanged.

Each method loses one success relative to ID, preserving AHR's margins of 36 percentage points over Base and 10 over Shared. The tested appearance change largely preserves the benefits of refinement.

3. Partial occlusion. ID: the target region is visible, without added occluders. OOD: a kettle and box partially obscure the target region during Cup Stacking.

AHR achieves 45/50 stacking successes. From ID, AHR and Base each lose four successes and Shared loses three. Both residual methods can use remaining visual cues and proprioception to correct recoverable errors. AHR retains a 6-percentage-point margin over Shared; reduced target visibility may limit the scope for further gains from matching corrective motion to duration.

4. Execution disturbances. ID: targets remain stationary. OOD: after grasping and before placement or insertion, an experimenter moves the target cup or translates and/or rotates the rack. The robot then continues autonomously.

AHR completes 46/50 stacking and 43/50 insertion trials. Base already uses feedback and replanning; the residual methods further improve recovery success. Shared and AHR both adapt horizons and refresh proposals, while AHR additionally permits short realignment and longer realignment-and-advance segments to begin with different corrections. Its 10-12-percentage-point gains over Shared support coupling recovery motion with execution duration under the tested disturbances.


Concluding Remarks

We thank the reviewer again for the constructive feedback. Our responses strengthen three aspects:

  • Cross-backbone validation: mechanism ablations across three backbones and four simulation tasks.
  • Real-world mechanism evidence: controlled Base/Shared Residual/AHR comparisons on CR5 and G1.
  • OOD robustness: comparisons under all four requested shifts, with AHR exceeding Base by 26-42 and Shared by 6-12 percentage points.

The additional protocols, results and visualizations are collected in Appendix F.3-F.7 and F.9. We hope these controlled comparisons clarify the contribution of horizon-conditioned corrections across the tested backbones, robot platforms and deployment conditions. We would be pleased to continue the discussion of the controlled comparisons and their implications for AHR's mechanism and robustness. We sincerely hope that the additional evidence and clarifications will lead to a more favorable assessment of our work.


(2 / 2)

Thank you for the comprehensive rebuttal and additional experiments. The new cross-backbone comparisons, real-world ablations, and OOD evaluations substantially address my original concerns. In particular, the controlled comparison with Shared Residual provides stronger evidence for the effectiveness of horizon-conditioned corrections.

Some limitations remain, particularly regarding generalization to unseen tasks and the relatively modest improvements in certain simulation settings. Nevertheless, I appreciate the authors' efforts and find the additional evidence sufficiently convincing to revise my rating from 2 to 3.

Reviewer_GnwzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the problem that small execution errors can prevent VLA policies from successfully completing manipulation tasks despite generating reasonable action proposals. It proposes Amplitude–Horizon Refinement (AHR), a residual RL framework that jointly learns horizon-conditioned action corrections and execution lengths using an off-policy actor–critic objective. Experiments demonstrate strong performance across simulation benchmarks and real-world manipulation tasks.

Critical strengths

The reliability of value estimation across different correction horizons remains insufficiently validated. AHR optimizes its horizon-selection policy using critic estimates for all candidate correction units, although only the selected unit is actually executed. It remains unclear whether these estimates accurately capture the relative downstream returns of alternative horizons.

Critical weaknesses

The reliability of value estimation across different correction horizons remains insufficiently validated. AHR optimizes its horizon-selection policy using critic estimates for all candidate correction units, although only the selected unit is actually executed. It remains unclear whether these estimates accurately capture the relative downstream returns of alternative horizons.

Questions for authors
  1. Can the authors evaluate how accurately the learned critic ranks correction units with different execution horizons, for example by executing alternative candidates from identical simulation states and comparing their actual downstream returns?
  2. Can the authors further clarify how much of the improvement comes from learning more effective horizon-conditioned corrections versus learning better horizon-selection decisions?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Gnwz (1 / 2)


We appreciate the reviewer’s careful assessment and encouraging comments on the clarity of our presentation and the strength of our experimental results.Your questions help us sharpen the evaluation of critic reliability and distinguish the contributions of correction quality and horizon selection.


Q1: Accuracy of Cross-Horizon Critic Rankings

We thank the reviewer for suggesting this direct same-state comparison. We compare the critic's ordering of AHR correction units with their ordering by complete rollout returns on Tool Hang and Threading.

Learning Values Across Execution Horizons. AHR learns a shared Q(s,C,k)Q(s,C,k) for observation ss, corrected command segment CC and horizon kk. Executed units supply observed rewards and successor observations. Initial uniform horizon sampling and horizon-balanced demonstration replay provide data across horizons, supporting predictions for candidates not selected at a given decision. This follows sampled value learning for temporally extended actions [1, Sec. 3.2].

The critic's temporal-difference update (manuscript Eq. (4)) fits the executed unit to

+(1-d_t)\gamma^{n_t}\bar V(s_{t+n_t}),$$ where $n_t$ is the actual duration, $d_t$ indicates termination, and $\bar V$ estimates continuation under the target actor. The first term is the observed segment return. Discounting continuation by $\gamma^{n_t}$ places different horizons under the same primitive-step return objective. The **actor objective (manuscript Eq. (5))** uses ensemble-mean critic predictions for all candidates to optimize corrections and selection probabilities. **Same-State Evaluation on Tool Hang and Threading.** The evaluation uses five independently trained AHR seeds per task, with GR00T N1.6 frozen. For each seed, one eligible nonterminal decision point is sampled uniformly from each of 50 held-out AHR episodes, requiring at least 16 primitive steps remaining. Sampling gives 250 states per task, independently of Q gaps and episode outcomes. At each state, all models remain frozen while the same VLA proposal, the four complete current-actor correction units with $k\in\{2,4,8,16\}$, and their ten-head mean Q scores are cached. Each candidate retains its horizon-conditioned correction segment. Each branch restores the same full environment/controller state, observation history, and original remaining time. It executes the designated cached unit once, then follows the common frozen target actor, with horizon sampling. Each branch follows its own observations and retains the original deadline. The protocol uses 128 independent repeats per candidate at each state, with matched random streams across candidates within each repeat. Complete returns include all rewards from the initial unit and subsequent execution through termination or the original deadline, discounted by $\gamma=0.99$ per primitive step. Returns are averaged across repeats for each candidate. **Kendall's $\tau_b$** [2] compares the critic-predicted ordering of the four candidates with their ordering by mean return. It accounts for ties across all six candidate pairs; values closer to 1 indicate stronger agreement. Defined per-state coefficients are averaged within each seed. We report the equally weighted mean and sample SD of the five seed means. **State-Level Example and Overall Results.** Table R1 presents a reported state-level example. Both critic Q scores and mean returns give the ordering $2>16>8>4$. All six candidate pairs are concordant, yielding $\tau_b=1$. Table R2 summarizes agreement and counts of states with defined coefficients. **Table R1. Critic scores and mean rollout returns for a reported state.** | Horizon $k$ | Critic Q | Mean rollout return | | ---: | ---: | ---: | | 2 | 0.747927 | 0.741950 | | 4 | 0.593702 | 0.589831 | | 8 | 0.614140 | 0.616037 | | 16 | 0.721486 | 0.716616 | **Table R2. Agreement between critic-predicted and mean-return-based rankings.** | Task | $\tau_b$ (mean ± sample SD) | Defined / total | Return-flat | Critic-flat only | | :--- | ---: | ---: | ---: | ---: | | Tool Hang | 0.821 ± 0.028 | 250/250 | 0 | 0 | | Threading | 0.849 ± 0.028 | 248/250 | 1 | 1 | These results show strong positive rank agreement under the specified continuation policy. Across both tasks, 498/500 states have defined coefficients; one return-flat and one critic-flat-only state remain in the coverage denominator. Appendix F.10 provides the protocol, per-seed results, tie handling and five reported examples. This statistic evaluates ranking agreement, rather than absolute value calibration or deployment-policy success. --- [1] Sutton et al. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. *Artificial Intelligence*. [2] Kendall. The treatment of ties in ranking problems. *Biometrika*. --- (1 / 2)
AuthorsOpenReview ↗

Response to Reviewer Gnwz (2 / 2)


Q2: Contributions of Horizon-Conditioned Corrections and Horizon Selection

We thank the reviewer for asking us to clarify the sources of improvement. We examine whether AHR produces better corrections at a given horizon and whether its learned horizon choice makes better use of the available candidates.

Existing policy-level evidence. Table R3 summarizes the submitted GR00T N1.6 controlled ablations. Best Fixed uses the task-level horizon selected by validation from nine independently trained fixed-horizon policies. Shared Residual jointly learns corrections and horizon selection, with candidates taking prefixes of one residual sequence. AHR instead conditions each correction on its intended horizon.

Table R3. Controlled ablations reported in the manuscript: success mean ± SD (%).

TaskBest FixedShared ResidualAHR
Threading90.2±2.891.8±1.996.0±2.4
Square97.2±0.898.0±0.799.2±0.4
Transport90.8±1.992.6±1.595.0±0.7
Tool Hang89.6±1.991.4±1.894.0±1.2

AHR improves over Shared Residual by 1.2-4.2 percentage points across the four tasks; Shared Residual improves over Best Fixed by 0.8-1.8 points. These full-policy results compare separately trained configurations. The matched-state tests below examine correction-source and horizon-selection effects at the initial decision.

Same-state evaluation. Threading and Tool Hang each use five paired AHR/Shared Residual training runs and 50 held-out decision points per pair: 250 states per task. One cached VLA proposal supplies four AHR-generated horizon-conditioned units and four Shared Residual prefix units at k∈{2,4,8,16}k\in\{2,4,8,16\}. Each branch forces only its initial unit, then follows the same complete frozen AHR deployment policy, recomputing proposals, corrections and actor-probability argmax choices from its own observations until termination or the original deadline. Each candidate uses 128 paired repeats and primitive-step discount γ=0.99\gamma=0.99.

Matched-horizon correction units. Row 1 compares Shared Residual and AHR-generated units at the same state and nominal horizon. Both sources use the same fixed horizon weights and complete AHR continuation. This isolates the initial correction source under the matched conditions.

Selection over fixed AHR-generated candidates. Row 2 compares the return of the unit selected by AHR's actor-probability argmax with the reference-weighted mean return of the same four cached AHR units. The fixed weights are AHR argmax-choice frequencies at 50 disjoint validation states per training pair. They ignore the current test state; candidate generation remains state-dependent. The reference is computed from the estimated candidate returns, without extra reference-policy rollouts. All 250 test states per task enter both comparisons.

Table R4. Mean complete downstream discounted return before → after each initial-decision intervention. Parentheses give paired increases. All branches use complete AHR continuation after the initial unit.

Controlled comparisonThreading return ↑Tool Hang return ↑
Matched horizons: Shared → AHR-generated units0.6392 → 0.6769 (+0.0377)0.5300 → 0.5628 (+0.0328)
Fixed AHR units: reference weights → AHR selector0.6769 → 0.7001 (+0.0232)0.5628 → 0.5886 (+0.0258)

Both interventions measure complete returns, but change only the initial decision. Their gains are conditional on the matched states and common AHR continuation; correction and selection may interact. They do not partition the full-policy success-rate gains in Table R3. Entries equally weight five training pairs; Appendix F.11 gives paired 95% intervals and per-seed values.


Concluding Remarks

We thank the reviewer again for the constructive feedback. Our response links cross-horizon value learning to a direct ranking check (Appendix F.10) and presents matched initial-decision comparisons of correction quality and horizon selection (F.11). The appendix also collects broader evaluations across policy backbones and the Dobot CR5 and Unitree G1 platforms (F.3, F.5-F.6). We would appreciate your feedback on whether these comparisons address your questions, and would be happy to clarify any remaining point.


(2 / 2)

AuthorsOpenReview ↗

Follow-up on Our Responses to Your Questions


Dear Reviewer Gnwz,

Thank you again for your thoughtful review and constructive questions. We sincerely appreciate your suggestions, which have helped us strengthen the evaluation of AHR's core mechanism.

We would like to briefly follow up on the two questions you raised. In our responses, we provided:

  1. Cross-horizon critic rankings: Same-state rollout evaluations validating cross-horizon critic rankings (Appendix F.10).
  2. Contributions of correction quality and horizon selection: Controlled comparisons examining the respective contributions of horizon-conditioned corrections and horizon selection (Appendix F.11).

We would greatly appreciate hearing your thoughts on whether these additional analyses have addressed your main concerns and strengthened the evidence supporting our approach.

If any questions or uncertainties remain, we would be more than happy to provide further clarification.

Thank you again for your time and consideration.

Best regards,
The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global Response: Paper Revisions

We thank all reviewers for their careful evaluation and constructive feedback. Following the reviewers' suggestions, we have collected the additional experiments, analyses, and evaluation protocols in Appendix F (pp. 38-50). AHR reformulates trajectory refinement for frozen VLAs as a unified semi-Markov decision problem over correction units, each pairing a horizon-conditioned residual sequence with its execution length. The additional results strengthen the evidence across policy architectures, robot platforms, and deployment conditions, while directly examining critic reliability and the sources of improvement.

  • In Secs. F.2-F.3, we added seven-task evaluations with ACT, OpenVLA-OFT, and StarVLA, complementing the existing GR00T N1.6 and π0.5\pi_{0.5} evaluations.
  • In Sec. F.3, we added complete correction-unit ablations with π0.5\pi_{0.5} and OpenVLA-OFT on four tasks, comparing Best Fixed, Horizon-only, Shared Residual, and AHR. Together with the submitted GR00T results, these comparisons cover three backbones and four tasks.
  • In Secs. F.4-F.6, we expanded the controlled real-world comparisons among Base, Shared Residual, and AHR, covering GR00T N1.6 and π0.5\pi_{0.5} on Dobot CR5, and GR00T N1.6 for bimanual dexterous manipulation on Unitree G1, with task-adapted frozen Base checkpoints.
  • In Sec. F.7, we added real-world OOD comparisons under unseen positions, changed lighting, partial occlusion, and target displacement during execution, together with a separate evaluation of AHR under unseen cup colors.
  • In Sec. F.8, we added evaluations at 10, 20, 25, and 30 Hz task-command frequencies, using the same AHR checkpoint.
  • In Sec. F.9, we added execution visualizations illustrating the tested spatial, visual, and disturbance conditions.
  • In Sec. F.10, we added same-state rollout comparisons of cross-horizon critic rankings, obtaining Kendall's τb\tau_b of 0.821 ± 0.028 on Tool Hang and 0.849 ± 0.028 on Threading.
  • In Sec. F.11, we added two controlled initial-decision comparisons: correction sources at matched horizons and state-dependent horizon selection over fixed AHR-generated candidates. Both use complete downstream returns under a common complete AHR continuation policy.

On RoboMimic-Pixel and DexMimicGen, AHR outperforms ResFiT by 19.7 and 9.7 percentage points, and DICE-RL by 4.4 and 5.0 percentage points, respectively. In the newly added ACT, OpenVLA-OFT, and StarVLA evaluations, AHR improves seven-task mean success over the corresponding frozen Base by 30.3–35.4 percentage points. In our controlled component ablations, full AHR further improves over its Shared Residual variant by 1.0–4.8 percentage points in cross-backbone simulation and 6–14 percentage points in matched CR5/G1 comparisons. Shared Residual retains the same horizon-selection mechanism and proposal-refresh rule while restricting candidate corrections to prefixes of one horizon-independent residual sequence. The consistent gains over this control support horizon-conditioned correction within joint correction-and-duration learning.

We have also provided detailed point-by-point responses to each reviewer. We sincerely thank the reviewers for helping us clarify the contribution and strengthen the experimental evidence, and hope the revisions address the concerns raised.

Dear Reviewers,

Thank you again for your thoughtful feedback. As the discussion deadline approaches, we would greatly appreciate hearing whether our responses have addressed your questions and concerns.

If any concerns remain, please feel free to share them whenever convenient for you. We remain available throughout the discussion period and will do our utmost to address any outstanding issues promptly and thoroughly. If our clarifications have resolved your concerns, we would be grateful if you could take them into account when updating your assessment.

Your feedback is very valuable to us, and we sincerely appreciate your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.