acceptodds
Under review as a conference paper at ICLR 2027

ACRO: Actor-Critic Rollout Orchestration for Exploiting VLA Capability Bounds

Abstract

Learning to act leaves open the question of when a robot should continue executing its policy and when it should intervene. We introduce ACRO, an Actor–Critic rollout orchestration layer that improves execution of a fixed vision-language-action policy. A learned Critic monitors changes in the success prospects of the policy's proposed actions. When intervention is needed, Trajectory Retraction reconstructs the execution path into a constrained search region, and a learned state-value function selects a promising continuation point. The Actor moves the robot toward that configuration and returns control to the policy. We formulate intervention through the difference between the value of continuing the current action and resuming after a correction. Across SIMPLER, RoboCasa, and real-robot manipulation, ACRO improves success by 10.4, 7.3, and 21.7 percentage points, respectively. These results show how execution-time orchestration can make more effective use of an existing policy's capabilities.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, policy evaluation, robot recovery, orchestration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_38jVOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work suggests that having the robot return to a more promising state along its historical trajectory, then observing and executing, may improve the overall success rate. The core of this approach is to determine “when to intervene” and “where to backtrack.” This work freezes VLA and uses rollout data to train a critic. Classifier use Q-value to assess the risk of failure; once triggered, system selects the historical state with the highest V-value.

Critical strengths

The motivation is sound, and the approach is simple yet effective, using Q to estimate risk and V to select historical states.

The writing and presentation are clear and easy to understand, such as Figure 1.

Critical weaknesses

One prerequisite for the method to work is that the scene remains static for most of time in task execution; the method only tracks the robot state and does not account for changes to the scene or objects that may occur during task execution. There has not been sufficient discussion on whether this affects method’s effectiveness.

The real-world experiment was small in scale.

Questions for authors

On average, how much additional time does this method add to each task execution.

It is generally believed that the Q-value for robotic manipulation tasks is difficult to estimate stabilityand can become a performance bottleneck; therefore, various approaches, such as training multiple critics, are typically employed to ensure stability. How does the stability and overall performance of the critics in this work? Although there are some methods in this work for optimizing stability, such as considers some of the recent Q-values, these are relatively lightweight.

See the weakness; consider providing some discussion or supplementary information for each point to support the main idea of the work.

Reviewer_AtsFOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VLA policies often find themselves in unseen regions making it hard for them to recover to a known good state. ACRO trains a critic to determine when to intervene and selects a state to resume the trajectory. The authors show improved performance on both simulated and real world rollouts.

Critical strengths
  1. ACRO showed improved performance for different types of policies - pi and GR00T.
  2. Authors were able to show favorable results on 2 benchmarks.
  3. Only the critic is trained while the VLA remains frozen making the method less complex and more compute efficient.
  4. ACRO shows test-time scaling opportunities (Figure 5).
Critical weaknesses
  1. Evaluation tasks seem to be short horizon with limited complexity. Would be interesting to see performance on long multi-stage tasks and increased diversity of task types.
  2. Retraction moves only the arm and doesn’t reset objects (App. A.4), so the target was chosen using a V recorded earlier, with the old scene and more time left, and the robot lands in arm-old/scene-new states the Critic never saw in training; the paper doesn’t measure how often this happens or what it costs.
  3. Robomonkey baseline was measured replacing the VLM with ACRO critic. I think we should also compare with performance with original Robomonkey setup to see performance when using a more general model / VLM.
  4. The authors should provide more qualitative examples of success and failure modes of retract and resume.
Questions for authors
  1. Does ACRO affect the generation speed? Please report latency and wall-clock times.
  2. How often does ACRO interrupt executions that would have succeeded without it, and what does that cost in lost successes and extra steps?
  3. How generalizable is the training - VLM based approaches offer generalization capability but for ACRO authors trained on at least 8 seeds per task and each evaluation task was in training set which means zero shot performance is untested?
Reviewer_Cs9xOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work proposes an actor critic rollout orchestration layer that requires no fine tuning of the base VLA model. The critic determines when to interrupt execution, while the actor selects high expected value rollback points from historical trajectories and performs efficient physical rollback via trajectory retraction. This enables the VLA to resume execution from more favorable states and improves the success rate of robotic manipulation tasks under an identical physical execution budget that accounts for rollback overhead.

Critical strengths
  1. The proposed method requires no fine‑tuning of the base vision‑language‑action model, avoiding the high cost of retraining large‑scale models.
  2. It is validated on both simulation environments and real‑robot platforms, with rollback overhead accounted for in the execution budget, which conforms to real‑world deployment constraints.
Critical weaknesses
  1. Candidate rollback points are limited to states on historically visited trajectories, and jumps to unexplored states are not supported. The method becomes inapplicable if the optimal recovery state lies outside the traversed paths.
  2. Objects may shift during physical retraction, deviating the real‑world environment from historical observations and even producing unreachable states, which poses challenges for the proposed method.
  3. The overall performance heavily relies on the prediction quality of the critic. Biases in value estimation will directly degrade task performance.
Questions for authors

See above Weaknesses.

Comments on the paper

2 comments
Area_Chair_GxxHOpenReview ↗

reviewer discussion

Dear Reviewers,

The authors have submitted their rebuttal. Please read their response carefully and engage in the discussion as soon as possible. Let us know whether the rebuttal addresses your concerns and update your final rating accordingly.

Thank you for your time and service to ICLR.

Best regards,

AC

Dear Reviewers and Area Chair,

Thank you for the constructive feedback and engagement throughout the discussion period. We appreciate Reviewer AtsF for raising their score to Clear Acceptance and Reviewer 38jV for confirming that all concerns were fully clarified. Before the discussion closes, we briefly summarize the additional experiments shared in response to the key questions raised across the reviews.

1. Safe recovery in dynamic scenes and the visited-tube design (Cs9x, AtsF, 38jV) A shared concern was whether physical retraction breaks when objects shift, or whether restricting targets to visited trajectories limits recovery. We conducted controlled ablations on RoboCasa episodes address both points:

  • Pre-contact intervention and safe retraction: In 64% of alarmed episodes, the action-conditioned Critic interrupts execution before the object shifts by more than 1 cm. Even in the remaining 36% where objects move, fresh post-retraction camera observations still outperform Base VLA (29.4% vs. 22.1%). During physical retraction, objects remain undisturbed (<= 1 cm) in 89% of cases, with shifts over 5 cm occurring in only 4.7%.
  • Why the visited tube matters: Decoupling when to intervene from where to retract shows that intervention timing drives the gain (Critic timing with random visited targets reaches 60.2% vs. 56.0% Base and 57.4% random timing, while full ACRO reaches 61.0%). Leaving the visited region via straight-line motion drops target reachability from 85.4% to 64.1%.

2. Single-Critic stability and near-zero control latency (Cs9x, AtsF, 38jV)

  • Multi-seed stability without ensembles: Training four independent single Critics from scratch (seeds 7, 17, 27, 37) yields consistent success (61.0% to 62.1% vs. 56.0% Base and 57.4% shuffled alarms).
  • Zero-blocking latency: The 0.44M GRU Critic takes 1.0 ms on an RTX 4090 (vs. 281.0 ms per VLA chunk). Pipelining the extra encoder pass on a parallel CUDA stream reduces control-loop blocking to ~0 ms while reaching 62.2% success.

3. Generalization to unseen and long-horizon tasks (AtsF, Cs9x)

  • Zero-shot task generalization: In Leave-One-Task-Out evaluation across 900 RoboCasa episodes, zero-shot ACRO improves success over Base VLA (59.1% vs. 56.0%).
  • Long-horizon tasks: On LIBERO-10 under matched step budgets (including rewind steps), ACRO improves success from 90.4% (226/250) to 93.6% (234/250).

Throughout the rebuttal, we have responded promptly to all feedback and will include all new results and qualitative examples in the final manuscript. We hope this summary assists in the final evaluation.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.