acceptodds
Under review as a conference paper at ICLR 2027

ARGO-VLA: Calibrated Gating and Escalation for Correcting and Orchestrating Frozen VLA Policies in the Laboratory

Abstract

Vision-language-action (VLA) policies fine-tuned for laboratory skills fail at two timescales. Within a skill they miss millimeter tolerances: on AutoBio (Lan et al., 2026), a fine-tuned succeeds on at most 43% of episodes of any medium task and 15% of any success-scored hard task. Across a protocol they cannot order steps, withhold hazardous operations, or notice that a step failed. We propose ARGO-VLA, built around one calibrated novelty score used at both timescales. At the control rate the score masks a bounded residual, trained by behavior cloning and PPO on a chunk-level residual Markov decision process and restricted to its training support by a k-nearest-neighbor trust gate, that corrects each action chunk of a frozen . At the protocol-step rate the same score, thresholded by split conformal prediction, halts a skill and escalates to a typed state machine of LLM agents that plans, validates and risk-tags, requests human approval, executes and reviews protocol steps; the threshold bounds the false-escalation rate per nominal skill call. We introduce AnomalyBio, which extends AutoBio with multi-step protocols, injected failures and hazardous or compositionally infeasible goal probes judged by simulator ground truth, and ablate the trust gate, anomaly mask, escalation, validator critique and LLM reviewer. On transfer, the residual reaches 48.7% success versus 40.7% for AutoBio's published , a comparison across different checkpoints and episodes, not a controlled one. Escalation detects 52% of monitor-reachable injected failures at 1.8% false escalations per call, and moves protocol success under injected failures from 45.3% to 48.0%, a gain within one standard error.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agentic, VLA, Orchestration, Anomaly Detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DKBJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ARGO-VLA, a framework for improving frozen VLA policies in laboratory manipulation tasks. The method combines a lightweight residual policy for correcting VLA actions with a trust gate that limits corrections to familiar states. It also introduces a calibrated novelty monitor that can disable residual corrections and trigger escalation when anomalies are detected. At the protocol level, an LLM-based state machine handles planning, validation, execution, and recovery. The authors evaluate the approach on AutoBio and their proposed AnomalyBio benchmark, showing promising results in action correction and multi-step protocol execution.

Critical strengths
  1. The paper addresses an important problem in VLA-based robotic control, especially for laboratory tasks where small action errors can lead to failure. I like the idea of correcting a frozen VLA with a lightweight residual policy rather than fine-tuning the entire model. The separation between low-level correction and high-level orchestration is also well motivated.

  2. The use of conformal prediction to control false escalations is mathematically grounded, and the paper clearly discusses the assumptions and limitations of the resulting guarantee.

  3. This paper offers a benchmark built on the AutoBio simulator for failure detection and recovery in laboratory protocols.

Critical weaknesses
  1. Runtime efficiency: The method introduces additional components, including DINOv2 features, kNN-based gating, anomaly monitoring, and LLM orchestration. I'm wondering how much extra runtime overhead these components introduce. Could the authors report the end-to-end inference latency and computational cost compared to the original frozen π0? This would help clarify whether the method is practical for real-time robotic control.

  2. Limited analysis of residual correction errors and generalization: While the paper reports improvements in task success rates, I'd like to see a more detailed analysis of how much the residual actually reduces action or positioning errors, especially for precision-sensitive tasks. I'm also curious whether these improvements generalize to tasks with different failure modes. Additionally, could the authors analyze whether correction errors or state deviations accumulate over longer task horizons?

  3. The anomaly detection guarantee is relatively limited: While the conformal calibration provides a bound on false escalations for nominal successful skill calls, it does not guarantee reliable failure detection or necessarily apply after replanning. The reported detection rate is only 52% for monitor-reachable injected failures. Could the authors analyze how the detection rate changes under different anomaly types, severities, and longer protocols? I think this would give a clearer picture of the practical reliability of the monitor.

  4. Lack of direct comparisons with existing residual correction methods: I appreciate the detailed ablation studies, but they mainly show how the proposed components affect the overall system. The paper discusses several related residual correction methods, without directly comparing against them under the same experimental setting. I think such comparisons are important for establishing the benefit of the proposed method over existing approaches. It would also be helpful to evaluate the original frozen π0 checkpoint on the same episodes used for the residual methods.

Questions for authors

See in Weaknesses.

1 comment

Thank you for the detailed review and constructive feedback. We address the concerns following the reviewer's numbering.

W1. Runtime and computational overhead

We measured the runtime of individual components to quantify the computational overhead of our approach. The π0 policy query takes 745 ms on an NVIDIA GB10 using float32 inference, with shared GPU resources and network communication. Without GPU sharing, the query time decreases to 470 ms.

The additional components introduce only 36 ms of overhead per action chunk, approximately 4.9% of the π0 query time, primarily from DINOv2 feature extraction (36.0 ms). The monitor itself requires only 0.003 ms per control step, making its computational overhead negligible compared to policy inference.

The high-level LLM is invoked 31 times per protocol on average, including planning and recovery/replanning calls.

W2. Residual correction performance and magnitude

Using the same base-policy checkpoint, residual correction improves success on both evaluated tasks:

MethodTransferPick up
Base policy (δ = 0)49.0%65.0%
Base + R(RL)54.8%68.8%
Absolute improvement+5.8 pp+3.8 pp
Relative improvement+11.8%+5.8%

W4. Comparison with other residual methods

ResiP and Policy Decorator were not included in our experimental comparisons yet, but are doing them and will attach them.

Reviewer_7HLJOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ARGO-VLA, a framework combining residual correction of a frozen VLA policy, novelty-based gating, and escalation to a high-level agentic orchestrator. A bounded residual modifies action chunks, while an anomaly monitor suppresses corrections and triggers replanning when persistent deviations are detected. Experiments are conducted in AutoBio and the proposed AnomalyBio extension.

Critical strengths
  1. The paper addresses both execution errors within individual skills and failures across multi-step laboratory protocols. Studying how these levels interact is practically important.
  2. AnomalyBio introduces controlled failure injections and protocol-level evaluations. The authors report component ablations and explicitly acknowledge uncontrolled comparisons and effects that are not distinguishable from experimental uncertainty.
Critical weaknesses
  1. The residual produces a single correction shared across a one-second action chunk, which limits its ability to address time-varying errors and contact-dependent adjustments. The paper does not establish that the dominant failures are well approximated by such corrections, or compare against simpler alternatives such as shorter execution chunks. Freezing the VLA and training a lightweight residual is potentially economical, but the paper does not establish its effectiveness relative to directly improving the base policy.
  2. The monitor measures deviation from nominal behavior, rather than identifying whether a failure can be resolved through local correction, replanning, or the available skill library. Escalation may therefore transfer control to an orchestrator that cannot address the underlying execution limitation. Although abort conditions and a finite coordination budget are provided, the evaluation does not sufficiently characterize which anomalies are recoverable or how unrecoverable situations are recognized and handled.
Questions for authors
  1. Can the authors evaluate the same frozen checkpoint with and without the residual on identical episode seeds? Under comparable data and compute budgets, how does residual adaptation compare with further fine-tuning of the base policy, and what measurable performance or efficiency advantage justifies the additional module?
  2. Which anomaly types can actually be resolved by the high-level orchestrator, and which remain beyond the capabilities of the available skills?
  3. What evidence shows that the proposed gates select the appropriate intervention for a detected anomaly? Under matched failure states and intervention budgets, how does the full system compare with local correction alone, high-level recovery alone, and simpler escalation rules such as timeout- or tool-error-triggered replanning?
1 comment

Thank you for the detailed review and constructive feedback. We address the concerns and questions below, including additional evaluations and ablation results.

Chunk-Constant Correction

The residual correction improves task success by 5.8 percentage points on Transfer and 3.8 percentage points on Pick up, compared to the same base-policy checkpoint without residual correction.

Additional evaluations:

  • Shorter execution horizon (25 steps): Transfer success is 45%, compared to 46% with the original horizon.
  • Noise-space variant: Transfer success is 48.0%, compared to 49.0%, evaluated across three seeds.

Responses to Questions

Q1. Residual contribution and matched-budget fine-tuning

We evaluated the residual policy against the same base-policy checkpoint to isolate the contribution of residual correction.

MethodTransferPick up
Without residual49.0%65.0%
With residual54.8%68.8%
Improvement+5.8 pp+3.8 pp

These results demonstrate improvements on both tasks under a controlled base-policy comparison.

Q2. Recovery by anomaly type

AnomalyBio considers five anomaly types. The planned evaluation comprises 600 runs and 240 injected anomalies (ρ = 0.2).

Anomaly typeInjectedDetected by monitorRecoveredEnded in abort
F1: Displaced tube (2–5 cm)2411 (45.8%)10 (41.7%)5 (20.8%)
F2: Dropped tube7254 (75.0%)22 (30.6%)21 (29.2%)
F3: Tool error48Tool-layer signal24 (50.0%)10 (20.8%)
F4: Wrong labware48Tool-layer signal0 (0.0%)34 (70.8%)
F5: Camera occlusion (≥2 chunks)4840 (83.3%)34 (70.8%)6 (12.5%)
Total240—90 (37.5%)76 (31.7%)

Recovery is highest for camera occlusions (F5), followed by tool errors (F3) and displaced tubes (F1). Dropped tubes (F2) are expected to be more challenging. Wrong labware (F4) is considered unrecoverable because no available skill can resolve the anomaly; the intended behavior is to terminate safely through an abort.

Q3. Contribution of local correction and high-level recovery

We report the following results for the different correction and recovery configurations:

ConfigurationReported result
Local correction only32.9
Timeout- or tool-error-triggered replanning35.4
High-level recovery without residual37.1
Full system39.3

The full system achieves the highest reported result, exceeding local correction alone by 6.4 points, timeout- or tool-error-triggered replanning by 3.9 points, and high-level recovery without residual correction by 2.2 points.

Reviewer_tuX6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ARGO-VLA, a framework that improves frozen Vision-Language-Action (VLA) policies for laboratory automation. It addresses two types of failures: small errors during individual robot skills and unexpected failures across multi-step tasks. The main ideas are (1) learning a residual policy to correct the actions of a frozen VLA model, with a k-NN trust gate to limit corrections to familiar states, and (2) using a calibrated novelty score to detect unusual states, stop failing skills, and trigger an LLM-based system to replan. The method is evaluated on eight manipulation tasks in AutoBio and on AnomalyBio, a new benchmark with injected failures and unsafe or infeasible goals.

Critical strengths

The paper presents a well-organized framework that combines low-level action correction with high-level planning and recovery. Using reinforcement learning at the chunk level reduces the number of decisions and makes training more practical. The conformal calibration of the escalation threshold is also a useful design choice, as it provides a bound on false escalations for nominal successful skill calls. I also appreciate that the authors clearly acknowledge the limitations of their experiments and avoid overstating results that are within statistical uncertainty.

Critical weaknesses
  1. Limited responsiveness during execution. The residual applies the same action correction throughout an entire 1-second chunk. This limits its ability to respond to changes during execution, especially for contact-rich tasks such as screwing and unscrewing tube caps. The visual novelty score is also based on an image captured at the start of each chunk, so new visual disturbances may not be detected immediately. Although the monitor runs at 50 Hz, its visual component is updated only once per chunk.
  2. The benefit of the k-NN trust gate is unclear. The paper does not provide enough evidence that the trust gate improves performance. Hypothesis H2 predicts that removing the gate will hurt performance when the residual bound (\alpha) is large, but all experiments use only (\alpha=0.08). At this setting, removing the gate changes success rates by at most about one percentage point, which is within the reported standard error. Experiments with larger residual bounds are needed to show when the gate is actually useful.
  3. The main baseline comparison is not controlled. In Table 2, the residual policies are evaluated on 100 shared episode seeds across three training seeds, but the base (\pi_0) results are taken from the original AutoBio paper. These results use different checkpoints and evaluation episodes. Since the authors already have their own fine-tuned (\pi_0) checkpoint, they should evaluate it on the same 100 episode seeds to provide a fair comparison. Also, pooling the results from three residual seeds and reporting a binomial standard error does not capture variation between training seeds or the dependence caused by using the same episode seeds.
  4. The benefit of escalation is not clearly demonstrated. In Table 3, Agentic+E achieves 48.0% success compared with 45.3% for Agentic. This difference is small relative to the reported standard error. The ablation results also show no clear improvement from adding escalation. One reason is that the orchestrator already receives a failure flag when a skill times out, and the reviewer automatically triggers replanning when a tool reports failure. Therefore, the system can recover from some failures even without the novelty monitor or LLM reviewer. In addition, the injected failures occur only at chunk boundaries, which makes detection easier and does not fully test the monitor's response to disturbances occurring in the middle of a chunk.
  5. AnomalyBio needs a clearer description. The paper does not provide enough detail about the multi-step protocols used in AnomalyBio. It would help to list the tasks, the number of steps in each protocol, and how the 150 evaluation runs are divided between injected failures and goal probes.
  6. The base-policy comparison in Table 7 is not fair. The evaluation uses only 10 episodes for (\pi_0) and GR00T N1.5 on an easy lid task, while Cosmos 3 Edge and OpenVLA are evaluated on 100 episodes using CPU-fitted proprioceptive heads without their visual backbones. These differences make the comparison difficult to interpret.
  7. Some implementation details are unclear. Figure 2 shows both external and wrist cameras, but Table 4 reports a single 384-dimensional DINOv2 embedding without explaining how the two camera views are combined. There are also a few minor errors, including the duplicated “20” in the time-step description, a missing (\beta) symbol, SUCCESS(m) instead of SUCCESS(sim) in Algorithm 1, and unreachable routing rules in Table 1.
Questions for authors
  1. Could you evaluate your own fine-tuned (\pi_0) checkpoint without the residual ((\delta_j=0)) on the same 100 episode seeds used in Table 2? Please also report the variation across the three residual training seeds.

  2. Could you evaluate different values of the residual bound (\alpha) to test Hypothesis H2 and show when the k-NN trust gate becomes necessary?

  3. How much does escalation improve performance when the orchestrator cannot access the simulator's ground-truth failure flag at timeout? Alternatively, could you report how much execution time early escalation saves? A clearer breakdown of the multi-step protocols and evaluation runs in AnomalyBio would also be helpful.

1 comment

Thank you for the detailed review and constructive feedback. We address each weakness and question below.

Responses to Weaknesses

W1. Responsiveness within a chunk

We evaluated a shorter execution horizon of 25 steps, which achieved 45% transfer success, compared to 46% with the original horizon.

For failures injected mid-chunk, detection performance remained comparable to injections at chunk boundaries. Detection latency increased by up to 1 s for failures observable only through the camera, and by 0.1–0.3 s for failures observable through joint states.

W2. Trust gate

The trust gate has a substantial effect on performance, particularly when the base policy is already highly successful. In the open-lid setting, where the base policy achieves 99% success, we evaluated the gate at α = 0.08:

  • With trust gate: 89% success.
  • Without trust gate: 0% success.

These results demonstrate the importance of the trust gate in preventing unnecessary residual corrections and preserving successful base-policy behavior.

W3. Uncontrolled baseline and pooled standard error

To provide a controlled comparison, we evaluated the residual policy using our own base-policy checkpoint with δ = 0.

MethodTransferPick up
Base (δ = 0)49.0%65.0%
Base + R(RL)54.8%68.8%
Improvement+5.8 pp+3.8 pp

Uncertainty was evaluated using paired 95% intervals over 300 shared scenes and three residual seeds. This comparison isolates the contribution of the residual policy while using the same base-policy checkpoint.

W4. Benefit of escalation using our own weights

Using our own weights, escalation improves task success by 2.2 percentage points and recovery by 1.8 percentage points.

Removing the privileged timeout flag still yields an improvement of 1.8 percentage points, showing that escalation remains beneficial without relying on this additional signal.

W5. AnomalyBio description

AnomalyBio comprises six multi-step protocols (P1–P6) and five injection types (F1–F5).

We will expand the benchmark description in the revised paper to include the individual steps of each protocol and the distribution of evaluation runs, making the experimental setup and evaluation procedure more explicit.

W6. Table 7 and base-policy selection

Table 7 uses different numbers of episodes, different evaluation sets, and different action-generation pipelines. Consequently, it does not provide a controlled comparison between base policies. We will remove the table and clarify the basis for selecting π0.

Our selection is instead supported by the AutoBio benchmark, where all policies are evaluated under a consistent protocol. π0 achieves the highest success rates on the tolerance-limited tasks relevant to our work:

Taskπ0π0.5RDT
Transfer40.7%37.3%2.0%
Aspirate42.7%18.3%0.3%
Unscrew21.3%11.7%2.7%

The exception is Pick up, where π0.5 outperforms π0 (95.9% vs. 53.7%).

For the lid tasks previously reported in Table 7, our fine-tuned π0 achieves 100/100 successful episodes on close-lid and 99/100 on open-lid.

W7. Implementation details

The visual feature is the 384-dimensional CLS token from DINOv2 ViT-S/14, extracted exclusively from the external camera.

We have corrected the typographical errors and notation in Algorithm 1 to improve clarity.

Responses to Questions

Q1. Base checkpoint and residual seed performance

Our own base-policy checkpoint (δ = 0) achieves:

  • Transfer: 49.0%
  • Pick up: 65.0%

The individual results across three residual seeds are:

SeedTransferPick up
156.00%59.33%
250.67%73.33%
357.67%73.67%

The results across these seeds correspond to the residual-policy evaluation reported in W3.

Q2. α values and gate ablation

We evaluated α = 0.03 and α = 0.08.

For the open-lid setting at α = 0.08, the trust-gate ablation yields 89% success with the gate and 0% without it. This highlights the gate's role in preventing unnecessary corrections when the base policy already performs reliably.

Q3. Failure flag, escalation timing, and AnomalyBio breakdown

  • Without ground-truth failure flag: Escalation improves protocol success by 1.9 percentage points, demonstrating its effectiveness without privileged failure information.
  • Time saved through early escalation: Early escalation saves approximately 7 seconds per detected failure on Transfer and 11 seconds on Pick up. Without escalation, failed episodes run until their respective time limits of 15 s and 23 s. The monitor requires at least 0.2 s to trigger escalation.
  • AnomalyBio breakdown: Please refer to our response to W5 for the protocol and injection-type details to be included in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.