acceptodds
Under review as a conference paper at ICLR 2027

COCOA: Action-Conditioned Correction for Vision-Language-Action Policies

Abstract

Vision-language-action (VLA) models have shown strong performance on complex robot manipulation tasks by generating low-level action chunks from multimodal observations and language instructions. However, an erroneous action chunk can derail long-horizon execution and lead to task failure. To address this challenge, we propose , an action-conditioned correction training framework that learns a separate corrective policy to revise action chunks proposed by a base VLA before execution. Given the current visual observations, language instruction, robot state, and a proposed action chunk, the corrective policy generates a corrected action chunk for execution. To train the corrective policy, introduces a two-stage training procedure that combines supervised flow-matching training with correction preference optimization for flow-based VLAs. We apply to three flow-based VLA backbones, including , FLOWER VLA, and Xiaomi-Robotics-0. Our experiments show that achieves consistent improvements on two simulation benchmarks, SimplerEnv and CALVIN, as well as on real-robot tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Robot Learning, Language-conditioned Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GXX3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose an action correction module (COCOA) for a pretrained VLA. The module is trained in two stages: 1) supervised fine-tuning to generate an expert action chunk conditioned on a proposed action chunk, and 2) DPO-style optimization to prevent the module from over-correcting. The training dataset contains both optimal and sub-optimal action chunks. The sub-optimal trajectories come from a weakly fine-tuned VLA, while human-collected trajectories serve as the optimal ones.

COCOA is applied to three VLAs, and the resulting policies are evaluated in two simulated environments (CALVIN and Simpler) and a real-world setting. VLAs with the action correction module outperform the plain VLAs.

Critical strengths

None

Critical weaknesses

The authors frame the method as “action correction,” but the correction happens before execution. To justify this framing, the policy would need to be aware of the failure, for example through a world model. Instead, the method seems to refine the policy by steering action generation toward the distribution of expert action trajectories. This raises the question of why one would not simply apply SRPO fine-tuning to the pretrained policy directly, and how the proposed approach differs from doing so.

The experiments compare the vanilla VLA with the VLA augmented by COCOA, but they do not include other action correction methods as baselines. Although the improvement over the vanilla VLA is noticeable, it is unclear how the method compares to related work. Moreover, the paper lacks ablations on the training components, so it is unclear which parts of the training recipe contribute to the improvement.

Questions for authors
  • How does your method compare to other action correction methods?

  • Could you provide intuition for why it is better to train a separate module that corrects the policy’s output, rather than directly fine-tuning the policy with preference optimization?

  • Do you have to train COCOA module separately for each VLA? This is a potential strength of the paper, but it is unclear from the writing.

Reviewer_Ncg2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes COCOA, an action-conditioned correction framework for vision-language-action (VLA) policies. Instead of directly executing action chunks generated by a pretrained VLA, COCOA introduces a separate corrective policy that takes the current observation, language instruction, robot state, and proposed action chunk as inputs to generate a revised action chunk before execution. The framework employs a two-stage training strategy consisting of supervised flow-matching fine-tuning and correction preference optimization. The authors evaluate COCOA on three VLA backbones (π0.5, FLOWER VLA, and Xiaomi-Robotics-0) across CALVIN, SimplerEnv, and real-world robot manipulation tasks. The reported results demonstrate enhancements in aggregate success rates, particularly on challenging long-horizon manipulation tasks.

Critical strengths

Strengths:

  1. A practical and modular framework for improving pretrained VLA policies.

  2. Broad empirical evaluation across different VLA backbones and benchmarks.

  3. The inclusion of execution-time analysis also provides useful information about the practical overhead of introducing a separate corrective model.

Critical weaknesses
  1. Lack of contributional gains from isolated components. A major weakness/concern is the absence of key ablation experiments. COCOA combines an additional corrective VLA, action-conditioned inputs, supervised fine-tuning, and correction preference optimization. However, the experiments mainly compare the complete framework against the original VLA backbones. For instance, the current manuscript does not provide an SFT-only baseline to demonstrate the independent contribution of Stage. A comparison with a second VLA without action-proposal conditioning is also necessary to establish that explicitly conditioning on the proposed action is beneficial.

  2. Experimental support for the claims that preference optimization helps prevent unnecessary modifications to already desirable actions.

  3. Preference-label reliability and prevention of over-correction. The training procedure assumes that actions generated by the weak VLA are inferior to the corresponding expert demonstrations. The reviewer wonders how reliable this assumption is, given that multiple action chunks may be valid for the same observation.

Questions for authors

Please check the weakness. The reviewer is willing to modify the final rating based on the rebuttal effort and results from the authors.

3 comments

Dear Reviewer,

Thank you for your review. We are adding more experiments based on your feedback, but we have a question about one point in your review. Could you please clarify what you mean by: "A comparison with a second VLA without action-proposal conditioning is also necessary to establish that explicitly conditioning on the proposed action is beneficial."?

In our current framework, both Stage I SFT and Stage II Correction Preference Optimization explicitly condition on the proposed action chunk. We are considering three possible interpretations of your comment:

  1. Direct preference optimization without action-proposal conditioning: Fine-tune a separate VLA using the same action preference pairs, but with a standard preference optimization objective (e.g., Flow-DPO) rather than our action-conditioned Correction Preference Optimization, since Flow-DPO does not require action-proposal conditioning.

  2. Inference-time ablation: Use our already-trained corrective VLA, but replace the proposed-action input with random noise or action chunks from other observation frames during inference to evaluate whether the corrective policy relies on the proposed action.

  3. Standard SFT without action-proposal conditioning: Fine-tune a separate VLA using the standard flow-matching objective on expert demonstrations, without conditioning on proposed actions. However, the base VLAs reported in Tables 1 and 2 have already been fine-tuned on expert demonstrations using standard flow matching, so we would appreciate clarification on how you envision this baseline differing from the existing base VLAs.

Could you please clarify which of these three experimental setups you had in mind, or whether you meant a different comparison?

Dear Reviewer,

We would like to follow up on our previous question. Could you please provide some clarification? We appreciate your feedback and want to make sure our additional experiments address your concerns appropriately.

Thank you!

Thank you for the authors’ response. Regarding the question raised by the authors, I would appreciate further clarification and experimental evidence for the first branch: direct preference optimization without action-proposal conditioning.

Reviewer_v6aFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

COCOA trains a separate VLA to revise action chunks before execution. The idea is clear, and the evaluation covers three backbones, two simulation benchmarks, and real-robot tasks.

Critical strengths
  • The paper tests three VLA backbones, two simulation benchmarks, and real-robot tasks.
  • The method reuses expert demonstrations to automatically construct action pairs, reducing the need to collect dedicated correction annotation
Critical weaknesses
  • COCOA adds a full corrective model and extra inference. Please compare against directly fine-tuning the base policy with the same data and comparable compute. Otherwise, the advantage of a separate corrector remains unclear.

  • The paper should test removing the candidate-action input, using supervised training alone, and removing the term intended to prevent over-correction. These comparisons would show whether the model actually uses the proposed action and whether preference optimization helps.

  • Weak-model actions are always labeled worse than expert actions without execution checks. Some may be valid alternatives. Training proposals also come from a weaker policy than the one used at deployment. Both issues need further analysis.

  • No repeated runs or confidence intervals are reported. Xiaomi’s CALVIN gain is only 0.4 percentage points, and real-robot evaluation uses 20 trials per task. More repetitions are needed to assess reliability.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.