- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose an action correction module (COCOA) for a pretrained VLA. The module is trained in two stages: 1) supervised fine-tuning to generate an expert action chunk conditioned on a proposed action chunk, and 2) DPO-style optimization to prevent the module from over-correcting. The training dataset contains both optimal and sub-optimal action chunks. The sub-optimal trajectories come from a weakly fine-tuned VLA, while human-collected trajectories serve as the optimal ones.
COCOA is applied to three VLAs, and the resulting policies are evaluated in two simulated environments (CALVIN and Simpler) and a real-world setting. VLAs with the action correction module outperform the plain VLAs.
Critical strengths
None
Critical weaknesses
The authors frame the method as “action correction,” but the correction happens before execution. To justify this framing, the policy would need to be aware of the failure, for example through a world model. Instead, the method seems to refine the policy by steering action generation toward the distribution of expert action trajectories. This raises the question of why one would not simply apply SRPO fine-tuning to the pretrained policy directly, and how the proposed approach differs from doing so.
The experiments compare the vanilla VLA with the VLA augmented by COCOA, but they do not include other action correction methods as baselines. Although the improvement over the vanilla VLA is noticeable, it is unclear how the method compares to related work. Moreover, the paper lacks ablations on the training components, so it is unclear which parts of the training recipe contribute to the improvement.
Questions for authors
-
How does your method compare to other action correction methods?
-
Could you provide intuition for why it is better to train a separate module that corrects the policy’s output, rather than directly fine-tuning the policy with preference optimization?
-
Do you have to train COCOA module separately for each VLA? This is a potential strength of the paper, but it is unclear from the writing.
Dear Reviewer,
Thank you for your review. We are adding more experiments based on your feedback, but we have a question about one point in your review. Could you please clarify what you mean by: "A comparison with a second VLA without action-proposal conditioning is also necessary to establish that explicitly conditioning on the proposed action is beneficial."?
In our current framework, both Stage I SFT and Stage II Correction Preference Optimization explicitly condition on the proposed action chunk. We are considering three possible interpretations of your comment:
Direct preference optimization without action-proposal conditioning: Fine-tune a separate VLA using the same action preference pairs, but with a standard preference optimization objective (e.g., Flow-DPO) rather than our action-conditioned Correction Preference Optimization, since Flow-DPO does not require action-proposal conditioning.
Inference-time ablation: Use our already-trained corrective VLA, but replace the proposed-action input with random noise or action chunks from other observation frames during inference to evaluate whether the corrective policy relies on the proposed action.
Standard SFT without action-proposal conditioning: Fine-tune a separate VLA using the standard flow-matching objective on expert demonstrations, without conditioning on proposed actions. However, the base VLAs reported in Tables 1 and 2 have already been fine-tuned on expert demonstrations using standard flow matching, so we would appreciate clarification on how you envision this baseline differing from the existing base VLAs.
Could you please clarify which of these three experimental setups you had in mind, or whether you meant a different comparison?