acceptodds
Withdrawn from ICLR 2027

Reshaping Action Flow: Behavior Correction for Flow-Matching Vision-Language-Action Policies

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models have shown remarkable capability across diverse manipulation tasks. After deployment, however, the pretrained policy may face situations unseen in training, for example an unexpected obstacle on its path it must manage to avoid on its own. Current methods either intervene at inference time, so the policy itself never learns the new behavior, or fine-tune the policy on new demonstrations, a route known to disturb the policy's original ability. In this paper, we propose a behavior correction framework for flow-matching VLA policies. Given a few correction demonstrations, it learns the correction through a guided training target and maintains the model's original ability through a gradient projection method. We further exploit the locality of the correction, giving more protection to the frames with larger correction and less to the rest, so the update interferes less with the original ability. Extensive experiments on robot manipulation verify the effectiveness of our method in correcting a deployed policy while preserving its original ability, with locality further strengthening the protection.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Flow Matching, Model Editing, Continual Learning, Gradient Projection, Imitation Learning, Safe Robot Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_stxtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to correct a deployed flow-matching VLA policy using demonstrations while preserving its existing capabilities. It combines an extrapolated velocity target with retained-feature gradient projection and adjusts the projection strength across correction frames. Experiments on SafeLIBERO and VLA-Arena show improved retention relative to supervised fine-tuning, with a trade-off in correction performance.

Critical strengths

The paper addresses a useful deployment problem and demonstrates meaningful retention gains across two policy architectures. For example, on SafeLIBERO, QwenGR00T retention improves from 74.9% with SFT to 85.3%, while safe success remains comparable. The method introduces no additional inference cost. The component ablations are informative: adaptive projection improves retention from 80.6% to 85.3% over fixed projection, with similar safe success.

Critical weaknesses

Critical Weaknesses

  1. The added value over established retention methods is not sufficiently demonstrated. The approach builds on GPM, with the main additions being the guided target and adaptive protection. The fixed-projection ablation is useful, but the paper lacks comparisons with replay, output distillation, or the parameter-merging approach already discussed in its related work. Such comparisons matter because the method accesses the original retain dataset. The adaptive mechanism is also mainly analyzed on one model and benchmark, without reported variation across training seeds.
  2. The exact-preservation claim in Section 3.3 and Appendix B.3 needs qualification. The basis parameterization preserves permanently masked, zero-initialized columns under a fixed mask. With sample-dependent ranks, however, other samples can update columns protected for a given sample, and existing AdamW momentum can affect updates even when the current gradient is masked. Per-sample gradient filtering therefore does not establish that the accumulated parameter change is zero in every sample’s protected subspace. This does not invalidate the empirical gains, but the claimed guarantee is too broad. Relevant references: Gradient Projection Memory; Robust Finetuning of VLA Robot Policies via Parameter Merging. Both concern approaches already discussed by the manuscript; the concern is the comparison, not missing citations.
Questions for authors

Questions For Authors

  1. Under adaptive ranks, what exactly is guaranteed: a constraint on each sample’s gradient, the optimizer update, or the accumulated parameter change? How is optimizer momentum handled?
  2. Which baselines use the retain dataset during adaptation, and how were their hyperparameters selected?
  3. Are results from additional training seeds already available for fixed versus adaptive projection? These would help assess whether the reported improvement is stable.
Reviewer_5qvGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method for adapting flow-matching VLAs to new requirements while preserving their original capabilities. The method uses guided velocity targets to learn the desired behavior from demonstrations and adaptive gradient projection to reduce forgetting. Experiments on SafeLIBERO and VLA-Arena show better retention than standard fine-tuning, with modest improvements in overall performance.

Critical strengths
  1. Well-motivated problem. Adapting a pretrained VLA to new situations without losing its existing capabilities is an important and practical problem.

  2. Thorough ablations. The paper provides useful ablations to understand the effects of different components of the method.

Critical weaknesses
  1. Modest empirical improvements. While the method improves retention, the overall gains over standard fine-tuning are relatively small. I'm not convinced that the added complexity is necessary, especially without comparisons to simpler alternatives such as KL regularization or residual policies.

  2. Limited generality. The method only modifies a few linear layers in the action expert, while keeping the VLM backbone frozen. I'm not sure how well this would generalize to other types of behavior changes, especially those that require changes in semantic understanding, such as adapting to user preferences.

Questions for authors
  1. How does the method compare to simpler approaches, such as fine-tuning with KL regularization, replay, or residual policies? It would be helpful to better understand whether forgetting is a significant issue that these alternatives cannot adequately address.

  2. How do you decide which capabilities to retain? In practice, how would you select the retain dataset for a general-purpose VLA with many different capabilities?

  3. Can the method handle behavior changes that require updating the VLM backbone, such as new user preferences or semantic instructions? How broadly do you expect this approach to generalize beyond the settings evaluated in the paper?

Reviewer_LRAGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors fine-tune a flow-matching VLA on correction demonstrations while limiting forgetting of previously learned tasks. They combine a previously introduced method, Gradient Projection Memory, which restricts weight updates along input directions that matter for the tasks to retain, with an adaptive gate which regulates the protection strength per training frame, depending on how far the frozen policy's prediction is from the demonstrated action. The method is simple in deployment because the edit is merged into the model's weights, so the resulting policy does not require any architectural changes or additional components at deployment.

Specifically, for each edited layer the authors first derive an orthonormal basis of the layer's input space on the retain data, and the leading directions are taken as the ones crucial for maintaining current capabilities. The model is then trained on the new demonstrations with a modified velocity target which extrapolates beyond the demonstrated velocity, away from the frozen policy's prediction, with each gradient being projected onto the non-protected subspace.

The method is evaluated using publicly available π0.5 and QwenGR00T checkpoints pretrained on LIBERO, using two benchmarks: SafeLIBERO and VLA-Arena. The correction dataset consists of the official VLA-Arena L0 training demonstrations and, for SafeLIBERO, of successful collision-free rollouts collected by the authors under AEGIS. Capability retention is evaluated as success rate on the original LIBERO tasks. The results show gains in retention, while limiting the negative impact on success rate on the edited tasks.

Critical strengths

The problem is well-motivated and there is a compelling story of benefits for deployment. The method appears well-suited for increasing retention of previous capability.

Critical weaknesses

Uncertainty is not reported. Confidence intervals, the number of independent training runs with different seeds, and the model selection protocol are not described. This is crucial because, while the retention gains are large, the gains in the combined metric H over the best fine-tuning baseline are small (about 0.6 to 2.6 points), and many differences in the analysis tables (Tables 2 to 4) are within a few points.

The relevant baselines which would help isolate the contribution of the paper are missing. Ordinary SFT (η = 1) with a fixed retain-subspace projection is not reported; Table 2(a) gives η = 1 only together with the gate, and Table 3 adds the projection only at η = 1.1. Also relevant are the established cheap alternatives, like RETAIN-style interpolation between weights of the base and fine-tuned models (https://arxiv.org/abs/2512.08333), which the paper cites (Yadav et al., 2026) but does not compare against; replay of the retain dataset during fine-tuning (the data which the method already requires access to), which showed strong results in a similar setting (https://arxiv.org/abs/2603.03818); and LoRA or other low-rank adapters on the same layers, which by construction limit the scope of the edit.

Questions for authors
  • COAST improves SR but not S∧S (on π0.5 SafeLIBERO-Goal, SR rises from 50.0 to 66.0 while Safe and S∧S are unchanged). Were the conceptors fitted with task success or safe success (S∧S) as the label?
  • Results are reported only for the easiest level of each benchmark, and every evaluated suite is covered by demonstrations in D_edit. What evidence is available of transfer to held-out scenarios, such as SafeLIBERO Level II or VLA-Arena L1/L2? Are the evaluation initial states disjoint from those used to collect the demonstrations and for hyperparameter tuning? How were the splits chosen?
  • What are the full configurations of the baselines, in particular SFT and ConSFT? What was the ConSFT temperature schedule and minimum weight? Do the baselines share the optimizer, learning-rate schedule, number of steps, and batch size given in Section 4.1?
  • What were the criteria for checkpoint selection within a given training run and for hyperparameter selection for particular methods (η and ϵ0 differ between models and benchmarks)?
  • The AEGIS authors report a collision-avoidance rate of 68.9% and a task success rate of 67.5% for AEGIS on π0.5, and 17.1% / 57.8% for the base policy. In Table 5 here, AEGIS reaches a Safe rate of 60.8 overall and 30.5 on the Object suite at Level I, and the base policy reaches 8.8 / 51.7 (Table 1). The levels are different, but the gap is large. What explains it?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.