acceptodds
Under review as a conference paper at ICLR 2027

Flow-based Policy Adaptation without Policy Updates

Abstract

Foundation models and other policies trained on large and diverse datasets can endow robots with semantic and task knowledge that would be difficult to acquire from task-specific demonstrations alone. Despite their broad scene understand- ing and reasoning capabilities, however, these models often lack the environment- and embodiment-specific knowledge required for precise and reliable execution. We propose GLOVES, a flow-based framework that complements an existing agent with a lightweight, goal-agnostic expert action prior that refines the action chunks proposed by the agent without updating its parameters. The prior captures observation-conditioned expert behavior, while the agent’s proposal supplies task- and context-specific intent that the correction aims to preserve. The learned flow also provides a natural in-distribution scoring mechanism through reverse flow evaluation. We use this signal as an intervention gate: actions that appear consis- tent with the expert distribution are passed through unchanged, while anomalous or out-of-distribution (OOD) actions are corrected. In this way, GLOVES only provides assistance when necessary. By learning local goal-agnostic expert ac- tion patterns and using them to adapt agent-proposed actions during execution, GLOVES provides a lightweight shared-control module for robust action adap- tation across tasks and environments. Across controlled OOD-recovery studies, simulation benchmarks, and real-robot experiments, GLOVES consistently im- proves the success of vision-language-action models (VLAs) and Code-as-Policy agents without retraining the underlying policy.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Policy Adaptation, Flow Matching, Shared Control

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_DtuROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a framework for improving a base policy's task execution without updating its weights, which may be inaccessible, computationally expensive to modify, or risk degrading its pretrained capabilities. The method first detects out-of-distribution actions using an expert-trained flow model, then selectively corrects them through two approaches: (1) GRIP, which partially noises the base policy's actions and denoises them toward the expert distribution while regularizing toward the original actions, and (2) LIFT, which uses the difference between the base policy's and expert's learned velocity fields to guide action correction while preserving the original intent. Experiments in simulation and real-world settings demonstrate that these corrections improve task success without modifying the base policy.

Critical strengths

As model size continues to grow in robotics, adapting them without updating their weights becomes increasingly important. The paper demonstrates the feasibility and effectiveness of this approach through both simulation and real-world experiments.

Critical weaknesses
  • Unclear benefit of OOD detection: Although selective intervention is presented as a core contribution, Figure 5 shows only marginal improvements or even performance degradation with OOD gating in some tasks. While Table 2 reports strong OOD detection performance, it is unclear why this does not translate into larger gains in task success. Could the authors clarify this discrepancy? One concern is that the injected perturbations in Table 2 may be sufficiently obvious to make OOD detection relatively easy, potentially limiting the relevance of these results to realistic policy errors.

  • Inconsistent performance across methods and benchmarks: In some benchmarks (e.g., Slalom and Keypad in Figure 5), GRIP underperforms existing baselines. Moreover, the relative performance of GRIP and LIFT varies across tasks, making it unclear which method to choose without evaluating both, which could be costly. Could the authors explain these performance differences? Potentially providing practical guidelines for selecting between the two methods could be helpful.

Questions for authors
  • Generalization of the expert prior: As I understand it, the expert prior is trained separately on target-task demonstrations. Could the proposed methods also work with a general expert prior trained across diverse tasks? This setting would be more broadly applicable and would better demonstrate whether the method can preserve the agent's task intent solely from its proposed action chunk, without explicit goal conditioning.
  • OOD detection and selective intervention: In realistic settings, under what conditions would some action chunks be OOD while others remain in-distribution? I would expect OOD actions to occur primarily when the policy encounters unseen states, potentially affecting consecutive chunks. Providing quantitative analysis of OOD intervention frequency and qualitative examples showing when flagged actions are actually erroneous would be helpful. Additionally, could switching between corrected and uncorrected chunks introduce temporal inconsistencies or unstable behavior?
  • Data requirements for LIFT: How many base-agent rollouts are required to train LIFT? Reporting this would help readers assess its practical applicability and data efficiency.
  • Comparison with fine-tuning: Could the authors include a baseline that fine-tunes the base policy using the same number of expert demonstrations as GRIP or LIFT? Although the proposed methods do not necessarily need to outperform fine-tuning, such a comparison would help contextualize their performance and the benefits of avoiding weight updates.
6 comments
AuthorsOpenReview ↗

Controlled Perturbations and OOD Contribution (W1, Q2)

We thank reviewer for the constructive comments.

  1. "… Figure 5 shows only marginal improvements or even performance degradation with OOD gating in some tasks... Could the authors clarify this discrepancy?"

High detection accuracy does not imply higher success, because the gate can only remove edits. We acknowledge that gating lowers success in some settings of Figure 5. A gated policy can outperform the ungated one only through the chunks it leaves unedited. Accurate detection ensures that these chunks are close to the expert distribution, and editing such chunks changes them little. We therefore attribute the success gains to the editors. The gate's role is to avoid overriding reasonable actions and to save computation, not to add success on its own.

  1. "One concern is that the injected perturbations in Table 2 may be sufficiently obvious to make OOD detection relatively easy, potentially limiting the relevance of these results to realistic policy errors."

Our practical evidence is not limited to injected noise. Figure 5 evaluates VLA and Code-as-Policy agents without wrappers, and the real-robot trials evaluate the fine-tuned VLA's own execution errors. The existing VLA rollout records show this directly, without injected perturbations:

SettingUngated successGated successEditing queries bypassed
VLA-Keypad, LIFT / LIFT+58.67%62.00%17.31%
VLA-CAN, GRIP / GRIP+39.38%41.25%13.00%

Both settings use ten seeds. The final column is the fraction of gated rollout queries that skip the editing solver. These settings show avoided editing work alongside higher mean success. Preserving reasonable commands also matters in human shared autonomy, where unnecessary correction would override the operator's control. Controlled perturbations serve a complementary purpose: they provide externally assigned, reproducible labels for measuring detection. An episode's failure alone cannot identify which individual action chunk was erroneous.

  1. "In realistic settings, under what conditions would some action chunks be OOD while others remain in-distribution? I would expect OOD actions to occur primarily when the policy encounters unseen states, potentially affecting consecutive chunks..."

The detector evaluates actions conditioned on (Image, action) pairs, not state novelty alone. A familiar state can receive an inappropriate offset, magnitude, or delayed command; consecutive chunks can therefore receive different decisions. Figure 6 illustrates this distinction by separating Shift-perturbed Keypad chunks from normal chunks. Conversely, Laggy's repeated command can remain appropriate, explaining why detecting every wrapper action is not the objective.

  1. "Additionally, could switching between corrected and uncorrected chunks introduce temporal inconsistencies or unstable behavior?"

OOD gating has negligible time effect. Switching occurs at existing policy-query boundaries, with unchanged action interfaces and execution frequency. Table 15 reports full correction-query latency, including observation conditioning: 153.60 ms for GRIP and 202.11 ms for LIFT at batch 1 on an RTX A6000, using respectively 2.5% and 17.8% of one FLOWER query's FLOPs. One-step OOD gating takes 0.356 ms per sample (Table 12), reusing the expert flow without a separate detector.

AuthorsOpenReview ↗

Benchmark Comparision and Adaptor Selection (W2)

  1. "In some benchmarks (e.g., Slalom and Keypad in Figure 5), GRIP underperforms existing baselines..."

We agree that GRIP can underperform the baselines. In Figure 5, this occurs on VLA-Slalom, where GRIP reaches 30.56% versus 33.47% for DDPM and 32.92% for CSA. On this panel, however, none of the non-LIFT methods improves the agent: DDPM, CSA, and GRIP all remain within 2.5 percentage points of unassisted execution (33.06%), whereas LIFT reaches 76.67%. As discussed in Section 6.2, Slalom's route choice is only weakly expressed in a short action chunk, so its intent cues are vulnerable to partial noising; LIFT retains them through source-to-expert transport without partially noising the editing state.

  1. "… making it unclear which method to choose without evaluating both"

By default, we recommend GRIP as the adaptation method, as it is more lightweight and achieves comparable performance on most tasks. However, when the base policy (RL, IL, or VLA) can be queried offline, we recommend LIFT, as it offers more substantial improvements on certain tasks, particularly those where preserving the agent's intention is important, such as Slalom. We will make it more clear in our paper.

AuthorsOpenReview ↗

A Cross-task Expert Prior (Q1)

  1. "… Could the proposed methods also work with a general expert prior trained across diverse tasks? This setting would be more broadly applicable …"

First, we acknowledge that our method can be directly applied to multi-task settings, as our adaptation module does not assume any prior knowledge of the goal, intention, or task description. However, we did not explore multi-task settings in this work, as the tasks we considered involve different action spaces and robot embodiments. Naively merging datasets across different embodiments is unlikely to yield good results. We thank the reviewer for this valuable suggestion. Universal cross-embodiment adaptation is a very interesting research direction, and we would like to explore it in future work.

  1. "… would better demonstrate whether the method can preserve the agent's task intent solely from its proposed action chunk, without explicit goal conditioning."

We thank the reviewer to provide a better experiment plan to demonstrate goal-agnostic. Importantly, we want to clarify that task-specific does not mean goal conditioned. In none of our settings can the goal be inferred from the image alone and the language instruction is excluded from the adaptive policy: our priors cover 24 Slalom routes, 4 CAN targets, and 729 Keypad sequences, without explicit goal conditioning. The action proposal must distinguish these intentions, so the current experiments have shown intent preservation beyond a fixed-goal setting.

Reviewer_CbcnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents GLOVES, a framework for improving an existing agent's execution by correcting its proposed action chunks without updating the underlying policy. It learns an image-conditioned, goal-agnostic expert action prior and introduces two flow-based editors: GRIP combines partial noising with energy-guided refinement, while LIFT learns an additional agent flow and performs source-to-expert transport with a source-consistency regularizer. The same expert flow supports OOD scoring through reverse-flow evaluation, allowing the system to select which action chunks to correct without training a separate detector. Evaluation covers FLOWER-based VLA and Code-as-Policy agents on three simulation tasks, VLA adaptation on two real-robot tasks, and controlled perturbation studies of detection and performance recovery. The reported results show substantial improvements in task success across several settings while keeping the base agents unchanged.

Critical strengths

1. A practical, modular approach to adapting existing agents. GLOVES operates on observations and proposed action chunks, without modifying the base policy or requiring a shared goal representation. Reusing the expert flow for both OOD scoring and action correction also avoids a separately trained detector. This design provides a useful way to incorporate task-specific execution knowledge into existing systems, with applicability demonstrated for both VLA and Code-as-Policy agents.

2. Substantial gains supported by real-robot validation. The evaluation includes two distinct agent types in simulation and physical execution on a UR5. In the real-robot experiments, GRIP improves charger-insertion success from 18/40 to 40/40 and cup-transport success from 2/30 to 22/30. These gains across precise insertion and obstacle-avoiding transport provide concrete evidence that the proposed action correction can improve task completion under different execution demands.

Critical weaknesses

1. Action correction may alter the goal or route conveyed by the original action chunk.

The agent's proposal is both what the adapter corrects and its main cue to the intended behavior. GRIP's energy penalty and LIFT's source-consistency regularizer constrain deviations from that proposal, but do not directly distinguish execution errors from differences that encode the agent's goal. The gains on multi-goal tasks support the framework's overall effectiveness, but do not clarify how editing balances these competing objectives. In particular, LIFT's Slalom results do not distinguish better preservation of route choice from more general improvements in execution, leaving the explanation in Section 6.2 only partly supported.

2. Conformity to the expert distribution is an indirect basis for deciding whether correction is beneficial.

The OOD gate uses this conformity to select interventions, but an anomalous action is not necessarily one that the editor can improve. The paper compares success rates with and without gating and profiles detection and editing costs. However, gating improves success in some settings and reduces it in others, without reporting the corresponding intervention rates and total inference costs. It therefore remains unclear whether selective intervention retains most of the performance gains at lower cost, and whether the computational savings justify the observed losses in success.

Questions for authors
  1. Section 6.2 suggests that LIFT better retains weak intent cues on Slalom. What evidence distinguishes better preservation of the agent's route choice from other execution improvements, such as fewer collisions?

  2. In Figure 5, enabling OOD gating reduces LIFT's success rate for the VLA on Slalom from approximately 76.7% to 64.3%. What intervention rate and computational savings accompany this decrease? How do changes in success rate compare with computational savings in the other settings?

  3. LIFT uses additional agent rollouts, and Table 9 lists a larger hidden width for the FlowAlign model than for the standard flow-matching model. What evidence separates the contribution of the editing mechanism from the effects of additional data and model capacity?

4 comments
AuthorsOpenReview ↗

Route Preservation Across Agents (W1, Q1)

1."Action correction may alter the goal or route conveyed by the original action chunk. … LIFT's Slalom results do not distinguish better preservation of route choice from more general improvements in execution, leaving the explanation in Section 6.2 only partly supported." "… What evidence distinguishes better preservation of the agent's route choice from other execution improvements, such as fewer collisions?"

We thank the reviewer for pointing out the need to distinguish intention preservation from general execution improvements. To investigate this, we re-analyzed the 720 Slalom episodes per method from Figure 5, separating successful completion along the intended route from completion along a different route.

VLA-Slalom: 720 episodes per method.

MethodCorrect-route completionWrong-route completionUnfinished
Fine-tuned VLA33.06%36.53%30.42%
VLA + LIFT76.67%13.75%9.58%
VLA + LIFT+64.31%22.22%13.47%

Importantly, the improvement is not limited to fewer collisions or higher overall completion rates. Among episodes that successfully reach the endpoint, the fraction following the wrong route drops from 52.50% for the base VLA to 15.21% with LIFT (25.68% with LIFT+). This conditional comparison separates route correctness from overall completion and provides more direct evidence that LIFT better preserves the agent's intended route, rather than simply improving its ability to reach the endpoint. We acknowledge that route preservation and execution quality are not entirely independent, and this analysis does not fully isolate their contributions. Nevertheless, the substantial reduction in wrong-route completion, even when conditioning on successful episodes, supports our explanation in Section 6.2.

AuthorsOpenReview ↗

Selective Intervention and Its Cost (W2, Q2)

1."Conformity to the expert distribution is an indirect basis for deciding whether correction is beneficial. The OOD gate uses this conformity to select interventions, but an anomalous action is not necessarily one that the editor can improve."

"… gating improves success in some settings and reduces it in others, without reporting the corresponding intervention rates … It therefore remains unclear whether selective intervention retains most of the performance gains at lower cost, …"

"… How do changes in success rate compare with computational savings in the other settings?"

We acknowledge that an anomalous action is not necessarily one that the editor can improve. For example, a purely random action without any meaningful intention may be difficult to correct. Our method assumes that the base policy can express meaningful intentions that the editor can refine. Therefore, expert conformity serves as a criterion for deciding when to intervene, rather than a guarantee that an intervention will be beneficial.

The motivation for OOD gating is to avoid unnecessary corrections when the original action already conforms to the expert distribution. Tables 2 and 13 evaluate the gate across three tasks and different perturbation wrappers, reporting macro precision of 0.975, AUROC of 0.938, and a false-positive rate of 0.003. In particular, the gate leaves 99.73% of normal-sample evaluations unedited, suggesting that it can preserve reasonable actions without invoking the editor.

To further examine the trade-off between intervention cost and task performance, we additionally measured gating in our existing VLA evaluations without injected perturbations:

SettingUngated successGated successEditing queries bypassed
VLA-Keypad, LIFT / LIFT+58.67%62.00%17.31%
VLA-CAN, GRIP / GRIP+39.38%41.25%13.00%

Both settings use ten seeds. The last column reports the fraction of editing queries bypassed by the gate. In these settings, gating reduces editing solver calls by 13–17% while slightly improving mean success. While gating does not consistently outperform always-on editing across all settings, these results show that selective intervention can reduce computational overhead without necessarily sacrificing performance. Beyond computational savings, avoiding unnecessary corrections is also desirable in human shared autonomy, where excessive intervention may override the operator's intended control.

2."… without reporting the corresponding intervention rates and total inference costs."

OOD scoring introduces negligible computational overhead and improves the success rate in some experiments. Table 15 already reports full correction-query latency, including observation conditioning: 153.60 ms for GRIP and 202.11 ms for LIFT at batch one on an RTX A6000. Their compute costs are 2.5% and 17.8% respectively of one FLOWER query's FLOPs. One-step OOD scoring takes 0.356 ms per sample (Table 12) and reuses the expert flow without a separately trained detector.

AuthorsOpenReview ↗

Source Data and the Editing Mechanism (Q3)

1."LIFT uses additional agent rollouts, … What evidence separates the contribution of the editing mechanism from the effects of additional data …"

LIFT does use additional data, but these data are neither real rollouts nor expert data.

Not real rollouts. We query the base policy offline on observations from existing recordings and save its predicted action chunks without executing them. Collecting them requires no new interaction with the environment or the robot, and the recordings need not come from the current agent or an expert.

Not expert data. The samples are the agent's own predictions. They contain no expert actions, correction labels, or rewards, so they describe only what the agent already does. The expert execution knowledge that drives the correction comes solely from the expert demonstrations, which GRIP uses as well. We collected zero additional real-robot rollouts for source-data construction. We will replace "source-agent rollouts" with "offline agent action samples" in Section 6.4.

2."… Table 9 lists a larger hidden width for the FlowAlign model than for the standard flow-matching model. What evidence separates the contribution of the editing mechanism from the effects of additional data and model capacity?"

The relevant evidence for the editing mechanism goes beyond the architecture comparison in Table 9:

  • Fixed-model ablation for guidance in GRIP (Table 10): with the same trained expert flow and starting time, enabling GRIP's alignment energy improves success in 9/12 task-wrapper settings, including 30.7% to 51.3% on Keypad-Noised. These gains require no additional parameters.
  • Learned correction baseline (Table 3, Appendix B.6): Residual uses the expert flow's architecture and parameter count, yet achieves 38.4%, 55.9%, and 53.8% on Slalom, CAN, and Keypad, versus GRIP's 41.9%, 65.9%, and 72.4%.

Thus, the gains come from the correction mechanism, not merely from adding a trainable network. The same expert demonstrations have also already been used to fine-tune the base VLA: LIFT further raises Slalom success from 33.06% to 76.67% while keeping that policy fixed.

Beyond this, model size alone does not explain the gains. The Unassisted VLA is already fine-tuned on the same expert demonstrations used by GLOVES. Before task-specific fine-tuning, its success is 0% in our setup. GLOVES then improves the fine-tuned policy without another VLA update or additional expert demonstrations:

TaskFine-tuned VLAWith GLOVES
Slalom, simulation33.06%76.67% (LIFT)
Charger insertion, real robot45.00% (18/40)100.00% (40/40, GRIP)
Cup transport, real robot6.67% (2/30)73.33% (22/30, GRIP)

Thank you for the clarifications. I will maintain my current score.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Data Cost Compared with Fine-tuning (W1, Q1)

1."… The paper would benefit from a clearer comparison between the data/training cost of GLOVES and directly fine-tuning or adapting the base policy." "How should the data cost of GLOVES be compared with directly fine-tuning the underlying policy?"

The Unassisted VLA is already fine-tuned on the same expert demonstrations used by GLOVES. Before task-specific fine-tuning, its success is 0% in our setup. GLOVES then improves the fine-tuned policy without another VLA update or additional expert demonstrations:

TaskFine-tuned VLAWith GLOVES
Slalom, simulation33.06%76.67% (LIFT)
Charger insertion, real robot45.00% (18/40)100.00% (40/40, GRIP)
Cup transport, real robot6.67% (2/30)73.33% (22/30, GRIP)

2."… the real-robot experiments use 200 expert trajectories for charger insertion and 235 for cup transport. This is a non-trivial amount of task-specific supervision. …"

All demonstrations were collected in under two hours in total. We will clarify this setup in Appendix B.5.2. We also show in the appendix that our method scales well with the amount of training data. By varying the number of trajectories, we can achieve different levels of policy adaptation performance. Therefore, 200 episodes are not strictly necessary if a lower level of performance is acceptable. We believe this is another advantage of our method, as it does not require a fixed minimum number of demonstrations to achieve reasonable performance.

3."'Without policy updates' does not imply low adaptation cost, … and training of an additional generative action prior"

Backbone-agnostic adaptation cost. One advantage of our method is that its adaptation cost is independent of the backbone's architecture and parameter count. While parameter-efficient fine-tuning methods such as LoRA can also reduce training costs, their costs still depend on the backbone's parameterization. In contrast, our method only trains lightweight action-adaptation models while leaving the base VLA untouched. Its computational cost depends primarily on the policy's inference cost rather than its model size.

Flexible data requirements. Another advantage is that our method places fewer requirements on the adaptation dataset than VLA fine-tuning. In particular, the expert demonstrations do not need to satisfy the data requirements of the base VLA's fine-tuning pipeline. For LIFT, we obtain source-agent actions by querying the base policy offline on observations from existing recordings, without executing its predictions. These recordings need not come from the current agent or an expert, and no paired agent-expert correction labels are required. The expert flow is trained using existing expert demonstrations. No additional real-robot rollouts were collected for source-data construction. We will replace "source-agent rollouts" with "offline agent action samples" in Section 6.4 to avoid confusion.

Inference overhead. Table 15 separately reports the deployment cost. GRIP and LIFT take 153.60 and 202.11 ms per query, respectively, including observation conditioning, corresponding to only 2.5% and 17.8% of the FLOPs of one FLOWER query.

AuthorsOpenReview ↗

Evaluation Breadth (W2, Q2)

1."The real-world evaluation remains relatively narrow. Both real-world tasks use the same UR5 platform and the same FLOWER-based VLA, and only two task families are evaluated. … would make the claim of broadly applicable frozen-policy adaptation substantially more convincing."

Evaluation breadth extends beyond the number of VLA backbones or robot embodiments. We acknowledge that all real-robot experiments use a single platform, a UR5 controlled by the FLOWER-based VLA. However, our evaluation covers diverse task demands and agent types, including fine-tuned VLAs, Code-as-Policy, and perturbed imitation-learning agents. The two real-robot tasks place complementary demands on the adapter: charger insertion requires precise alignment near contact, while cup transport requires sustained corrections to maintain a smooth trajectory around an obstacle. The three simulation tasks further cover route selection, pick-and-place, and sequential button presses. Together, these experiments evaluate our method across different sources of action errors and execution demands, rather than relying solely on the number of backbones or embodiments.

AuthorsOpenReview ↗

Choosing GRIP versus LIFT (Q3)

1."Could the authors clarify when one should prefer GRIP versus LIFT beyond the current empirical recommendation?"

By default, we recommend GRIP as the adaptation method, as it is more lightweight and achieves comparable performance on most tasks. However, when the base policy (RL, IL, or VLA) can be queried offline, we recommend LIFT, as it offers more substantial improvements on certain tasks, particularly those where preserving the agent's intention is important, such as Slalom. We will make it more clear in our paper.

2."In particular, is there a measurable indicator of intent ambiguity that could automatically trigger the more expensive LIFT editor?"

We thank the reviewer for this insightful suggestion. In this work, we have not yet explored how to quantify intent ambiguity or determine when such ambiguity matters for adaptation at a given state. We agree that developing a measurable indicator to dynamically select between GRIP and LIFT would be an interesting and valuable extension of our method, and we would like to explore this direction in future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.