- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a framework for improving a base policy's task execution without updating its weights, which may be inaccessible, computationally expensive to modify, or risk degrading its pretrained capabilities. The method first detects out-of-distribution actions using an expert-trained flow model, then selectively corrects them through two approaches: (1) GRIP, which partially noises the base policy's actions and denoises them toward the expert distribution while regularizing toward the original actions, and (2) LIFT, which uses the difference between the base policy's and expert's learned velocity fields to guide action correction while preserving the original intent. Experiments in simulation and real-world settings demonstrate that these corrections improve task success without modifying the base policy.
Critical strengths
As model size continues to grow in robotics, adapting them without updating their weights becomes increasingly important. The paper demonstrates the feasibility and effectiveness of this approach through both simulation and real-world experiments.
Critical weaknesses
-
Unclear benefit of OOD detection: Although selective intervention is presented as a core contribution, Figure 5 shows only marginal improvements or even performance degradation with OOD gating in some tasks. While Table 2 reports strong OOD detection performance, it is unclear why this does not translate into larger gains in task success. Could the authors clarify this discrepancy? One concern is that the injected perturbations in Table 2 may be sufficiently obvious to make OOD detection relatively easy, potentially limiting the relevance of these results to realistic policy errors.
-
Inconsistent performance across methods and benchmarks: In some benchmarks (e.g., Slalom and Keypad in Figure 5), GRIP underperforms existing baselines. Moreover, the relative performance of GRIP and LIFT varies across tasks, making it unclear which method to choose without evaluating both, which could be costly. Could the authors explain these performance differences? Potentially providing practical guidelines for selecting between the two methods could be helpful.
Questions for authors
- Generalization of the expert prior: As I understand it, the expert prior is trained separately on target-task demonstrations. Could the proposed methods also work with a general expert prior trained across diverse tasks? This setting would be more broadly applicable and would better demonstrate whether the method can preserve the agent's task intent solely from its proposed action chunk, without explicit goal conditioning.
- OOD detection and selective intervention: In realistic settings, under what conditions would some action chunks be OOD while others remain in-distribution? I would expect OOD actions to occur primarily when the policy encounters unseen states, potentially affecting consecutive chunks. Providing quantitative analysis of OOD intervention frequency and qualitative examples showing when flagged actions are actually erroneous would be helpful. Additionally, could switching between corrected and uncorrected chunks introduce temporal inconsistencies or unstable behavior?
- Data requirements for LIFT: How many base-agent rollouts are required to train LIFT? Reporting this would help readers assess its practical applicability and data efficiency.
- Comparison with fine-tuning: Could the authors include a baseline that fine-tunes the base policy using the same number of expert demonstrations as GRIP or LIFT? Although the proposed methods do not necessarily need to outperform fine-tuning, such a comparison would help contextualize their performance and the benefits of avoiding weight updates.
6 comments
Benchmark Comparision and Adaptor Selection (W2)
- "In some benchmarks (e.g., Slalom and Keypad in Figure 5), GRIP underperforms existing baselines..."
We agree that GRIP can underperform the baselines. In Figure 5, this occurs on VLA-Slalom, where GRIP reaches 30.56% versus 33.47% for DDPM and 32.92% for CSA. On this panel, however, none of the non-LIFT methods improves the agent: DDPM, CSA, and GRIP all remain within 2.5 percentage points of unassisted execution (33.06%), whereas LIFT reaches 76.67%. As discussed in Section 6.2, Slalom's route choice is only weakly expressed in a short action chunk, so its intent cues are vulnerable to partial noising; LIFT retains them through source-to-expert transport without partially noising the editing state.
- "… making it unclear which method to choose without evaluating both"
By default, we recommend GRIP as the adaptation method, as it is more lightweight and achieves comparable performance on most tasks. However, when the base policy (RL, IL, or VLA) can be queried offline, we recommend LIFT, as it offers more substantial improvements on certain tasks, particularly those where preserving the agent's intention is important, such as Slalom. We will make it more clear in our paper.
A Cross-task Expert Prior (Q1)
- "… Could the proposed methods also work with a general expert prior trained across diverse tasks? This setting would be more broadly applicable …"
First, we acknowledge that our method can be directly applied to multi-task settings, as our adaptation module does not assume any prior knowledge of the goal, intention, or task description. However, we did not explore multi-task settings in this work, as the tasks we considered involve different action spaces and robot embodiments. Naively merging datasets across different embodiments is unlikely to yield good results. We thank the reviewer for this valuable suggestion. Universal cross-embodiment adaptation is a very interesting research direction, and we would like to explore it in future work.
- "… would better demonstrate whether the method can preserve the agent's task intent solely from its proposed action chunk, without explicit goal conditioning."
We thank the reviewer to provide a better experiment plan to demonstrate goal-agnostic. Importantly, we want to clarify that task-specific does not mean goal conditioned. In none of our settings can the goal be inferred from the image alone and the language instruction is excluded from the adaptive policy: our priors cover 24 Slalom routes, 4 CAN targets, and 729 Keypad sequences, without explicit goal conditioning. The action proposal must distinguish these intentions, so the current experiments have shown intent preservation beyond a fixed-goal setting.
Controlled Perturbations and OOD Contribution (W1, Q2)
We thank reviewer for the constructive comments.
High detection accuracy does not imply higher success, because the gate can only remove edits. We acknowledge that gating lowers success in some settings of Figure 5. A gated policy can outperform the ungated one only through the chunks it leaves unedited. Accurate detection ensures that these chunks are close to the expert distribution, and editing such chunks changes them little. We therefore attribute the success gains to the editors. The gate's role is to avoid overriding reasonable actions and to save computation, not to add success on its own.
Our practical evidence is not limited to injected noise. Figure 5 evaluates VLA and Code-as-Policy agents without wrappers, and the real-robot trials evaluate the fine-tuned VLA's own execution errors. The existing VLA rollout records show this directly, without injected perturbations:
Both settings use ten seeds. The final column is the fraction of gated rollout queries that skip the editing solver. These settings show avoided editing work alongside higher mean success. Preserving reasonable commands also matters in human shared autonomy, where unnecessary correction would override the operator's control. Controlled perturbations serve a complementary purpose: they provide externally assigned, reproducible labels for measuring detection. An episode's failure alone cannot identify which individual action chunk was erroneous.
The detector evaluates actions conditioned on (Image, action) pairs, not state novelty alone. A familiar state can receive an inappropriate offset, magnitude, or delayed command; consecutive chunks can therefore receive different decisions. Figure 6 illustrates this distinction by separating Shift-perturbed Keypad chunks from normal chunks. Conversely, Laggy's repeated command can remain appropriate, explaining why detecting every wrapper action is not the objective.
OOD gating has negligible time effect. Switching occurs at existing policy-query boundaries, with unchanged action interfaces and execution frequency. Table 15 reports full correction-query latency, including observation conditioning: 153.60 ms for GRIP and 202.11 ms for LIFT at batch 1 on an RTX A6000, using respectively 2.5% and 17.8% of one FLOWER query's FLOPs. One-step OOD gating takes 0.356 ms per sample (Table 12), reusing the expert flow without a separate detector.