acceptodds
Under review as a conference paper at ICLR 2027

Steering Generative Robot Policies with Optimal Transport Maps

Abstract

Generative policies have demonstrated significant potential in robotic manipulation. Despite acquiring a rich repertoire of motor skills, they remain brittle under train-test shifts: even mild changes in geometry or context can cause execution failures, limiting scalable deployment in the real world. While retraining or fine-tuning offers a remedy, it often incurs impractical costs and may impair the pretrained generalist capabilities. To this end, we propose LOTUS, a training-free framework based on Optimal Transport (OT) that adapts the behavior of frozen policies during inference. Following the VLM-guided paradigm, LOTUS first synthesizes stage-wise reward functions to evaluate candidate actions. Unlike previous efforts that use rewards for best-of-N selection or gradient perturbation, we construct virtual trajectories along high-reward directions, which can be viewed as samples from a task-aligned distribution. Our key idea is to project the original policy distribution toward such reward-induced regions while preserving the learned action prior. This can be formulated as an OT problem with motion discrepancy as the transport cost, and we further relax the target marginal to avoid over-steering toward implausible actions. We evaluate LOTUS on both simulated and real-world manipulation tasks, showing substantial improvement against diverse perturbations. The code is provided in the supplementary material.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Optimal Transport, Vision-Language-Action Models, Inference-Time Steering

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_8rF6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes LOTUS, a training-free method for steering diffusion policies. LOTUS constructs virtual actions that move the end-effector along straight lines toward the task keypoints. Both these virtual chunks and samples from the policy are noised to an intermediate denoising time, and a semi-unbalanced optimal transport problem is solved between them. The resulting transport plan gives a refined latent, which the policy then denoises into the executed action.

The authors evaluate LOTUS on Libero-Pro, Libero-Plus and VLABench, and on five real-world tasks with a Dobot Nova2 arm. LOTUS improves the base policy across all benchmarks, for example from 23.69% to 40.81% on Libero-Pro, and outperforms VLA-Pilot and VLS on VLABench. On the real robot, it improves the overall success rate from 23% to 59%. The paper also includes component ablations, hyperparameter sensitivity, and an efficiency comparison with VLS.

Critical strengths
  1. The method is training-free and works with frozen policies, which is practical for large VLAs where fine-tuning is expensive. It consistently improves the base policy across three simulated benchmarks. The real-world experiments are also a useful addition: on five tasks with a Dobot Nova2 arm, LOTUS improves the overall success rate from 23% to 59%.

  2. The paper includes useful ablations. In particular, it ablates the key components of the method (Figure 5) and provides an efficiency analysis against VLS (Table 4). The visualization in Figure 6 is also informative, as it clearly shows that the actions refined by LOTUS stay closer to the original policy's actions than those of VLS.

Critical weaknesses
  1. Viewing the straight-line trajectories as "samples from a reward-induced action distribution" is not supported with mathematical rigor. In particular, no target distribution is defined. Moreover, the straight-line construction is a heuristic that is unlikely to help in tasks that are contact-rich or rotation-dominated, such as turning a knob, or where the target is ambiguous, such as tying shoelaces. The failure on the Stack Bowl task (Table 3) is consistent with this limitation.

  2. From my understanding, the algorithm ultimately reduces to a weighted average of noised policy and virtual particles, with weights determined by reward and proximity, followed by a short denoising pass. The optimal transport formulation adds considerable complexity, which makes the paper harder to read, and it is unclear whether it is necessary. A comparison against a simpler alternative, such as averaging the same particles with weights proportional to exp(βR) and then denoising, would clarify this.

  3. The method introduces many new hyperparameters (β, λ, ε, τ_s, k, b, N, M). Figure 5 shows that performance is sensitive to β and λ, ranging from about 22% to 30% on VLABench, and Table 5 shows that λ, τ_s and N are set differently across benchmarks. The sensitivity to the remaining parameters is not reported, and there is no principled way to select them. Moreover, most results compare only against the base policy, and Figure 4 includes only two additional steering baselines.

Questions for authors
  1. Can the authors include additional steering baselines in the main results? In particular, methods that do not fine-tune the policy, such as V-GPS [1], and SDEdit [2] applied to the best virtual chunk (noising it and denoising it with the policy).

  2. Can the authors report the sensitivity of LOTUS to the remaining hyperparameters (ε, τ_s, k, b)? In addition, can the authors compare against a simple baseline that averages the same particles with weights proportional to exp(βR), without optimal transport?

References

[1] Nakamoto et al. Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance, '24
[2] Meng et al. SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations, '22

Reviewer_PwrSOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LOTUS, a training-free inference-time steering method that uses optimal transport (OT) to adapt frozen generative robot policies. It uses VLM-generated, stage-wise reward functions to construct virtual target trajectories, then transports policy samples toward these targets in latent denoising space, using motion discrepancy as the transport cost. The authors evaluate the method in simulation and on a real robot. The paper addresses an interesting question: how can a pretrained policy distribution be reshaped toward high-reward regions while preserving its learned action prior?

Critical strengths
  • Compared with selection-based and gradient-based steering, the paper introduces an interesting OT-based formulation. It combines geometric guidance with the pretrained policy’s behavioral prior to balance task alignment and motion preservation.

  • The framework grounds task-relevant object keypoints and uses VLMs to generate stage-aware, programmatic reward functions. This makes task-specific reward construction more accessible and explicitly incorporates the task’s stage structure.

  • The method has a clear mathematical formulation, and comparisons with selection-based and gradient-based steering help demonstrate its empirical effectiveness.

  • The evaluation covers both object-centric and observation-centric distribution shifts. These settings are practically relevant, particularly for vision-based robot policies deployed under changes in object positions, camera viewpoints, and robot initial states.

The method is interesting and offers a useful perspective on inference-time policy steering. Further analysis of target-trajectory construction and stronger controlled baselines would make its contribution more convincing.

Critical weaknesses
  • Construction of virtual target trajectories. The method constructs straight-line trajectories toward stage targets and uses them as the target support for OT. However, straight-line guidance may be unsuitable for tasks requiring obstacle avoidance, orientation changes, or complex contact interactions. Would alternative target trajectories, such as those retaining more of the original policy’s motion structure, improve performance?

  • Contribution of optimal transport. It remains unclear how much of the improvement comes from the OT formulation versus the construction of useful virtual target trajectories. A controlled baseline using the same virtual trajectories and reward functions, but replacing OT with simple interpolation or reward-weighted latent averaging, would help isolate the contribution of OT.

Questions for authors
  • Comparison with existing steering methods: It would be helpful to compare with additional baselines, such as DSRL.

  • Efficiency analysis: In Table 4, LOTUS improves the success rate over VLS by only 8–10 percentage points while introducing greater steering overhead.

  • Performance under unperturbed conditions: What happens when the base policy already succeeds under the original, unperturbed conditions? I am curious whether OT finds a more efficient way to solve the task or preserves the original trajectory.

  • Real-world evaluation: The real-world images are not sufficiently clear. It would be helpful to include clearer manipulation sequences and images from another camera angle, such as the front view.

  • Stack Bowl failure: Both the base policy and LOTUS fail on Stack Bowl under the OOD setting. Could the authors provide further analysis of the failure causes? What is the base policy’s success rate under the in-domain setting?

  • Latent-space versus action-space OT: The paper states that transport is performed in latent denoising space rather than action space to preserve the original action manifold. Could the authors include a baseline that applies OT directly in action space?

Reviewer_kaR1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors have discussed the problem of distribution shifts when deploying generative robot policies. Although such pretrained policies ideally have seen a lot of robotic data that should help in deployment, distribution shifts and OOD scenarios cause failure for these policies. A promising direction is steering the behavior of the model at test time using task information. In the literature, a VLM can be used to express the task as a reward function, which can be used to evaluate action candidates. The authors have categorized the methods in the literature as selection-based and gradient-based methods, mentioning their corresponding advantages and disadvantages and hence demonstrating the need for a motion-constrained steering paradigm that adaptively reconciles the original (prior) and task-aligned (induced by reward functions) action distributions, enabling more stable and safe steering under practical deployment variations. Empirical results show the potential of the proposed method, LOTUS, in simulated environments as well as on a real setup and in comparison to related steering methods.

Critical strengths
  • The proposed method, LOTUS, sounds promising. More importantly, I really like how the overall method tries to prevent the actions from deviating from the prior distribution of the base policy. Ideally, the performance of LOTUS is aimed to be lower-bounded by the performance of the base policy. "Ideally" means when the configurations (hyperparameters) of the algorithm are tuned correctly.
  • The empirical results are very strong. Especially, the comparison to selection-based and gradient-based methods shows the potential of LOTUS over such methods. I also appreciate the ablations and the sensitivity analysis provided, but more importantly, the insightful analysis in Figure 6, which shows how LOTUS prefers to stay near the prior distribution.
Critical weaknesses
  • Although the results are strong, especially in comparison to methods from both categories (selection-based and gradient-based), the authors have demonstrated the limitations of each category well, in addition to motivating the proposed method, without supporting their argument with a clear empirical analysis of the behavior of VLS and Pilot, an analysis more in line with Figure 6. The analysis should support that under OOD conditions, selection-based methods may struggle to obtain feasible actions by only relying on a bigger action search space or that injecting reward gradients into the denoising dynamics may disrupt the policy’s learned action manifold.
  • Another crucial point is that the method is complicated, which is fine in case the introduced hyperparameters are easy to tune. Yet, the method introduces four new parameters; two of them seem hard to tune, as shown in Figure 5 (left). Moreover, the method's performance, I believe, relies heavily on the VLM as well as the Visual Foundation Model used. I understand that this is an issue with other methods too, yet this could be critical for the effectiveness of these steering methods for robotics. A study on the performance relative to the effect of the quality of the VLM or the visual foundation model used on the performance could show us how the method would behave.
  • Although the authors have discussed related work on steering and categorized them into two method types, I believe there is a body of work that is related to steering and guiding the behavior of pretrained policies at test time, which is either cited without discussion or never mentioned. For instance, steering policies with RL, as in DSRL [1], and language steering [2]. I would have appreciated it if the authors had discussed more deeply the difference between the proposed method and these methods, either by showing empirical comparisons or at least an insightful discussion.

[1] Wagenmaker, Andrew, et al. "Steering your diffusion policy with latent space reinforcement learning." The 9th Conference on Robot Learning, 2025.

[2] Chen, William, et al. "Steerable vision-language-action policies for embodied reasoning and hierarchical control." arXiv preprint arXiv:2602.13193 (2026).

Questions for authors
  1. For the frozen policy used in the evaluation, especially in the simulated results, why is there an inconsistency between Libero-Pro and Libero-Plus? Why no evaluation of pi0.5 and the proposed method on top of it in Libero-Plus? Any technical reason for doing that? I can understand skipping pi0 for some experiments, as pi0.5 is expected to be a stronger base model. This would also be a nice addition to study the effect of the prior (base model) on LOTUS.
  2. I have mentioned in the "Critical Strengths" section that ideally, LOTUS aims for a lower-bounded performance of the base policy. I would like the authors to provide their insights on this and what are the factors that would cause LOTUS to be outperformed by the base policy? Is it the VLM model used, or the visual foundation model for keypoint generation, or what?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.