acceptodds
Under review as a conference paper at ICLR 2027

Training-Free Adaptation of Generalist Robot Policies via Distributional Policy Steering

Abstract

Pretrained vision-language-action (VLA) policies possess broad motor skills but can struggle to apply them under semantic and spatial distribution shifts. Vision-language models (VLMs) offer complementary reasoning capabilities for out-of-distribution (OOD) inference, given its strong capability on semantic understanding and commonsense knowledge. However, how to formalize and integrate VLM reasoning into visuomotor action sampling while leveraging pretrained motor skills remains an open challenge. We introduce Distributional Policy Steering (DPoS), a training-free, inference-time steering method that provides distributional alignment guidance from VLM-derived spatial information to a frozen VLA policy. DPoS constructs a shell-supported target distribution around a reasoned spatial estimate and jointly guides sampled action chunks toward it. An energy function is designed based on the squared maximum mean discrepancy measures between the sampled action candidates cloud and the target distribution. DPoS enables the repulsion among sampled candidates, aiming for a better coverage among the action space. The attraction to the target distribution aims for a reasoned spatial target alignment for the task success. Evaluated on LIBERO-PRO simulator and 9 real-world OOD challenging tasks, DPoS achieves the highest success rate among all leading baselines. It improves over the base model by 44.4 and 19.9 percentage points on real-world and LIBERO-PRO tasks respectively, demonstrating distributional guidance as an effective way to integrate reasoning prior into pretrained robot control without additional model training.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotics Manipulation, Vision-Language-ActionModels, Inference-Time Steering

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_2ivLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Distributional Policy Steering (DPoS), a training-free method for improving the OOD generalization of pretrained VLA policies. It constructs a shell-supported target distribution from VLM-estimated spatial targets and uses MMD-based guidance to steer action generation through attraction and particle repulsion. Experiments are conducted on both LIBERO-PRO and real-world robotic tasks.

Critical strengths
  1. The training-free design enables adaptation of pretrained VLA policies without additional fine-tuning, offering practical value for robotic deployment.

  2. The paper is well-organized, with clear and intuitive illustrations that make the overall methodology easy to follow.

Critical weaknesses
  1. DPoS relies on one-step estimates of the final action chunks to compute Shell-MMD gradients at every flow-matching step. However, these estimates may deviate substantially from the final generated actions during early, high-noise stages, and may even average over multiple action modes. Consequently, the resulting attraction and repulsion gradients may be unreliable or even misleading, particularly when early action estimates collapse toward similar predictions, making the repulsion term ineffective. This weakens the methodological justification for the proposed Shell-MMD guidance.

  2. OmniGuide (Song et al., CoRL 2026) has already explored inference-time guidance of frozen VLA policies using one-step action estimates and 3D target-based spatial energies, while VLS (Liu et al., CoRL 2026) further incorporates particle repulsion into gradient-based action guidance. Moreover, MMD Guidance (Sani et al., ICML 2026) has already employed MMD-based distribution matching during diffusion sampling, with gradients naturally decomposing into repulsion among generated samples and attraction toward a reference distribution. Therefore, the methodological novelty of DPoS appears somewhat limited, primarily lying in the shell-supported target distribution over end-effector displacements. However, Proposition 1 only establishes a near-optimal neighborhood under certain assumptions, without guaranteeing the feasibility or near-optimality of the constructed shell. Furthermore, while the comparison with Point Attraction (R=0) demonstrates benefits over a single-point target, it does not establish the unique advantage of the shell over other spatial target distributions, such as Gaussian or uniform-ball distributions.

Questions for authors

For Weakness 1:

  1. Could the authors visualize the one-step action estimates from early denoising steps (e.g., Steps 1–2) and compare them with the fully denoised unguided actions from the same initial noise? Additionally, how consistent are the early Shell-MMD gradients with those at later stages across different sampling seeds?

  2. Could the authors provide an ablation comparing Shell-MMD guidance applied only at early steps, only at later steps, and throughout the entire denoising process, to verify whether early-stage guidance is actually beneficial?

For Weakness 2:

  1. Given the similarities to related works, could the authors reconsider how the paper's core methodological contribution is positioned relative to existing methods?

  2. How does the shell-supported target distribution compare with alternative distributions, such as Gaussian or uniform-ball distributions, under the same experimental settings?

  3. How does Proposition 1 justify the feasibility and choice of radius of the target shell used in practice?

5 comments

R4Q1. Early-step estimates, and consistency of early and late gradients

Visualization and comparison of early-step action estimates. Figure 10,11 (L979) visualizes the step-1 and step-2 estimates alongside the fully denoised unguided action from the same initial noise. We agree that the one-step estimate is coarse at the noisiest steps. In flow matching, x^0=xt−tvθ(xt,t)\hat x_0 = x_t - t v_\theta(x_t, t) approximates the posterior mean E[x0∣xt,o]\mathbb{E}[x_0 \mid x_t, o] [1,2]. At t=1t = 1, xtx_t carries no information about the final sample, so the estimate reduces to the policy's mean action and averages over modes, as the reviewer notes.

An imprecise estimate, however, does not by itself make the guidance misleading. What matters is whether the gradient computed from it moves the final action in the intended direction, and whether early and late steps agree on that direction. To isolate this effect, we replay deployment steps in which every run shares the same configuration. We then apply Shell-MMD guidance at a single denoising step at a time:

guidance at step12345678910
Δ\Delta angle to target vs. unguided (°, median)−0.9-0.9−1.0-1.0−1.3-1.3−1.4-1.4−1.6-1.6−1.9-1.9−2.5-2.5−3.2-3.2−4.7-4.7−4.9-4.9
cases turned toward target71.0 %72.2 %73.8 %73.7 %73.7 %73.3 %72.1 %70.2 %69.3 %62.8 %

Guidance applied only at the noisiest step, where the estimate is least accurate, still turns the executed action toward the target in 71 % of cases. The median effect has the correct sign at every step, so early and late gradients are consistent in direction. The magnitude of the correction increases toward the final steps, where the estimate is closest to the final action. The high-noise steps therefore contribute the least, but they do not oppose the later ones. This is consistent with the deterministic ODE transport of flow matching: conditioned on xTx_T, no stochasticity is injected during sampling, so a shift introduced early is propagated to the final sample rather than washed out [1,2].

The reviewer also raises the concern that near-identical early estimates could render the repulsion term ineffective and collapse the candidates. If this occurred, guidance would reduce the spread of the KK final actions. We observe no such reduction: the mean pairwise distance is 12.5 cm without guidance and 12.4 cm with it. Meanwhile, the median distance to the shell decreases from 14.0 cm to 7.4 cm. The guidance thus moves the candidates toward the shell while preserving their diversity.

[1] Lipman et al., Flow Matching for Generative Modeling, ICLR 2023.

[2] Liu et al., Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow, ICLR 2023.

R4Q2. Is early-stage guidance beneficial?

The single-step analysis indicates that each step helps in isolation. To test whether early-stage guidance remains beneficial within a full schedule, we restrict guidance to the first half (steps 1–5), the second half (steps 6–10), or all steps, under the same matched-noise protocol:

guidanceangle to target (°)distance to shelldistance to target
none24.414.0 cm17.6 cm
early only (1–5)20.012.4 cm16.0 cm
late only (6–10)17.78.0 cm11.6 cm
full (1–10)16.57.4 cm11.0 cm

Early-only guidance improves all three measures over the unguided baseline, confirming that early-stage guidance is beneficial on its own. Late-only guidance yields a larger improvement, consistent with the single-step results, and the full schedule performs best on every measure. Early and late guidance are therefore complementary rather than redundant (Fig. 12; full results in Appendix I, L921).

R4Q3. Positioning of the core contribution relative to existing methods

The key difference from prior methods is our assumption about the VLM target. Methods such as OmniGuide and VLS rely on a strong reasoning agent and treat the VLM’s output as the target, so their performance depends on the quality of that output. DPoS instead assumes that a VLM-pointed target is only an estimate and may be imperfect, so it steers toward a distribution of valid targets rather than a single point. This matters most with less capable models such as Qwen, whose targets may be slightly off, and the distributional target can recover from these less optimal estimates. Because the target is a distribution, steering becomes the problem of matching the action samples to it, and both the attraction and repulsion terms follow directly from this one objective (Eqs. 11 and 12). Prior methods instead add separate, hand-designed diversity terms. For the same reason, the formulation is not tied to reaching: any task-space quantity that admits a set of valid targets is handled in the same way.

We support this positioning in both theory and experiments. Proposition 1 shows that a whole neighborhood of the VLM estimate is near-optimal. Compared with a single-point target, the distributional target improves success from 45.1% to 51.1% in simulation and from 26 to 64 successes out of 90 real-robot trials. With an open-weight Qwen3-VL-32B, DPoS reaches 51.1% on LIBERO-PRO, compared with 33.2% for OmniGuide and 42.3% for the published VLS result. On the real robot it achieves 64 versus 31 successes out of 90 trials against OmniGuide. Replacing the VLM target with the ground-truth target adds only 3.5 points (54.6%), so the method is not limited by VLM quality. We will revise the introduction and related work to state this distinction explicitly.

R4Q4 Shell vs. Gaussian and uniform-ball target distributions The design of the shell geometry in our proposed method is mainly inspired by two motivations:

  • Theoretically, shells provide a natural decomposition of the target space: any candidate q≠gq \ne g can be expressed as q=g+rdq = g + rd, with r=∥q−g∥2r = \lVert q - g \rVert_2 and ∥d∥2=1\lVert d \rVert_2 = 1. Consequently, a general target distribution can be viewed as a mixture of distributions supported on concentric shells. Our fixed-radius construction isolates directional variability at a controlled deviation from the estimate. Under the assumptions of Proposition 1, a shell with R≤(ε−δest)/LR \le (\varepsilon - \delta_{\mathrm{est}})/L lies within the near-optimal region, provided its support is feasible. This connects shell-supported guidance directly to our theoretical analysis, and Appendix C further identifies spherical manifolds of equivalent optima under rotational symmetry.
  • Practically, the shell maintains (∥qm−g∥2=R\lVert q_m - g \rVert_2 = R) while varying the approach direction. Compared with ball or Gaussian distributions, which also allocate probability mass to locations closer to the center, the shell uses the full prescribed radius to explore alternative approaches. Together with Shell-MMD's particle repulsion, this design encourages broader candidate coverage while avoiding excessive concentration around a single estimated target.

R4Q5 How Proposition 1 relates to the feasibility and radius of the shell

Proposition 1 does not guarantee that the shell is feasible, and it does not prescribe a radius. It only shows that the VLM estimate is not a unique target: under the Lipschitz and δest\delta_{\mathrm{est}} assumptions, every feasible point within (ε−δest)/L(\varepsilon-\delta_{\mathrm{est}})/L of the estimate is ε\varepsilon-optimal. Feasibility is an assumption of the result (the ∩,X\cap,\mathcal X in Eq. 6), and ε\varepsilon, LL and δest\delta_{\mathrm{est}} cannot be measured for real tasks. What the proposition supports is steering toward a set of targets around the estimate rather than toward the estimate itself. Within that set, we choose a shell for two reasons:

Theoretically, any candidate q≠gq\neq g can be written as q=g+rdq=g+rd with r=∥q−g∥2r=\lVert q-g\rVert_2 and ∥d∥2=1\lVert d\rVert_2=1, so a general target distribution is a mixture of distributions on concentric shells. A fixed-radius shell isolates the approach direction at a controlled distance from the estimate. A shell with R≤(ε−δest)/LR\le(\varepsilon-\delta_{\mathrm{est}})/L lies inside the near-optimal region of Proposition 1, provided its points are feasible. Appendix C further shows that under rotational symmetry the optima themselves form a sphere. Practically, the shell keeps every target at distance RR while varying the approach direction. Ball or Gaussian targets also put probability mass closer to the center, which pulls candidates back toward the single estimated point. Together with the repulsion term, the shell spreads candidates over approach directions. Shell points are only targets for the candidate distribution and are never executed, so an infeasible shell point cannot produce an infeasible command.

Choice of the radius. RR is an empirical design choice. Proposition 1 only says the shell should stay inside the near-optimal region. As R→0R\to0 the shell reduces to point attraction, which does worse (45.1% vs. 51.1% in simulation, 26 vs. 64 of 90 on the real robot). In Fig. 4b, success stays within 1.4 points of the best for RR from 5 to 15 cm, so the exact value matters little (Table 1 uses R=5R=5 cm). The method also tolerates VLM localization error: shifting the VLM point by 5 px leaves real-robot success almost unchanged (63 vs. 64 of 90).

Reviewer_aRAmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Distributional Policy Steering (DPoS) to adapt a frozen VLA policy to semantic and spatial distribution shifts at inference time. A VLM identifies a spatial target, and DPoS constructs a shell-supported distribution around it. An MMD-based guidance rule attracts action candidates to the shell and repels them from one another. The method selects an action chunk by its alignment with the target. Experiments cover six LIBERO-PRO suites and nine real-world tasks, with ablations on guidance strength, shell radius, and inference-time test.

Critical strengths
  1. Improving target selection without retraining the manipulation policy is a useful goal, and the reported gains are substantial. With VLM targets, DPoS achieves 51% success on LIBERO-PRO. The real-world results reach 71.1%, compared with 26.7% for the frozen policy.

  2. Comparing oracle and VLM-predicted targets helps assess the effect of grounding errors.

Critical weaknesses
  1. It is unclear how much of the improvement comes from shell attraction. Best-of-N selects the endpoint closest to the target, whereas DPoS selects by cosine alignment, so the comparison changes both guidance and selection. I would like to see these methods compared with the same selector to assess the additional benefit of the shell. The point-target control provides some evidence, but near-optimal proposition does not explain why a spherical shell, or the chosen radius, should suit these tasks.

  2. My main concern is the significance of the contribution. DPoS guides predicted end-effector translation toward a spatial target, and all nine real-world tasks involve pick-and-place, with difficulty arising mainly from object identification. The results show that this guidance helps the policy approach the intended object while preserving its existing grasp behavior, but provide limited evidence of benefits when the difficulty lies in orientation constraints, contact interactions, or coordinated manipulation. The authors explicitly acknowledge this scope. Even within that scope, however, I am not yet convinced that the shell-based formulation offers a sufficiently substantial advance over existing spatial guidance and particle-diversity methods.

Questions for authors

See weakness.

4 comments

R3W1. Benefit of the shell under a matched selector, and why a shell and this radius

Same selector. We agree that the original comparison changed both guidance and selection, so we reran it with the selector fixed (π0.5, six LIBERO-PRO suites, paired episodes with identical seeds and targets):

ArmObj swapObj taskSpa swapSpa taskGoal swapGoal taskMean
Paper: DPoS (cosine)58.447.252.476.241.031.651.1
DPoS + nearest endpoint54.241.850.080.437.029.548.8
DPoS + random selection on shell56.444.051.072.034.027.047.4
Best-of-N + nearest endpoint10.07.651.054.634.622.230.0
Best-of-N + cosine27.011.852.456.036.022.034.2

Under either rule, DPoS beats Best-of-N by about 19 points, while swapping the rule within DPoS changes success by around 2.3 points. The reason is that a selector can only choose among the candidates it is given. Best-of-N's candidates come from the unguided policy. When the policy's prior points at the wrong object, all NN candidates tend to miss, and no selection rule can recover. DPoS changes the candidates themselves: guidance moves them toward the target region during denoising, so most of them are already good. Even a random pick among the four particles closest to the target keeps most of the gain. Guidance and selection work together, but the gain comes from guidance, not from the choice of selector.

Shell vs. point. To isolate the shell itself, we keep the same guided sampler and selector and only shrink the shell to a single point (R=0R = 0):

TargetLIBERO-PRO successReal robot (successes / 90)
Single point (R=0R = 0)45.1%26
Shell (DPoS)51.1%64

The rollouts show why. Point attraction pulls every candidate toward one location, and that location is not where the gripper should end up. This produces overshoot, poor grasp initialization, and abrupt corrections during the transition from approach to grasp. On the real robot these corrections lead to tabletop contact and failed grasps, which is why the gap is larger there. The shell only asks the end-effector to come within distance RR of the estimate, from any direction, and leaves the final approach and grasp to the policy's learned behavior.

Why a shell. The VLM point tells us roughly where the object is, but not from which direction to approach it, and an object can usually be grasped from many directions. The shell encodes exactly this information: be at distance RR from the estimate, in any direction. This is also general: any target distribution around the estimate can be written as a mixture of concentric shells (q=g+rdq = g + rd), and a fixed-radius shell isolates the direction at a controlled distance. A ball or a Gaussian puts most of its mass near the center, so matching it pulls candidates back toward the single point and reintroduces the failure modes of point attraction. With the shell, attraction brings candidates to the right distance while repulsion spreads them over directions. This diversity survives selection: the executed candidate sits a median of 45° away from the straight line to the target, rather than collapsing onto it.

Radius. Proposition 1 does not pick a specific radius, and we do not claim it does. What it says is that the near-optimal targets form a neighborhood around the estimate, and that a shell stays inside that neighborhood as long as R≤(ε−δest)/LR \le (\varepsilon - \delta_{\mathrm{est}})/L. This predicts a trade-off. A radius that is too small collapses to point attraction. A radius that is too large places targets outside the near-optimal region. The ablation (Fig. 4b) shows exactly this shape:

Shell radius RR0.1 cm10 cm30 cm
Success53.5%58.4%42.5%

The constants ε\varepsilon and LL depend on the task and cannot be measured directly, so we choose RR empirically, as stated in the paper. The optimum is broad, so performance is not sensitive to the exact value. Consistent with this, shifting the VLM point by 5 px leaves real-robot success unchanged.

R3W2. Significance of the shell-based formulation over existing guidance methods

Our key idea is simple: a VLM-pointed target is only an estimate, so it should be a distribution of valid targets and not a single point. This idea really benefits if we have less frontier models, such as Qwen, which may have a slightly not that correct target estimation. However, our distribution shell can resolve from these poor or less optimal estimates. Additionally, based on this design, both the attraction and the repulsion term follow directly from matching the action samples to the target distribution (Eqs. 11 and 12). Neither is designed by hand, unlike the separate diversity terms in prior methods. The formulation is therefore not tied to reaching: any task-space quantity that admits a set of valid targets is handled in the same way. We support the distribution target both in theory and in experiments. Proposition 1 shows that a whole neighborhood of the VLM estimate is near-optimal. Against the single-point target, DPoS reaches 51.1% versus 45.1% in simulation, and 64 versus 26 successes out of 90 trials on the real robot.

Unlike OmniGuide and VLS, DPoS does not rely on a strong reasoning agent to solve the task. Those methods depend on the quality of the VLM's output, whereas DPoS is built on the assumption that the VLM target is imperfect. This is why we steer toward a distribution of valid targets and not a single point, and why attraction and repulsion both follow from one objective without hand-designed terms. With an open-weight Qwen3-VL-32B, DPoS reaches 51.1% on LIBERO-PRO, against 33.2% for OmniGuide and 42.3% for the published VLS result, and 64 versus 31 successes out of 90 real-robot trials against OmniGuide. Replacing the VLM target with the ground-truth target adds only 3.5 points (54.6%), so the method is not limited by VLM quality. The design is supported in theory by Proposition 1, which shows that a whole neighborhood of the VLM estimate is near-optimal, and in experiments by the comparisons above.

Reviewer_aRAmOpenReview ↗

Thank you for the response and additional experiments.

Thank you for the response and additional experiments. However, my main concern remains the significance of the contribution. The results demonstrate useful target-directed guidance, but I am still not convinced that the shell formulation provides a sufficiently substantial advance over existing spatial guidance and particle-diversity methods. Its potential applicability to other task-space quantities does not establish benefits beyond the evaluated pick-and-place setting. Thus, I keep my rating in this situation.

Thank you for getting back to us! We would like to clarify two points.

First, our goal and main claim is to make the best use of the pre-trained motor skills. For example, we use π0.5 in our experiments, and we inherit and preserve most of its visuomotor skills. Our claim is that, with the proposed guidance, the same frozen policy can complete more tasks that are OOD for it. Adding other fine-grained motor skills is not the target of our paper, and we think this is the first misunderstanding. The paper already states this scope (Appendix K): “DPoS steers a frozen policy but does not explicitly learn new motor skills.”

Second, we have already discussed how our method differs from other guidance work in robotics, such as VLS, OmniGuide and Flow Reversal Steering, in both the submitted draft and the rebuttal. Our experiments also show higher success than both VLS and OmniGuide: 51.1% vs 42.3% (VLS) and 33.2% (OmniGuide) on LIBERO-PRO, and 71.1% vs 34.4% (OmniGuide) on the real robot. These findings are consistent with what we proposed and support our hypothesis. We believe we follow the scientific process exactly: we observe what current methods lack, propose our method, and use experiments to demonstrate its effectiveness. If this chain of logic is consistent, and you note that the results “demonstrate useful target-directed guidance”, in what respect is our work not significant? It would be great if you could be more specific about the lack of significance. For example, what does it mean here, and which component of our work is not consistent? This would help us improve the draft in the future. Thanks!

Reviewer_z5qtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

A pretrained robot policy may already have the required manipulation skills but still approach the wrong target when object semantics or positions change. This paper proposes DPoS, which uses a target located by a vision-language model to guide a frozen flow-based VLA without retraining. The method places a shell of targets around the goal and derives attraction and inter-particle repulsion terms from an MMD objective. During sampling, it normalizes these terms separately and uses them to adjust several action candidates. A directional score then selects the action chunk to execute. Experiments on LIBERO-PRO and nine real-robot tasks show improvements over the original policy, point-target guidance, and other guidance methods. Presentation: 2: Generally understandable, but some parts are difficult to follow. Informativeness: The experiments show the value of guiding policies with a spatial target distribution. However, the connection between the proposed objective and the actual sampling procedure needs to be clearer. Soundness: 2: I have some concerns that could be addressed with clearer explanations or additional experiments.

Critical strengths

The method addresses a practical problem without updating the pretrained policy. The gains are substantial. On LIBERO-PRO, DPoS achieves 51.1% success, compared with 31.2% for π0.5 and 45.1% for point attraction. On the real-robot tasks, DPoS achieves 71.1%, compared with 34.4% for OmniGuide. Each method is evaluated on 3,000 simulation episodes and 90 real-robot trials. The tests with ground-truth target locations, different shell radii, and separate attraction and repulsion terms help explain the method. The paper also acknowledges that spatial guidance cannot provide complex contact skills that the policy does not already have.

Critical weaknesses
  1. Equations 10–12 derive the MMD objective and its attraction and repulsion terms. Equation 13 then normalizes the two terms separately, gives them independent weights, and normalizes their sum again. This changes their relative magnitudes and may also change the final update direction. Deriving the original MMD gradient therefore does not show that the implemented sampler follows that gradient or decreases the objective. The empirical gains may still hold, but the paper should clarify whether the method directly follows the MMD gradient or uses an update inspired by it. A comparison between the raw gradient and the normalized update is also needed.
  2. Proposition 1 gives a near-optimal ball under Lipschitz continuity and a sufficiently accurate target estimate. The result itself is reasonable. However, the paper does not estimate the Lipschitz constant or how far the estimated target’s cost is from the optimum. The result therefore does not explain the preferred 10 cm shell. The symmetry result in Appendix C also assumes that feasibility and task cost are unchanged by rotation, which may not hold in scenes with obstacles. The theory explains why it may help to keep several options near the target, but it does not show that a shell is better than a ball, a Gaussian distribution, or a target region that accounts for feasibility.
  3. The paper includes useful controls: point-target guidance with the same multi-particle sampler, repulsion alone, and attraction alone. However, Figure 4 reports 58.6% for the full method and 55.4% for repulsion alone without uncertainty estimates for this difference. Best-of-N selects actions by endpoint distance, whereas DPoS uses the cosine similarity between the action direction and the target direction. That comparison changes both the guidance and the selection rule. The VLS results also come from a published paper rather than a new evaluation under the same settings. The authors should use the same candidate-selection rule, report uncertainty, and compare other target distributions.
  4. Equation 8 treats the first three action components as Cartesian translations, but Appendix E says the real arm uses joint-velocity control. Kinematics or an intermediate control interface could connect the two, but the paper does not describe the differentiable mapping. It should also explain when the arm enters or leaves the reaching stage and when guidance is enabled. In addition, the VLM is called once per reaching stage, but this time is excluded from the latency measurements. The separately provided code may contain these details, but the paper should state the actual action interface and the total execution time for each stage.
Questions for authors

Q1. What objective does the separately normalized update in Equation 13 optimize? How do the raw MMD gradient and the normalized update compare in task success and MMD values during sampling? Q2. Why use a shell rather than a ball, a Gaussian target distribution, or a reachable target region? Can the theoretical radius be related to measured localization error or task sensitivity to position error? Q3. Does DPoS still perform better when it uses the same candidate-selection rule as Best-of-N? Please also report the gain over repulsion alone and its uncertainty. Q4. How are joint-velocity action chunks mapped differentiably to displacements in world coordinates? Please explain stage transitions, when guidance is enabled, and the total latency including VLM calls.

3 comments

R2 W1 and Q1. What Eq. 13 optimizes, and raw vs. normalized gradient

In Eq. 12, KK is the number of particles, MM the number of shell points, and hh the kernel bandwidth. The prefactors 4/(Mh2)4/(Mh^2) and 4/(Kh2)4/(Kh^2) are constants and do not change either term's direction.

What does Eq. 13 optimize? It's the same objective as derived. It is the Φshell\Phi_{\mathrm{shell}} gradient with the two terms of Eq. 12 rebalanced per particle: each term keeps its exact direction, and only their relative weight changes.

Does it decrease the objective or follow the gradient? Yes. With watt=wrepw_{\mathrm{att}} = w_{\mathrm{rep}} it is a descent direction of Φshell\Phi_{\mathrm{shell}} for every particle, since

⟨uiatt+uirep,ϕi⟩=(∥ϕiatt∥+∥ϕirep∥)(1+cos⁡θi)≥0,\langle u_i^{\mathrm{att}} + u_i^{\mathrm{rep}}, \phi_i \rangle = (\lVert \phi_i^{\mathrm{att}} \rVert + \lVert \phi_i^{\mathrm{rep}} \rVert)(1 + \cos\theta_i) \ge 0,

where θi\theta_i is the angle between the two terms. It is not the steepest descent direction; normalizing the full gradient, ϕi/∥ϕi∥\phi_i / \lVert \phi_i \rVert, is.

Why normalize. The raw scales differ because of the kernel values, not the prefactors. Unguided particles lie close to each other but far from the shell (mean κh\kappa_h of 0.93 vs 0.03), so ∥ϕirep∥/∥ϕiatt∥\lVert \phi_i^{\mathrm{rep}} \rVert / \lVert \phi_i^{\mathrm{att}} \rVert has a median of 76, and 16 after guidance. Without normalization, ρ\rho and the term weights must be retuned whenever KK, MM, hh or RR changes.

Raw gradient vs normalized update. We evaluate π0.5 on the six LIBERO-PRO suites under the same testing standard as Table 1 in main. We show that normalization vs raw is very similar in success rate. Normalization is just a design choice and we choose it since it's convenient for us to see the scale of each term and act more scientifically.

All three updates decrease the MMD objective and are statistically similar in success, each 17.5 to 19 points above unguided sampling. Table 1 uses row 1 in main paper.

UpdateObj swapObj taskSpa swapSpa taskGoal swapGoal taskMean
Joint norm (Table 1)58.447.252.476.241.031.651.1
Raw gradient57.446.644.078.640.535.050.4
Eq. 13 separate norm54.044.646.881.038.531.049.3

R2Q2. Shell vs. ball or Gaussian targets, and choosing the radius

The design of the shell geometry in our proposed method is mainly inspired by two motivations:

  • Theoretically, shells provide a natural decomposition of the target space: any candidate q≠gq \ne g can be expressed as q=g+rdq = g + rd, with r=∥q−g∥2r = \lVert q - g \rVert_2 and ∥d∥2=1\lVert d \rVert_2 = 1. Consequently, a general target distribution can be viewed as a mixture of distributions supported on concentric shells. Our fixed-radius construction isolates directional variability at a controlled deviation from the estimate. Under the assumptions of Proposition 1, a shell with R≤(ε−δest)/LR \le (\varepsilon - \delta_{\mathrm{est}})/L lies within the near-optimal region, provided its support is feasible. This connects shell-supported guidance directly to our theoretical analysis, and Appendix C further identifies spherical manifolds of equivalent optima under rotational symmetry.
  • Practically, the shell maintains (∥qm−g∥2=R\lVert q_m - g \rVert_2 = R) while varying the approach direction. Compared with ball or Gaussian distributions, which also allocate probability mass to locations closer to the center, the shell uses the full prescribed radius to explore alternative approaches. Together with Shell-MMD's particle repulsion, this design encourages broader candidate coverage while avoiding excessive concentration around a single estimated target.

R2Q3. Matched-selector comparison and uncertainty of the repulsion-only gap

Yes. The selection rule is an empirical design choice and not the source of the gain. With the rule matched, DPoS outperforms Best-of-N by 18.8 points under Best-of-N's endpoint rule (48.8 vs 30.0) and by 16.9 points under our cosine rule (51.1 vs 34.2), while swapping the rule within DPoS changes success by only 2.3 points. Even a random pick among the four of eight particles nearest the target keeps a gain of 17.4 points over Best-of-N (47.4 vs 30.0).

ArmObj swapObj taskSpa swapSpa taskGoal swapGoal taskMean
Paper table: DPoS (cosine)58.447.252.476.241.031.651.1
DPoS + nearest endpoint54.241.850.080.437.029.548.8
DPoS + random selection on shell56.444.051.072.034.027.047.4
Best-of-N + nearest endpoint10.07.651.054.634.622.230.0
Best-of-N + cosine27.011.852.456.036.022.034.2

R2Q4. Joint-velocity mapping, stage transitions, and total latency

Joint velocities →\to world displacement. We add up the first HgH_g joint velocities to get qH=q0+c∑τaτq_H = q_0 + c \sum_{\tau} a_{\tau}, and compute y=FK(qH)−FK(q0)y = \mathrm{FK}(q_H) - \mathrm{FK}(q_0) in the world frame. FK is closed-form and differentiable, so ∇aR=cJp(qH)⊤∇yR\nabla_a R = c J_p(q_H)^\top \nabla_y R. No IK or retraining is needed. In simulation this reduces to Eq. 8 (Jp=IJ_p = I). (To save space, see similar answer for R1Q2 (erPD).)

Stages and guidance. A rule-based parser splits the instruction into ordered stages. Before the robot moves, the VLM is queried once per stage on the first frame, and each point is lifted to 3D. At every re-plan, the current stage's target is re-expressed relative to the end-effector. The robot moves to the next stage on gripper events: closing the gripper ends a reach-and-grasp stage, and opening it ends a place stage. Both are read from the gripper state with a short debounce. Guidance is on while the end-effector is at least 3R3R (≈\approx 10 cm) from the target. Closer in, the policy's own grasp and place behavior takes over.

Latency. (See R1Q5 (erPD).) One VLM query takes 1.16 ±\pm 0.43 s, with one query per stage: 1.1 s against 18.5 s of motion for single-stage tasks, and 4.7 s against 25.1 s for four-stage tasks. No query happens inside the control loop. Guided sampling adds 18.1 ms per decision (72.4 ms vs. 54.3 ms), or 79.1 ms for the whole client decision.

R2W3. Uncertainty, matched selector, VLS under the same settings, and other target distributions

Uncertainty for full method vs. repulsion alone (Fig. 4). We re-evaluated both settings with five independent policy seeds, keeping all other settings fixed.

settingsuccess rate
full method (attraction + repulsion)58.2% ±\pm 0.9%
repulsion only54.8% ±\pm 1.2%

The full method outperforms repulsion alone by 3.4 pp, a gap larger than the seed-to-seed variation of either setting.

VLS under the same settings. VLS needs frontier VLMs. The VLM writes differentiable reward code for each sub-task stage, grounds the keypoints that code refers to, and is queried up to 50 times per episode to recognize the current stage. The released code uses GPT-5.1 and Gemini-3-Flash for these steps. Our method asks the VLM only for the target location, so it runs with an open-weight model (Qwen3-VL-32B). Before submission, we evaluated VLS with Qwen3-VL-32B in place of its frontier models, keeping VLS's default hyperparameters and using the same 800 episodes as our ablations. It reached 31.9%, versus 58.3% for ours (table). We left this out of the paper because it tests VLS outside the frontier-model setting it was designed for, which we judged unfair to VLS.

methodVLMObj-SwapObj-TaskSpa-SwapSpa-TaskOverall
π0.5, no guidance–17.510.549.554.032.9
VLSQwen3-VL-32B40.527.533.526.031.9
OursQwen3-VL-32B55.048.051.079.058.3

VLS runs on its released LeRobot π0.5 checkpoint, and ours runs on openpi π0.5. Each episode's task, initial state and seed are the same for every method.

Thank you again for your detailed reviews! As we approach the end of the interactive rebuttal phase, we would like to follow up and ask whether our responses have fully addressed your concerns. If any concerns remain, we would greatly appreciate it if you could point out the specific issues that have not been fully addressed so we can provide further clarification, even the response window is short now. Thank you for your time and consideration!

Reviewer_erPDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Distributional Policy Steering (DPoS), a way to adapt a pretrained VLA policy at inference time without updating its weights. A VLM first identifies a target in the image, which is projected into 3D. Instead of steering actions toward that single point, DPoS samples points on a shell around it. During flow-matching inference, an MMD-based gradient pulls candidate end-effector displacements toward the shell while pushing candidates apart. Once sampling finishes, the method executes the action chunk whose predicted displacement has the highest cosine similarity with the target direction.

Critical strengths
  1. Good motivation. Adapting an existing VLA to distribution shifts without collecting new demonstrations or fine-tuning the policy is a useful problem to study.

  2. Relevant ablations. The paper tests several choices that matter to the proposed method: a point target versus a shell target, the attraction–repulsion weights, the shell radius, and the guidance strength. These comparisons help show what changes when the proposed guidance is used.

Critical weaknesses
  1. Bias in the final action selection. The method spreads candidates over a target shell, but Eq. 15 chooses the action chunk by cosine similarity to the direction of the shell center. Geometrically, this favors candidates near the two points where the end-effector–target line intersects the shell (when the end effector is outside it). The selected actions may therefore come from a narrow part of the shell even though the sampler deliberately generates different approaches. The paper does not show how the actual selected candidates are distributed.

  2. Dependence on Cartesian action representations. Eq. 8 takes the first three dimensions of an action chunk as end-effector translation. That is not directly applicable to policies producing joint positions or joint velocities. Such policies would need a task-space mapping and a way to propagate its gradients back to the generated actions. The paper does not address this extension, so the method's applicability across generalist VLA policies remains unclear.

  3. Missing VLM cost in the latency results. Fig. 8 reports the overhead of guided action sampling but leaves out the external VLM query. Querying the VLM only once per reach stage reduces its frequency, but does not remove its cost, especially when a task has several stages. Without query timings and end-to-end execution times, the reported latency does not describe the complete system.

Questions for authors

Q1. Which shell candidates are actually selected? Could the authors report the spatial distribution of the K candidate end-effector displacements and the distribution of the candidates ultimately chosen by Eq. 15? In particular, how often do selected candidates lie near the two intersections of the target shell and the end-effector–target line?

Q2. How would the method handle joint-space actions? Eq. 8 assumes that the first three action dimensions describe end-effector translation. For a VLA outputting joint positions or velocities, how would the task-space displacement and corresponding action-space gradient be computed?

Q3. What does the VLM's target point mean? Is the returned pixel intended to mark the object center, a visible surface point, or a grasp-relevant point? How much does the choice of point on the same object affect the resulting guidance?

Q4. What is sent to and returned by the VLM? Please provide an actual prompt and an example of its raw output. Is the output just one pixel coordinate, or does it contain other reasoning or target candidates? How are multi-stage instructions handled?

Q5. What is the total inference and execution cost? Please report VLM latency, the number of VLM calls per task, and end-to-end execution time, including for multi-stage tasks.

4 comments

R1Q1: Distribution of candidate and selected end-effector displacements

We conduct a candidate-geometry analysis across our entire real-robot corpus. All statistics below are from the main configuration. We use one quantity throughout: θ\theta, the angle between a candidate's endpoint displacement and the straight line from the end-effector to the target. The two points where that line crosses the shell lie at θ=0\theta = 0, so "how close is a candidate to those points" is simply "how small is its θ\theta".

Where the KK candidates point. They point away from the straight line, by a median of 84°. The new figure (Figure.9, L882) in the appendix shows this decision by decision for three tasks: every candidate path drawn on the point cloud of that frame, with the executed one and the line to the target marked.

Where the chosen candidate points. The cosine score does pull the choice towards the line, but not onto it. The executed candidate sits at a median of 45°, and 6.0 % of executed candidates are within 10° of the line. Broken down by how far the robot still has to go:

remaining distancemedian θ\theta, all candidatesmedian θ\theta, executedexecuted with θ<10∘\theta \lt 10^\circ
d>20d \gt 20 cm60°23°15.7 %
10–20 cm89°46°4.0 %
d<10d \lt 10 cm93°59°1.4 %
all84°45°6.0 %

The choice is most aligned with the line early in the reach, when the robot is still far away and heading straight at the object is also the useful thing to do, and it turns away from the line as the robot closes in and has to commit to a side to grasp from.

The choice does not collapse onto the two intersections: the executed candidate sits a median of 45° away from them, and only 6.0 % come within 10°. The candidates the sampler offers are themselves a median of 84° off that line, so the spread of approach directions survives the selection step rather than being discarded by it. The Shell-MMD term pulls the policy towards the target mode while keeping the diversity of the selected candidates.

The figure, the table, and the full description are in the revised appendix (L851).

R1Q2. Extending the action-to-displacement map to joint-space actions

With a VLA outputting joint positions or velocities, only the map from an action chunk to its task-space displacement changes. We use forward kinematics: y=FK(qH)−FK(q0)y = \mathrm{FK}(q_H) - \mathrm{FK}(q_0), where q0q_0 is the current joint configuration and qHq_H the configuration at the end of the chunk, given by the last waypoint for joint positions, the summed deltas for joint deltas, or the integrated velocities for joint velocities. The action-space gradient follows from the chain rule, ∇aR=c⊙Jp(qH)⊤∇yR\nabla_a R = c \odot J_p(q_H)^\top \nabla_y R, with JpJ_p the positional Jacobian and cc the action scale. Eq. 8 is the special case Jp=IJ_p = I. This needs no inverse kinematics and no retraining: FK is closed-form and differentiable, and the Jacobian transpose stays bounded near singularities.

R1Q3. Meaning of the VLM point, and robustness to its exact location

The VLM point is only a coarse geometric cue. It is not intended to mark the object center, a particular surface point, or a grasp point, and it carries no specific meaning. DPoS guides samples toward a shell distribution around this point rather than toward the point itself, so the exact pixel chosen on the object has little effect on the resulting guidance.

To verify this, we shifted the VLM-predicted pixel by 5 px in each of four directions (up, down, left, right). This gives 5 target points, and we ran 2 real-robot trials for each (the same trails as our original setup).

Object / EasyObject / MediumObject / HardSpatial / EasySpatial / MediumSpatial / HardCompositional / EasyCompositional / MediumCompositional / Hard
Original1008070808070705040
Point-shifted1008070807070804040

Performance remains consistent across all perturbed points, so DPoS is robust to the choice of the VLM point.

R1Q4. The VLM prompt, an example output, and stage handling

Each query sends Qwen3-VL-32B a labeled object reference image, the current camera frame, and this prompt:

The first image is a labeled reference showing what each object in this environment looks like, with its name written next to it. [reference image] The second image is the current camera view of a robot workspace containing some of those objects. Using the reference to identify objects, locate the {target} in the SECOND image and point to it. Answer with exactly one JSON object like {"point_2d": [x, y], "label": "name"} where x, y are the point coordinates in the second image. A robot arm may be visible in the camera view; never point at the robot arm, and never point at a different, similar-looking object. If the {target} is not visible in the second image, answer {"point_2d": null}. [current frame]

Example raw output: {"point_2d": [580, 431], "label": "alphabet soup"}. The output is a single point with a label; it contains no reasoning and no alternative candidates. The point is lifted to 3D with depth and used as the shell center.

A rule-based parser splits a multi-stage instruction into ordered stages (e.g., [reach+grasp X, transport+place Y]), and the VLM is queried once per stage.

R1Q5. VLM latency, number of calls, and end-to-end execution time

We added a second panel to Fig. 8 (L447) that reports the per-episode cost. We find that one-time VLM calling cost is quitet low compared to the execution time of the whole episode. For a four-stage Compositional task, VLM queries take 4.7 s against 25.1 s of robot motion (about 16% of the combined time); for a single-stage task, 1.1 s against 18.5 s (about 6%).

One VLM pointing query takes 1160 ±\pm 430 ms. The VLM is queried once per guided stage (once for Object and Spatial tasks, four times for Compositional tasks), and all queries are issued from the first frame before the first action is sent.

per episodeObject / SpatialCompositional
VLM queries14
VLM time1.1 s4.7 s
robot motion18.5 s25.1 s

During execution, guided sampling adds 18.1 ms per decision (72.4 ms vs. 54.3 ms for vanilla inference; 79.1 ms for the complete client decision), and the VLM adds no latency inside the control loop. The number of VLM queries depends on the number of stages, not on episode length.

The new candidate-selection analysis and VLM latency measurements address my original concerns. The explanation of joint-space extensions is also reasonable, although it remains unverified experimentally.

However, I also share some of the concerns raised by other reviewers regarding the methodological novelty and broader significance of this work. While the empirical improvements are encouraging, the main contribution seems to be the shell-supported target distribution, building on existing inference-time guidance methods. The real-world experiments mainly focus on target grounding and pick-and-place tasks, so it remains unclear how broadly the method would benefit more complex manipulation settings.

I therefore maintain my original rating

We thanks the reviewer for the valuable feedback.

Regarding the experimental evidence of joint-space guidance, the policy in our real-world experiment operates in joint-space. As stated in L795, Pi05 Droid directly outputs joint-space control signal, so our guidance is propagated into the joint-space as described in our previous response.

As for the novelty question, our contribution and the novelty is threefold, and all three points are already stated in the paper.

  1. Many works nowdays use a VLM to orchestrate a base VLA, but much of this guidance is unnecessary and wastes tokens and compute. How a VLM can directly guide the base VLA on OOD tasks while leveraging its pretrained motor skills remains underexplored. In particular, what shared representation connects a VLM’s spatial reasoning to the flow denoising steps remains unsolved, and our paper answers this question. Evidence: DPoS queries the VLM once per stage (1.1 s to 4.7 s per episode) and adds 18.1 ms per decision, while VLS queries the VLM up to 50 times per episode. With our proposed MMD based energy function guide the flow denosing, success on LIBERO-PRO rises from 31.2% to 51.1% with no training.

  2. Many works, such as VLS and OmniGuide, assume a frontier VLM and rely on its agentic performance. For example, VLS uses a frontier VLM to divide the task into stages and to write a reward function for each stage, and OmniGuide relies on a frontier VLM to point to a very precise location as the guidance direction. They all fail when the VLM’s pointed estimate is suboptimal, and they fail often with an open-source VLM. In contrast, our method assumes an imperfect VLM point estimate and tolerates it because of distribution matching. Evidence: VLS reaches 31.9% when the open-weight Qwen3-VL-32B replaces its frontier models, versus 58.3% for ours on the same episodes in LIBERO-PRO; OmniGuide reaches 33.2% on LIBERO-PRO and 34.4% on the real robot, versus 51.1% and 71.1% for ours. Our method with Qwen points is within 3.5 points of ground-truth targets (51.1% vs 54.6%), and shifting the VLM point by 5 px changes real-robot success only from 71.1% to 70.0%. These are the strong evidance that our experiments meet our hypothesis that our distribtional-matching target steering the flow steps are effective with an imperfect VLM signal.

  3. Our method is general: it has no hard-coded or hand-designed repulsion or attraction function in the flow denoising steps. Many other text-to-image and robot steering works design these terms by hand and lack this generality. Evidence: Both terms follow from one MMD based energy objective (Eqs. 11 and 12). The same formulation is used in simulation and on the real robot (51.1% vs 31.2% and 71.1% vs 26.7% for the frozen policy).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response including the novelty and the contribution

General response to all reviewers

We thank the reviewers for their valuable feedback. All four reviewers find the work well motivated: adapting a frozen VLA without new demonstrations or fine-tuning is "a useful problem to study" (erPD), "a practical problem" (z5qt) and "a useful goal" (aRAm), "offering practical value for robotic deployment" (2ivL). They also value the experiments: the "gains are substantial" (z5qt, aRAm), the experiments "show the value of guiding policies with a spatial target distribution" (z5qt), the ablations "help explain the method" (z5qt) and "help show what changes when the proposed guidance is used" (erPD), and the oracle-target comparison "helps assess the effect of grounding errors" (aRAm). The paper is also "well-organized, with clear and intuitive illustrations that make the overall methodology easy to follow" (2ivL).

Contribution, novelty and significance. DPoS is built on the assumption that the VLM target is imperfect. A VLM-pointed target is only an estimate, so we steer toward a distribution of valid targets around it and not toward the point itself. Prior guidance methods such as OmniGuide and VLS depend on the quality of the VLM's output; DPoS is designed to tolerate its error. It therefore does not need a highly precise VLM: with an open-weight Qwen3-VL-32B it reaches 51.1% on LIBERO-PRO, within 3.5 points of ground-truth targets (54.6%). The design is also general. Attraction and repulsion both follow from one distribution-matching objective, with no hand-crafted terms, and the formulation is not tied to reaching: any task-space quantity with a set of valid targets is handled the same way. We use Maximum Mean Discrepancy (MMD) as this objective. MMD also appears in prior work in other settings, but it is a standard mathematical tool shared across fields, like an energy or potential function in probabilistic modelling, and we do not claim it as the contribution. What is new is what we match: a target distribution built around an imperfect VLM estimate, used to steer a frozen VLA at inference time. Proposition 1 supports the distribution target in theory, and the shell outperforms the single-point target in experiments (51.1% vs 45.1% on LIBERO-PRO; 64 vs 26 successes out of 90 real-robot trials). See A2 (aRAm).

Key questions answered.

  • Is the gain from guidance or from the selector? From guidance. With the selector matched, DPoS outperforms Best-of-N by more than 16 points under either rule, and swapping the rule within DPoS changes success by under 2 points. See R2Q3 (z5qt), R3W1 (aRAm).
  • Why a shell, and how is R chosen? A shell fixes the distance to the estimate and leaves the approach direction free, while a ball or a Gaussian pulls candidates back toward the point. R is chosen empirically and the optimum is broad. See R2Q2 (z5qt), R4Q4 (aRAm).
  • What does Eq. 13 optimize? The same shell objective. The raw gradient, joint normalization and Eq. 13 are statistically similar in success (50.4%, 50.7%, 49.3%). See R2Q1 (z5qt).
  • Which candidates are selected, and does the exact VLM point matter? The executed candidate sits a median of 45° from the line to the target, so selection does not collapse onto it. Shifting the VLM point by 5 px gives 70.0% vs 71.1% on the real robot. See R1Q1 (erPD), R1Q3 (erPD).
  • Is guidance at early denoising steps reliable? Guidance at the noisiest step alone still turns the action toward the target in 71% of cases. Early and late guidance are complementary, and candidate diversity is preserved. See R4Q1 (2ivL), R4Q2 (2ivL).
  • How does VLS compare under the same VLM? With the same Qwen model, VLS reaches 31.9% vs 58.3% for ours. See R2W3 (z5qt).
  • Joint-space actions and cost. Forward kinematics maps joint actions to displacement with no retraining. The VLM is queried once per stage before motion (1.1 s to 4.7 s per episode), and guidance adds 18.1 ms per decision. See R1Q2 (erPD), R1Q5 (erPD), R2Q4 (z5qt).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.