- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Distributional Policy Steering (DPoS), a training-free method for improving the OOD generalization of pretrained VLA policies. It constructs a shell-supported target distribution from VLM-estimated spatial targets and uses MMD-based guidance to steer action generation through attraction and particle repulsion. Experiments are conducted on both LIBERO-PRO and real-world robotic tasks.
Critical strengths
-
The training-free design enables adaptation of pretrained VLA policies without additional fine-tuning, offering practical value for robotic deployment.
-
The paper is well-organized, with clear and intuitive illustrations that make the overall methodology easy to follow.
Critical weaknesses
-
DPoS relies on one-step estimates of the final action chunks to compute Shell-MMD gradients at every flow-matching step. However, these estimates may deviate substantially from the final generated actions during early, high-noise stages, and may even average over multiple action modes. Consequently, the resulting attraction and repulsion gradients may be unreliable or even misleading, particularly when early action estimates collapse toward similar predictions, making the repulsion term ineffective. This weakens the methodological justification for the proposed Shell-MMD guidance.
-
OmniGuide (Song et al., CoRL 2026) has already explored inference-time guidance of frozen VLA policies using one-step action estimates and 3D target-based spatial energies, while VLS (Liu et al., CoRL 2026) further incorporates particle repulsion into gradient-based action guidance. Moreover, MMD Guidance (Sani et al., ICML 2026) has already employed MMD-based distribution matching during diffusion sampling, with gradients naturally decomposing into repulsion among generated samples and attraction toward a reference distribution. Therefore, the methodological novelty of DPoS appears somewhat limited, primarily lying in the shell-supported target distribution over end-effector displacements. However, Proposition 1 only establishes a near-optimal neighborhood under certain assumptions, without guaranteeing the feasibility or near-optimality of the constructed shell. Furthermore, while the comparison with Point Attraction (R=0) demonstrates benefits over a single-point target, it does not establish the unique advantage of the shell over other spatial target distributions, such as Gaussian or uniform-ball distributions.
Questions for authors
For Weakness 1:
-
Could the authors visualize the one-step action estimates from early denoising steps (e.g., Steps 1–2) and compare them with the fully denoised unguided actions from the same initial noise? Additionally, how consistent are the early Shell-MMD gradients with those at later stages across different sampling seeds?
-
Could the authors provide an ablation comparing Shell-MMD guidance applied only at early steps, only at later steps, and throughout the entire denoising process, to verify whether early-stage guidance is actually beneficial?
For Weakness 2:
-
Given the similarities to related works, could the authors reconsider how the paper's core methodological contribution is positioned relative to existing methods?
-
How does the shell-supported target distribution compare with alternative distributions, such as Gaussian or uniform-ball distributions, under the same experimental settings?
-
How does Proposition 1 justify the feasibility and choice of radius of the target shell used in practice?
5 comments
R4Q3. Positioning of the core contribution relative to existing methods
The key difference from prior methods is our assumption about the VLM target. Methods such as OmniGuide and VLS rely on a strong reasoning agent and treat the VLM’s output as the target, so their performance depends on the quality of that output. DPoS instead assumes that a VLM-pointed target is only an estimate and may be imperfect, so it steers toward a distribution of valid targets rather than a single point. This matters most with less capable models such as Qwen, whose targets may be slightly off, and the distributional target can recover from these less optimal estimates. Because the target is a distribution, steering becomes the problem of matching the action samples to it, and both the attraction and repulsion terms follow directly from this one objective (Eqs. 11 and 12). Prior methods instead add separate, hand-designed diversity terms. For the same reason, the formulation is not tied to reaching: any task-space quantity that admits a set of valid targets is handled in the same way.
We support this positioning in both theory and experiments. Proposition 1 shows that a whole neighborhood of the VLM estimate is near-optimal. Compared with a single-point target, the distributional target improves success from 45.1% to 51.1% in simulation and from 26 to 64 successes out of 90 real-robot trials. With an open-weight Qwen3-VL-32B, DPoS reaches 51.1% on LIBERO-PRO, compared with 33.2% for OmniGuide and 42.3% for the published VLS result. On the real robot it achieves 64 versus 31 successes out of 90 trials against OmniGuide. Replacing the VLM target with the ground-truth target adds only 3.5 points (54.6%), so the method is not limited by VLM quality. We will revise the introduction and related work to state this distinction explicitly.
R4Q4 Shell vs. Gaussian and uniform-ball target distributions The design of the shell geometry in our proposed method is mainly inspired by two motivations:
- Theoretically, shells provide a natural decomposition of the target space: any candidate can be expressed as , with and . Consequently, a general target distribution can be viewed as a mixture of distributions supported on concentric shells. Our fixed-radius construction isolates directional variability at a controlled deviation from the estimate. Under the assumptions of Proposition 1, a shell with lies within the near-optimal region, provided its support is feasible. This connects shell-supported guidance directly to our theoretical analysis, and Appendix C further identifies spherical manifolds of equivalent optima under rotational symmetry.
- Practically, the shell maintains () while varying the approach direction. Compared with ball or Gaussian distributions, which also allocate probability mass to locations closer to the center, the shell uses the full prescribed radius to explore alternative approaches. Together with Shell-MMD's particle repulsion, this design encourages broader candidate coverage while avoiding excessive concentration around a single estimated target.
R4Q5 How Proposition 1 relates to the feasibility and radius of the shell
Proposition 1 does not guarantee that the shell is feasible, and it does not prescribe a radius. It only shows that the VLM estimate is not a unique target: under the Lipschitz and assumptions, every feasible point within of the estimate is -optimal. Feasibility is an assumption of the result (the in Eq. 6), and , and cannot be measured for real tasks. What the proposition supports is steering toward a set of targets around the estimate rather than toward the estimate itself. Within that set, we choose a shell for two reasons:
Theoretically, any candidate can be written as with and , so a general target distribution is a mixture of distributions on concentric shells. A fixed-radius shell isolates the approach direction at a controlled distance from the estimate. A shell with lies inside the near-optimal region of Proposition 1, provided its points are feasible. Appendix C further shows that under rotational symmetry the optima themselves form a sphere. Practically, the shell keeps every target at distance while varying the approach direction. Ball or Gaussian targets also put probability mass closer to the center, which pulls candidates back toward the single estimated point. Together with the repulsion term, the shell spreads candidates over approach directions. Shell points are only targets for the candidate distribution and are never executed, so an infeasible shell point cannot produce an infeasible command.
Choice of the radius. is an empirical design choice. Proposition 1 only says the shell should stay inside the near-optimal region. As the shell reduces to point attraction, which does worse (45.1% vs. 51.1% in simulation, 26 vs. 64 of 90 on the real robot). In Fig. 4b, success stays within 1.4 points of the best for from 5 to 15 cm, so the exact value matters little (Table 1 uses cm). The method also tolerates VLM localization error: shifting the VLM point by 5 px leaves real-robot success almost unchanged (63 vs. 64 of 90).
R4Q1. Early-step estimates, and consistency of early and late gradients
Visualization and comparison of early-step action estimates. Figure 10,11 (L979) visualizes the step-1 and step-2 estimates alongside the fully denoised unguided action from the same initial noise. We agree that the one-step estimate is coarse at the noisiest steps. In flow matching, x^0=xt−tvθ(xt,t) approximates the posterior mean E[x0∣xt,o] [1,2]. At t=1, xt carries no information about the final sample, so the estimate reduces to the policy's mean action and averages over modes, as the reviewer notes.
An imprecise estimate, however, does not by itself make the guidance misleading. What matters is whether the gradient computed from it moves the final action in the intended direction, and whether early and late steps agree on that direction. To isolate this effect, we replay deployment steps in which every run shares the same configuration. We then apply Shell-MMD guidance at a single denoising step at a time:
Guidance applied only at the noisiest step, where the estimate is least accurate, still turns the executed action toward the target in 71 % of cases. The median effect has the correct sign at every step, so early and late gradients are consistent in direction. The magnitude of the correction increases toward the final steps, where the estimate is closest to the final action. The high-noise steps therefore contribute the least, but they do not oppose the later ones. This is consistent with the deterministic ODE transport of flow matching: conditioned on xT, no stochasticity is injected during sampling, so a shift introduced early is propagated to the final sample rather than washed out [1,2].
The reviewer also raises the concern that near-identical early estimates could render the repulsion term ineffective and collapse the candidates. If this occurred, guidance would reduce the spread of the K final actions. We observe no such reduction: the mean pairwise distance is 12.5 cm without guidance and 12.4 cm with it. Meanwhile, the median distance to the shell decreases from 14.0 cm to 7.4 cm. The guidance thus moves the candidates toward the shell while preserving their diversity.
[1] Lipman et al., Flow Matching for Generative Modeling, ICLR 2023.
[2] Liu et al., Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow, ICLR 2023.
R4Q2. Is early-stage guidance beneficial?
The single-step analysis indicates that each step helps in isolation. To test whether early-stage guidance remains beneficial within a full schedule, we restrict guidance to the first half (steps 1–5), the second half (steps 6–10), or all steps, under the same matched-noise protocol:
Early-only guidance improves all three measures over the unguided baseline, confirming that early-stage guidance is beneficial on its own. Late-only guidance yields a larger improvement, consistent with the single-step results, and the full schedule performs best on every measure. Early and late guidance are therefore complementary rather than redundant (Fig. 12; full results in Appendix I, L921).