acceptodds
Withdrawn from ICLR 2027

ModeFlow: Breaking the Connectivity Barrier of One-Step Flow Policies

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

One-step flow policies enable efficient action generation. However, we identify a fundamental connectivity barrier. Mapping connected Gaussian noise through a continuous generator produces a proposal distribution with connected support, so when high-value behaviors occupy separated regions, the policy necessarily places probability mass on the low-value bridges between them, and suppressing that mass below forces the generator's Lipschitz constant to grow as . We formalize this barrier and derive a Gaussian-isoperimetric lower bound on the bridge mass. To overcome this challenge, we propose **ModeFlow**, a mode-conditioned one-step MeanFlow policy that discovers discrete behavior modes from offline trajectories, conditions the generator and critic on mode labels, and stratifies a fixed proposal budget across modes. This allows the policy to represent separated behaviors as distinct proposal branches while retaining efficient one-step generation. We further prove the return improvement of **ModeFlow** over one-step policies of this class, as the bridge mass induces a nonzero probability of missing the optimal behavior. Experiments demonstrate that **ModeFlow**, consistently outperforms the baselines across the Robomimic and OGBench tasks, with particularly large gains on long-horizon manipulation. On the most challenging Cube-triple-task5, \method achieves success while all baselines fail.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, one-step policy generation, connectivity barrier

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PpGUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that best-of-N one-step flow policies such as MVP face a "connectivity barrier": a continuous map of Gaussian noise has connected support, so when high-value action chunks form separated regions, some proposal mass falls between them unless the Lipschitz constant is large. ModeFlow clusters offline trajectories into K labels that condition both the generator and the critic, and splits a fixed proposal budget evenly across labels. A second theorem lower-bounds the baseline's return gap to an idealized mode-conditioned policy by the chance that all its proposals miss the optimal region. On ten sparse-reward Robomimic and OGBench tasks, ModeFlow raises average final success over MVP from 0.79 to 0.99 and solves cube-triple-task5, where MVP fails entirely.

Critical strengths
  1. A minimal change yields large gains at no extra inference cost. ModeFlow adds only a label embedding and stratified sampling to MVP at essentially MVP's latency. Because the cube-triple-task5 comparison holds every other setting fixed (App. C), the jump from complete failure to near-perfect success reflects the labels and stratification rather than tuning.

  2. The obstruction is stated precisely. Theorem 1 carries known connected-support and isoperimetry arguments (Tanielian et al.; Salmona et al.) over to best-of-N proposals, the proofs appear correct, and the lemma that stratified proposals dominate i.i.d. ones is clean and reusable. This gives the paper a sound formal core.

Critical weaknesses
  1. The causal story behind the gains is not established. The paper attributes ModeFlow's success to breaking a connectivity barrier, but its own analysis does not support this at the operating point. In Theorem 2 the bridge mass enters only through b^N, which is negligible at N = 32, so what the bound effectively captures is how proposals are allocated across regions, an effect that does not depend on connectivity. The experiments point the same way: the toy study (App. D) is equally consistent with an allocation explanation, and success keeps rising with K up to one proposal per label (Fig. 5c) instead of saturating at a plausible number of strategies, which fits added proposal diversity or exploration as well as mode discovery. The method may well work, but the paper's central explanation of why it works remains unverified.

  2. The empirical comparison is less controlled than it appears. On nine of the ten tasks, the baseline numbers coincide exactly with those reported by Zhan et al. (2026) under a smaller configuration than ModeFlow's (fewer critic heads and, on Robomimic, fewer proposals), and Table 1 does not indicate this; a matched comparison is given only for a subset of OGBench tasks (Fig. 4). In addition, ModeFlow makes almost no progress during the offline phase on two Robomimic tasks where the baselines do (Fig. 3), which the paper does not discuss, and the number of modes K is set per benchmark although performance is highly sensitive to it. Together these make it hard to judge how much of the headline improvement is a fair and general gain.

Questions for authors
  1. On cube-triple-task4/5 and one Robomimic task, does ModeFlow keep its advantage when the K-means labels are replaced by random labels with the same K, or when stratified sampling is replaced by i.i.d. label sampling? How often does some proposal succeed, and how often does the critic pick it?

  2. Could you report MVP under ModeFlow's exact configuration on all ten tasks (and with larger proposal budgets, if run), report end-of-offline success for every method, and explain why ModeFlow makes almost no offline progress on Robomimic-can and square?

  3. How was K chosen for each benchmark without using evaluation success, and what happened on cube-quadruple and antmaze-giant-navigate, whose partitions appear in Table 5, if you ran them, including negative results?

  4. LDCQ (Venkatraman et al., ICLR 2024) selects trajectory-derived latent skills with a Q-function, and DDiffPG (Li et al., NeurIPS 2024) learns mode-specific Q-functions over trajectory clusters; neither is cited. What is new beyond pairing such clustering with a one-step generator and stratified best-of-N?

Reviewer_wPUAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ModeFlow to address the possibility that one-step flow policies allocate probability mass to low-value bridges between separated behavioral modes, potentially producing incoherent actions. ModeFlow clusters offline trajectories into discrete behavioral modes and uses their labels to condition a MeanFlow generator and a value network. It stratifies a fixed proposal budget across modes to improve behavioral coverage and action selection while retaining one-step generation. Experiments show improvements over the evaluated baselines, particularly on long-horizon manipulation tasks.

Critical strengths
  1. The paper has a clear motivation and addresses a practically relevant problem. It examines the coverage of separated behavioral modes in one-step flow policies and connects proposal quality to subsequent value-based selection. This provides a useful perspective on improving multimodal policies.
  2. The method is simple and effective, with largely unchanged inference cost. Mode conditioning and stratified proposal sampling improve behavioral coverage and action selection while retaining a shared network and one-step generation.
  3. The experimental results are encouraging, particularly on challenging tasks. The method achieves strong overall performance on Robomimic and OGBench, with substantial gains on long-horizon triple-cube manipulation tasks.
Critical weaknesses
  1. A gap remains between the theoretical analysis and the practical method. The return guarantee assumes that the conditional branches cover the relevant high-value regions. However, direct evidence that trajectory clustering recovers these behavioral modes is lacking. The attribution of gains on actual tasks to reduced bridge mass relies largely on interpretation rather than direct measurements of proposal quality and mode coverage.
  2. The ablations do not sufficiently disentangle the contributions of individual design choices. The method jointly introduces mode-conditioned generation, stratified sampling, and a mode-conditioned critic. However, it does not compare independent uniform mode sampling with stratified sampling while holding the generator fixed. These omissions make it difficult to identify the sources of the performance gains.
Questions for authors
  1. The policy loss in Figure 2 appears to correspond to the objective in Equation (11), but the time arguments differ. Could the authors confirm which formulation is implemented and check the consistency of notation across the relevant equations and figures?
  2. Further discussion of the method’s applicability to more complex settings would be helpful. Would the current mode partitioning and proposal allocation mechanisms remain effective in tasks combining discrete strategy selection, complex continuous control, and multiple objectives? What additional challenges might arise?
  3. The paper provides qualitative guidance on choosing the number of modes based on strategic diversity. Could the authors clarify how to assess this diversity in a new task and translate that assessment into a specific choice of (K)?
Reviewer_V5AYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ModeFlow clusters offline trajectories, conditions a shared one-step MeanFlow actor and critic on the resulting mode labels, and stratifies the proposal budget across modes before best-QQ

Critical strengths

The empirical gains on the hard cube-manipulation tasks are large and consistent over five seeds. The method retains inference cost close to MVP, making the practical result potentially important.

Critical weaknesses

The novelty and theory are not yet convincing. The disconnected-support expansion argument appears to be a specialization of the continuous Gaussian push-forward bound of Salmona et al., rather than a property specific to one-step generation. More importantly, DDiffPG, which already combines trajectory clustering, a mode-conditioned generative policy, and mode-specific critics, is not discussed. Theorem 2 assumes that each ideal branch is already supported inside an optimal region and that critic error is uniformly bounded. Its positive-gain condition ΔρN>2ϵ\Delta\rho^N>2\epsilon becomes nearly vacuous at N=32N=32: for ρ=0.6\rho=0.6, ρ32≈8×10−8\rho^{32}\approx8\times10^{-8}. It therefore does not explain the learned system. Theorem 1(iv) also implicitly assumes that every nonempty closed target set contains a positive-radius ball, which is false for singleton or lower-dimensional sets. Experimentally, actor conditioning, stratified sampling, critic conditioning, and additional capacity are not separated. Table 1’s baseline numbers also appear to reuse the published MVP results despite different proposal and critic-ensemble settings; only the 250k-step comparison covers a subset under matched conditions. Finally, because zz is reselected at every chunk and is not an environment state variable, the semantics and calibration of Q(s,A,z)Q(s,A,z) require justification.

Questions for authors

Please clarify novelty relative to DDiffPG and the Salmona bound. Can the authors report final-time controlled reruns with identical NN, critic ensemble, parameter budget, and compute? A factorial ablation of actor conditioning, critic conditioning, stratification, random labels, and i.i.d. mode sampling is needed. Why is Q(s,A,z)Q(s,A,z) the correct value function rather than an auxiliary predictor whose calibration may differ across labels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.