acceptodds
Under review as a conference paper at ICLR 2027

Phaseformer: Multimodal Policies without Iterative Compute or Stochastic Inference

Abstract

Deterministic behavior cloning policies struggle to capture multimodal robotic action distributions, suffering from mode averaging. Accepted solutions transform stochastic samples into robot actions, often via iterative computation (e.g. diffusion and flow matching). Flow matching backbones themselves are deterministic however, leading us to ask: why do flow matching models succeed at capturing flows between endpoint distributions while deterministic policies fail in similar situations? In this work, we prove that flow matching is a special case of behavior cloning, show that conditioning a deterministic policy on task progress lets it reproduce multimodal demonstrations whose start states vary, and characterize a score-based regularizer that pulls the policy toward the locally dominant demonstrated velocity where modes overlap. We demonstrate this empirically on multimodal robot navigation and in-plane manipulation tasks. Our findings indicate that lightweight, single-step deterministic policies can solve multimodal tasks while providing a Pareto-optimal trade-off for ultra-low latency, CPU based control.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Behavior Cloning, Flow Matching, Mode Averaging, Optimal Transport, Generative Control Policies, Robot Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7JkSOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors propose adding a "phase" parameter to denote elapsed time or task progress and build a theoretical framework relating behavior cloning policies trained with phase conditioning to generative flow matching/diffusion policies which are conditioned on the flow matching process' "time." The authors show on synthetic left/right style obstacle navigation environments that phaseformer empirically improves performance on a set of baseline environments.

Critical strengths

A unified model for behavior cloning and diffusion/flow matching models would be highly beneficial. In particular the ability to design learned policies that traverse the demonstration time marginals in a manner that is more efficient that the original demonstration dataset could be of significant benefit to the field. The authors do not quite investigate that direction fully, but this work is one step in that direction.

Critical weaknesses

The central premise of the paper is essentially the introduction of time or progress-conditioning to a behavior cloned policy. In my view this is not a particularly novel insight. Time-conditioned policies have existed for a long time and it is known that using time-conditioned imitator policy allows for modeling of a non-markovian expert using a markovian state-feedback policy. See e.g. https://arxiv.org/pdf/2307.14619 and specifically remark 2.2.

The introduction and setup are incredibly difficult to read. The notation used and the problem setting are never defined, making it difficult to understand what the setting or dynamics are and what exact claims are being made. (e.g. are we dealing with a continuous time system for eq (1)?, where is the data coming from etc). I think the paper could be majorly improved by a ground-up redesign of the presentation by introducing a general framework and showing how both behavior cloning and conditional flow matching can be recovered for particular choices of systems/parameters, rather than the current ad-hoc approach.

For a highly empirical intervention and to substantiate claims of "pareto-optimality," the experiments are far too toy and do not even cover standard manipulation benchmarks (at a minimum the authors should include robomimic-style tabletop manipulation tasks, or newer environments as those are outdated).

Questions for authors

Consider a one-dimensional state space P_S = P_G = Unif {-1, 1} but where the demonstrations are coupled such that x_0, x_1 are of the form (-1, 1) and (1, -1) (i.e. always oppositely paired) and x_rho = rho x_1 + (1 - rho) x_0. A simple behavior-cloning policy will not move at all while a time-conditioned policy will move to the midpoint and then back to the original start point. Doesn't this suggest that by making the policy time-varying sometimes we can inject extraneous information that yields worse policies?

For other tasks where there is a natural notion of "progress" (as in your toy examples) the phase is already implicitly available to the policy via the state/observation. Do you think that with more careful architecture design (e.g. with proper positional encoding or input feature representation) or with more data the differences between phase-conditioned BC and regular BC can disappear?

1 comment

Thank you for the expert review and the pointer to Block et al.

Novelty. We agree that time conditioning is not new, and the paper now says so and cites Block et al. (2023, Remark 2.2) and the DMP literature (Sec. 6). Our contribution is the account of what phase buys and what it cannot. A phase-free policy regresses onto the target averaged over its posterior belief about task progress, and the loss it pays, the temporal aliasing A, is exactly the variance of the target under that belief (Sec. 3, Eqs. 9–11). Phase removes A. It cannot remove the label conflict C or choose a branch at a shared input; that needs inputs that differ before the split (Lemma 2, Fig. 6).

Presentation and setting. We rebuilt the presentation along the lines you suggest. Sec. 3 opens with one setting: demonstrations as random paths of robot states over a continuous phase, actions as state velocities, and rollouts solving dx/dρ = π(x, ρ). Flow matching is recovered as the special case of straight-line demonstrations (Lemma 1), and three properties carry over to general demonstrations, each with its assumptions in Table 2. Full definitions are in App. A.1.

Benchmarks. The paper is no longer framed as an empirical method, and Pareto optimality is now a short remark on practical consequences. The experiments test the theory's predictions, which requires controlling the demonstration distribution: we vary start spread and path noise one factor at a time (Fig. 6). Fixed human-demonstration benchmarks do not allow this, and they exercise mode selection little: Mazza et al. (2026) find Push-T close to unimodal given the observation (App. H). SE(3) manipulation is listed as a limitation.

Q1. Can phase make the policy worse? We added your example after Prop. 1 (App. A.2). The phase-conditioned rollout goes to the midpoint and returns; the phase-free optimum is zero, so its rollouts never move. Both end at their starts and neither reproduces the pairing, but with phase the rollouts also match every intermediate marginal. Phase never raises the minimum loss. Reproducing the pairing needs an input that still differs at the crossing, which is Lemma 2.

Q2. Is phase already implicit in the state? Eqs. 9–11 answer this exactly. When phase can be recovered from the input, A = 0 and the two policies coincide, as you suggest. When states are revisited at different phases, A is a floor on the loss of every phase-free function of that input, so no architecture or amount of data removes it. Whether other policies recover progress implicitly, through observation histories or learned representations, is a hypothesis we state in Sec. 6 and do not test.

Reviewer_KRU9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors prove that flow matching is a special case of behaviour cloning on straight-line demonstrations with task progress rho as an input. Then they argue that injecting rho into a deterministic policy removes temporal aliasing, and adding start-state variation in the demonstrations can resolve mode averaging without stochastic sampling at inference. Their policy conditions a deterministic network on rho and adds a score-based regulariser during training, which moves predictions toward the median demonstrated velocity along the score where demonstrations overlap. On synthetic SE(2) navigation tasks, the resulting policies reach collision rates close to Diffusion Policy at far lower latency; On Push-T, they reach similar stall rates but much lower success rates than DP and ACT.

Critical strengths

1 The theory part is laid out clearly, with detailed steps and preliminaries. 2 The discussion is informative about how flow matching and behavior cloning coincide on straight-line demonstrations, and about the different ways mode averaging arises. 3 The experiments are thorough and carefully reported, and the figures present the results clearly.

Critical weaknesses

W1. The link between the theory and the Push-T experiments is weak. Proposition 2 covers the nonholonomic navigation task but not objects moved through contact, such as the Push-T block (Line 218). The improvement is the stall rate, and it appears only for the TF backbone, not the MLP (Line 346). This is not explained by the paper’s own theory that stalls come from zero-action averaging, which phase should remove regardless of architecture. Meanwhile, Phaseformer’s success rate is well below DP and ACT (Fig. 6). The only remaining advantage is latency in Fig. 7, but both tasks run at 10 Hz or below, so the experiments never reach the low-latency regime the paper motivates (1 kHz tactile control).

W2. The baselines look suspicious. ACT collides in 72% and 86% of rollouts in scenarios A and C but only 3% in B (Fig. 3). MIP reaches a mean coverage of 0.17 on Push-T (Fig. 5), far below what its original authors report [1].

Questions for authors

1 The latency motivation cites 1 kHz tactile control (Fig. 7), but tactile control involves contact, which the theory explicitly excludes (Line 218). Neither evaluated task matches this claim: both run at within 10 Hz, and Push-T falls outside the theory. What is the method's intended application? 2 Why does ACT collide in 72% and 86% of rollouts in scenarios A and C but only 3% in B (Fig. 3)? Also, why does MIP reach a mean coverage of 0.17 on Push-T (Fig. 5), far below what its original authors report [1]? [1] Pan C, Anantharaman G, Huang N C, et al. Much ado about noising: Dispelling the myths of generative robotic control[C]//International Conference on Learning Representations. 2026, 2026: 90575-90614.

2 comments

Thank you for the detailed review. Your two concerns, the link to Push-T and the baselines, both led to changes.

W1. Theory and Push-T. We agree, and now state the scope. Table 2 lists each result's assumptions and which task meets them: the loss results (Eqs. 9–11, Lemma 2(a),(c), Theorem 5) cover Push-T; the rollout results (Props. 1 and 3) do not, because the pusher moves the block the policy observes. Push-T is also close to unimodal given the observation (App. H), so it tests what phase adds about when. Our evidence on branch selection comes from navigation.

W1. Stalls. We removed the stall claim. Stalls arise in closed-loop contact, which our rollout results do not cover, so the theory does not predict them. Push-T now reports score, position error, orientation error and success (Fig. 4).

W1. Success below DP and ACT. Correct, and the Limitations now say so: the gap comes mainly from final position error. Beating DP or ACT is not the paper's claim. The phase variants also stop when their progress estimate reaches 1 (Fig. 5), and since the score is the maximum coverage over an episode, stopping early can only lower the score we report.

W1 / Q1. Latency and application. The 1 kHz and 60 Hz lines in Fig. 7 are reference budgets; the caption now says neither task runs at those rates. The intended setting is closed-loop control on CPU-only hardware such as embedded robot computers. The Limitations state that we do not evaluate a high-rate task and that tactile control involves contact, which our rollout results do not cover.

W2 / Q2. ACT in navigation. ACT's collisions are not mode averaging. Scenario A has a single path, and in B, ACT splits 60/36 with 3% collisions. In A and C it cuts corners and clips the obstacle on the inside of the turn. Two parts of our setup contribute (App. G): ACT executes 8 actions of each chunk without feedback, and we train every navigation model for a fixed budget and evaluate its final weights, when ACT's loss was still decreasing (Fig. 11). We therefore present ACT's collision rate as a reference under our settings, not as a limit of the method.

W2 / Q2. MIP on Push-T. Thank you; this led us to audit our baselines. Our MIP matched the authors' network, objective and sampler, but it was trained with Diffusion Policy's optimizer recipe, not MIP's own, and Fig. 5 reported final coverage where [1] reports the episode maximum. A baseline we could not train as its authors intended is not evidence, so we removed MIP from all comparisons and discuss it only as related work (Table 1, Sec. 6). App. G now states the settings used for every policy. No claim in the paper rests on MIP's numbers.

Thanks to the authors for the detailed response, and the addition of Table 2 and the honest acknowledgment of the MIP implementation error are genuinely helpful. However, four concerns from my side remain unresolved: 1) removing MIP (Table 1; rebuttal W2/Q2) leaves the paper without its most directly comparable deterministic baseline designed for multimodal BC — ACT (z=0) and SFP (σ₀=0) are deterministic at test time but were not designed with this goal; 2) the architecture-dependent stall-rate pattern originally cited at Line 346 was removed rather than theoretically explained, despite the theory applying to any L(ρ)-Lipschitz policy regardless of architecture (Secction 3); 3) the Push-T success gap versus DP and ACT is now acknowledged in the Limitations (Page 9, lines 456–458) but not resolved, leaving the practical empirical contribution narrower than the paper’s framing suggests; and 4) the latency motivation is undermined by the revised caption itself (Figure 7, Lines 416–419: “neither task here runs at those rates”), and the stated intended setting of CPU-only embedded control (Lines 421–424) is not demonstrated by any experiment in the paper. I maintain my rating.

Reviewer_aFhmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The time step normalized between 0 and 1 called phase is used as input to learn deterministic multimodal policies. The time step and the random initial start state allow multimodal behaviour even with deterministic BC policy that is trained on MSE loss.

Critical strengths
  • Deterministic BC policy while also reliably getting multi-modal behaviour that is not only multi-modal according to demonstrations, but also gives good performance in each mode.
  • Good results on two representative environments.
  • Entire codebase available online - so very little concerns about reproducibility.
Critical weaknesses
  • Motivation:
    • Why do we need deterministic Behaviour Cloning? (Is it perhaps because in robotics settings, we want very fast decisions and stochastic policies are slow? This, or whatever the motivation is, needs to be strengthened.)
    • Line 89-90 seem to imply that only MSE can be used for continuous control. This is not true. You can have negative log likelihood: optimize to raise negative log likelihood of seen actions from a parametrized policy. Then if you want determinism, take the mode.
    • In the above case you will not have mode-averaging. You will still have unimodal behaviour / mode collapse.
  • Writing and Presentation
    • Found the paper hard to follow and required multiple readings. Some places still unclear even after multiple passes. e.g.: figure 3 referenced first, we find out much later what TF means.
    • I have not seen in my experience the term "Pareto optimality" to be used for time taken in addition to performance. Although technically, sure, time could be considered a goal, but I find this usage slightly strange. Maybe it's my unfamiliarity with papers in robotics? I am not sure.
  • Scope of Contribution
    • I have some concerns about the scope of the contribution. It is known that deterministic policies can create multimodal behaviour if the start states are not fixed and have some randomness in them (or one could just add random noise). There are also works that use time as an additional input, but I am not sure if there's any whose main goal is multimodality and uses time for that purpose. But both time + random starts for multimodality, I am not aware of being systematically studied. But still, I am concerned about limited scope of this contribution.

Minor Formatting

  • not sure if beginning with a figure instead of abstract is allowed according to format. Leaving this comment here for other reviewers, authors and chairs.

Justification for Recommendation and Confidence

  • Imitation / Reinforcement is my area of research focus so I am familiar with RL, but I don't study robotics in particular so I have chosen confidence level 2.
  • My main concern with this paper is writing and presentation. The scope even if limited, is still an important research to share with the community, and the motivation perhaps can be clarified with minor edits.
Questions for authors
  • How is different from conditioning on the mode, except that now different modes map to different start states? So, how would it compare to a goal conditioned deterministic BC policy?
  • What is meant by "iterative compute"? It is said that other methods use 'iterative compute' but I could not figure what this means - even after briefly looking at [Chi 2024] and [Zhao 2023]. It it the diffusion process that's being called iterative compute, or something else?
  • Theorem 1, iii that says 'demonstrations do not cross in state-phase space'. Does this mean: (a) a single demonstration will not have the same state again (which is intuitively pretty obvious, given there is a phase alongside the state), or (b) different demonstrations. no two demonstration trajectories will have the same (state+phase)?]
  • It seems from the results that the only advantage over DP (DP has fewer stalls and higher score) is that this method takes less time? This is still an important contribution, but I would hope that there is some comment in the paper that discusses this.
1 comment

Thank you for the careful review. Presentation seemed to be your main concern, and the paper has been rewritten around a single question; brief answers follow.

Presentation. Sec. 3 now opens with a Setting and notation paragraph. The model table is in the main text (Table 3) and defines TF, Phaseformer (TF+phase) and the clock before any figure uses them. The abstract now comes before the first figure.

Why deterministic BC? Most accepted remedies for mode averaging already run deterministically at test time: ACT decodes its prior mean, MIP's two passes are deterministic, and SFP sets its noise width to zero (Table 1). We ask what such a policy minimally needs. Speed is a practical consequence, not the motivation.

MSE is not the only loss. Agreed, and we now say so after Eq. 1. A Gaussian likelihood has its mode at the same conditional mean. A mixture likelihood avoids the average, but decoding its mode gives a deterministic map of the input, so Lemma 2(c) applies: its rollouts reach different branches only if their inputs differ before the split.

Pareto optimality. We use the standard definition: a policy is Pareto-optimal if no other policy has both a higher score and a lower latency. Accuracy and latency frontiers are common in efficient deep learning (e.g., MnasNet, Tan et al., CVPR 2019). It is now a short paragraph on practical consequences (Sec. 5), not a headline claim.

Scope. As you note, phase and start variation had not been studied together for multimodality. We now do so one factor at a time (Fig. 6): with one shared start, no variant reaches the goal in any seed, with or without phase; with varied starts, success returns to 60–97%. Lemma 2 explains why, and Eqs. 9–11 give the exact loss a policy pays without phase.

Q1. Conditioning on the mode or goal. The start does not encode the mode. Each demonstration's branch is drawn independently of its start, and at phase 0 the branch cannot be predicted from the pose better than chance (App. H, Fig. 13). Varied starts only give the policy inputs that differ before the split. Goal conditioning adds nothing in scenarios A and C or in Push-T, which have one goal; in B it would reveal the mode.

Q2. Iterative compute. Evaluating the network several times in sequence per action or chunk, as in DP's denoising passes. The introduction now defines it.

Q3. "Do not cross." Reading (b): no two demonstrations pass through the same state at the same phase with different velocities. This is now Theorem 4(iii), stated as zero label conflict.

Q4. Advantage over DP. Outperforming DP is not our aim. DP scores higher on Push-T, and the Limitations say so. Our contribution is the account of what a deterministic policy needs; in navigation, a single-pass policy that meets those requirements comes close to DP (3% vs. 1% collisions in scenario C).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.