acceptodds
Under review as a conference paper at ICLR 2027

Learning to Optimize Actions with Flows

Abstract

Model-based planning requires repeatedly solving complex non-convex optimization problems over sequences of actions. These problems are predominantly tackled by sampling-based optimizers, whose hand-crafted update rules cannot exploit structure patterns across planning instances. From a complementary perspective, generative modeling has enabled expressive policies that directly generate action sequences, yet without optimizing them over a horizon through a predictive model. We bridge these two paradigms by introducing **FlowOpt Policy**, which casts planning as a learned transport problem over action sequences: rather than generating actions directly, a flow iteratively transports a population of candidate action sequences toward lower cost, conditioned on their rollout evaluations. The flow is parameterized by an attention mechanism in which every candidate attends over the evaluated population, so that its update depends jointly on where the other candidates lie and how they scored. We further reveal connections to classical methods such as MPPI and CEM, whose updates emerge as particular fixed attention patterns. FlowOpt Policy is trained in a self-supervised manner via flow matching on improvement steps computed by a sampling-based optimizer on the populations the planner encounters, without requiring any demonstration data. Experiments on classical robotics and world model planning tasks show that FlowOpt Policy consistently outperforms sampling-based planners at equal rollout budget, and is more robust than state-conditioned generative policies, generalizing better to unseen objectives because it learns how to optimize rather than what to output.

Primary area:
applications to robotics, autonomy, planning
Keywords:
learning-to-optimize, trajectory optimization, model predictive control, generative models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aB5cOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FlowOpt Policy, a learned sampling-based optimizer for model predictive control. Unlike methods such as MPPI or CEM—which condense candidate action sequences into a weighted mean before resampling—this approach leverages cost-conditioned attention and flow matching to directly learn how to iteratively transform an entire population of candidate action sequences. The training objective is generated online by a more robust sampling-based "teacher," guiding the model to evolve the current population of candidates toward one with lower costs. Experimental results demonstrate that, given the same rollout budget, this method outperforms various classical optimizers across a range of physical control tasks and TD-MPC2 world-model benchmarks. Furthermore, it exhibits superior out-of-distribution adaptability compared to "Flow Policy," a model that generates actions directly.

Critical strengths
  1. Formulating online action optimization as "cost-constrained candidate-ensemble transport" is both conceptually elegant and insightful. This framing naturally preserves the structure among multi-modal candidate actions, making it a promising direction for further research.

  2. The paper compares MPPI, CEM, CMA-ES, and DIAL-MPC across tasks including CarNav, HalfCheetah, Walker, Push-T, and the Cheetah-WM and Humanoid-WM environments (based on TD-MPC2). It also includes comparisons with a "Flow Policy" architecture. Notably, in the Push-T and world-model-based tasks, the results demonstrate that leveraging rollout costs during the online optimization process significantly outperforms one-shot, direct action generation.

Critical weaknesses
  1. In the kk-th iteration of the outer loop, the paper performs only a single rollout for the initial population AkA^k, while keeping CkC^k fixed across all Nτ=5N_\tau=5 Euler steps. As x(τ)x(\tau) moves away from the original candidates, the network continues to rely on the ranks and standardized costs computed at the previous positions—information that becomes invalid once the candidates have shifted.

  2. The current ablation studies primarily focus on increasing the number of outer-loop optimization iterations in the CarNav and Push-T environments. This fails to clarify whether the observed performance gains stem from population attention, cost ranking, cost standardization, flow integration, teacher quality, or simply the use of a larger network.

Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer aB5c

We are thankful to the reviewer for highlighting the strengths of our work and for their valuable suggestions for demonstrating more clearly several aspects of our method.

W1 (Fixed rollout costs during flow integration)

We thank the reviewer for their point, as it highlights an important distinction between optimization iterations and flow integration.

The context is fixed by design, exactly as in MPPI and CEM. Every sampling-based planner computes its entire update from the costs evaluated at the pre-update samples, and re-evaluates only afterwards. FlowOpt Policy has the same information structure: the population Ak\mathcal{A}^k is evaluated once per update. The NτN_\tau Euler steps numerically integrate this single learned update; they are not NτN_\tau separate planning iterations. Indeed, Corollary 1 shows that MPPI and CEM correspond to one Euler step under the same fixed context.

The context CkC^k is not intended to represent the costs of the transported candidates at their intermediate positions. Instead, the original population Ak\mathcal{A}^k, together with its evaluations CkC^k, forms a fixed scored memory, in which the evaluated candidates remain at the positions where they were evaluated. Each transported candidate queries this memory using its current position and flow time, and carries the score of its own evaluated copy as a label of its origin, not as an estimate of its current cost.

More fundamentally, our revised paper now establishes a principled theoretical foundation for this design, deriving the planning formulation from a heat–proximal splitting of the Wasserstein gradient flow and providing a mathematical justification for the structure of the learned update. The training target of each update (Propositions 1–2) is determined entirely by quantities fixed before the flow starts: the perturbed population and the evaluated candidates with their costs. The optimal velocity field is therefore the conditional expectation of the source-to-target displacement given the current position, the flow time and the fixed context (Proposition 3), which Theorem 1(i) expresses in closed form as a softmax-weighted aggregation of the evaluated candidates. Costs at intermediate positions do not enter. CkC^k thus plays the role of the conditioning variable of a conditional flow, held fixed during integration.

W2 (Ablation of individual components)

We thank the reviewer for this valuable suggestion. We have added component ablations on CarNav and Push-T (Table 1), isolating the effects of population attention, cost ranking and standardization, flow integration, teacher quality and network size. Each component is varied independently while keeping the deployment rollout budget and evaluation seeds fixed.

AblationVariantCarNav cost ↓Δ vs fullPush-T SR (%) ↑Δ vs full
n/aFull model7095n/a99.0n/a
Population attentionNo attention7512+418 ± 11068.8−30.2 ± 4.8
Cost rankingZ-score only7116+21 ± 11583.9−15.1 ± 3.3
Cost standardizationRank only7169+75 ± 10292.7−6.3 ± 4.1
Flow integration1 Euler step8079+984 ± 16849.0−50.0 ± 6.2
Flow integration2 Euler steps7583+488 ± 10971.9−27.1 ± 5.5
Flow integration10 Euler steps6966−129 ± 7798.4−0.5 ± 0.9
Teacher quality4× fewer rollouts7193+98 ± 8694.3−4.7 ± 1.6
Teacher quality16× fewer rollouts7278+183 ± 9189.6−9.4 ± 3.1
Network sizeWidth 64 / 2567378+283 ± 10393.8−5.2 ± 1.8
Network sizeWidth 384 / 10246833−261 ± 12399.0+0.0 ± 1.6

The results identify population attention and the number of flow integration steps as the most influential design choices. Removing attention reduces Push-T success by 30.2 percentage points, while using a single Euler step reduces it by 50 points, with substantial cost increases on CarNav as well. Cost conditioning also contributes, particularly ranking on Push-T, while reducing teacher quality progressively degrades performance. Varying network width has more modest effects.

These experiments complement our Flow Policy and newly introduced Learned Gaussian Update (LGU) (Figure 2 and Table 2) baselines , which respectively assess the benefits of optimization using rollout feedback and learning population transport over Gaussian proposal updates.


We sincerely thank the reviewer for their constructive feedback and valuable suggestions, which motivated us to substantially strengthen both the theoretical foundations and experimental evaluation of our work. The resulting revisions have fully addressed the reviewer's points and significantly improved the paper.

In light of these improvements, we hope the reviewer will consider increasing their evaluation.

AuthorsOpenReview ↗

Follow-up on Our Revisions and Responses

As we have entered the final day of the discussion period, we wanted to briefly follow up on our response.

Both of your comments were very helpful in improving the paper. The question on the fixed context during flow integration led us to ground this design choice in a principled theoretical foundation based on the Wasserstein gradient-flow perspective, while the request for ablations resulted in a comprehensive component study that identifies which elements drive performance.

We are grateful for the suggestions, as the paper is now much clearer in its exposition and considerably stronger in its experimental evidence. Given these significant improvements, we would greatly appreciate it if you would consider raising your evaluation.

Reviewer_Kg1jOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors present a framework that casts optimal control as a transport problem, from base action distributions towards optimal action distributions, and do so iteratively by learning flows that condition on action sequences and costs and output a distribution that improves the actions. The authors state that their contribution is first conceptual (through casting the problem as a transport problem) and empirical (through the results they obtain in the empirical evaluations). They validate their method on a wide array of benchmarks.

Critical strengths
  • The authors do a good job at motivating their method, and at laying out the main insights and results.
  • The experimental section is comprehensive.
Critical weaknesses
  • The main insight around which the paper contributions pivot is that one can cast the synthesis of an optimal control policy as a transport problem, from some base (reference) policy to some optimal policy. My main issue with this is that this insight is hardly novel. See below for a point on this.
  • Some of the claims when positioning the work feel imprecise. For example, stating that in sampling based methods 'their core optimization mechanism compresses the population into a single nominal sequence' is a stretch. MPPI actually collapses to a distribution of trajectories; one simply chooses to run the mean.
  • The paper seems to omit a whole body of literature both on the bayesian/MPPI planning subfield (eg TD-MPC and subsequent work) and the RL as inference subfield (see Levine's work, and Eysenbach's work). Furthermore, such baselines are absent from the experimental results, which I believe is extremely misleading. MPPI is over 15 years old, and given that the authors method requires pre-training (in their case a flow), I find it strange that no newer method is compared (eg. TD-MPC or TD-MPC2, where computation is also amortised).
Questions for authors
  • On the novelty of the arguments. First, on the MPC literature, the entire theory behind MPPI is based around the same insight authors present as original. MPPI casts the planning problem as an inference from a prior trajectory (or policy) distribution to a desired posterior (optimal) distribution. Second, on the RL literature, the entire subfield of 'RL as inference' (see eg. Levine's work, and much of Eysenbach's work on model-based RL) is literally founded on the same point: RL can be seen as inferring a posterior optimal policy starting from some prior (reference) policy, usually taken to be a maximum entropy policy, for example. Could authors position their claim agains these bodies of work?
  • On a similar line, the theoretical results regarding MPPI and CEM feel slightly like a tautology. If I write planning as a sequential optimisation problem, such that given some objects (environment, rewards, etc) I want a map that outputs an optimal action sequence, then I can claim that my framework is a generalisation over any existing method.
3 comments
AuthorsOpenReview ↗

Response to Reviewer Kg1j (Part 1/2)

We thank the reviewer for their positive remarks on the motivation of our method and the comprehensiveness of our experiments, and for their suggestions on positioning our work more precisely within the literature.

W1 and Q1 (Novelty and theoretical foundations; connections to MPPI and RL as inference)

We are particularly thankful to the reviewer for raising this point, which motivated us to more clearly articulate the novelty of our contribution and significantly deepen its theoretical foundations.

We agree that planning as inference toward a Gibbs posterior is well established in MPPI and RL as inference. We build on this foundation rather than claim it as novel, and have revised Section 4.1 and the Related Work accordingly.

Our contribution is not a new formulation of the optimal distribution, but a learned optimization procedure that directly transports and improves candidate action sequences through a flow conditioned on their rollout costs. The inference view characterizes the target distribution, but it does not prescribe how a finite population of candidates should reach it under a rollout budget. The three lines of work differ in this aspect:

  • In MPPI and CEM, the update rule is prescribed: evaluated candidates are used to update a parametric proposal distribution, from which the next population is sampled.
  • In RL as inference, a policy is trained for a given reward and generates actions directly, without further optimization through a predictive model at deployment.
  • In FlowOpt Policy, we learn the optimization update itself. Each candidate is transported based on the evaluated population and its costs, so that different members can reach different promising regions, and the optimizer continues to evaluate and improve action sequences through the model at deployment.

Crucially, motivated by the reviewer's observation, we have also substantially strengthened the theoretical foundations of our approach. We now ground our learned optimization procedure in a Wasserstein proximal (JKO) step of the same classical free energy, whose minimizer is the Gibbs distribution, establishing a principled connection between the optimization formulation, training procedure, and architecture. Specifically, we have introduced three key theoretical results:

  1. Population re-expansion. A heat–proximal splitting yields Gaussian perturbations with variance 2λh2\lambda h, determined by the free-energy temperature and step size (Proposition 1).
  2. Training targets. Restricting the proximal substep to evaluated candidates yields a rectangular assignment balancing cost and displacement (Proposition 2), with cost and displacement guarantees (Lemma 3). Our flow learns to approximate the assignment on a higher-budget teacher's evaluated set using only the population and its rollout costs.
  3. Architecture. The optimal flow-matching field involves a posterior-weighted aggregation of candidate targets, while an entropic relaxation of the proximal step is represented exactly by cross-attention (Theorem 1). Attention is therefore mathematically motivated by the optimization formulation, rather than being an arbitrary architectural choice.

We have expanded the Related Work section to explicitly discuss the connections to control and RL as inference, including the works highlighted by the reviewer, and to clarify how our contribution differs.

W2 (Imprecise wording on sampling-based methods)

We agree that describing MPPI and CEM as collapsing the population into a single nominal sequence was imprecise, and we have corrected this wording. These methods summarize evaluated candidates through the parameters of a proposal distribution, typically updating its mean and, in some variants, covariance, before sampling the next population. Our method instead retains and individually transports the candidates, without refitting a proposal distribution. This methodological distinction is the one we intended to emphasize.

AuthorsOpenReview ↗

Response to Reviewer Kg1j (Part 2/2)

W3 (Related work and comparison with amortized model-based planners)

We thank the reviewer for highlighting the importance of better positioning our work against the literature.

We have substantially expanded and reorganized the Related Work section to discuss the foundations of control and RL as inference, including the works highlighted by the reviewer. We also discuss recent model-based planning approaches, including TD-MPC, TD-MPC2, model-based diffusion, DIAL-MPC, and learned sampling-distribution updates. These additions clarify our central distinction from prior work: rather than learning a policy that generates actions or a proposal distribution from which actions are sampled, we learn how to iteratively improve the candidate action sequences themselves, using their rollout costs to guide a population-wide transport.

We have also strengthened the experimental comparisons to address the reviewer's concern about modern learned planners. Beyond the existing comparisons with MPPI, CEM, CMA-ES, and DIAL-MPC, we have introduced the Learned Gaussian Update (LGU) baseline, inspired by Sacks and Boots (ICRA 2022). LGU learns cost-informed Gaussian proposal updates using the same evaluated populations, training targets, network backbone, and rollout budget as FlowOpt Policy. Despite being a strong baseline, LGU is outperformed by FlowOpt Policy across all six tasks (Figure 2 and Table 2). Together, our comparisons with DIAL-MPC, Flow Policy, and LGU establish the benefits of learned population transport over non-learned diffusion-style updates, direct generative policies, and learned Gaussian proposal updates, respectively.

Regarding TD-MPC/TD-MPC2, these methods retain MPPI as their underlying optimizer, with amortization entering through the learned world model, terminal value function, and policy prior that seeds some of the samples. Our contribution is complementary: we learn the optimization update itself. To evaluate this distinction, we perform experiments on Cheetah-WM and Humanoid-WM using pretrained TD-MPC2 world models, keeping the latent dynamics, reward and value functions fixed across all planners. These models were trained in conjunction with MPPI, yet FlowOpt Policy achieves the highest returns without any retraining of the world model. This isolates the benefits of the learned optimizer under the same predictive model and rollout budget.

Q2 (Theoretical connection to MPPI and CEM)

We thank the reviewer for raising this point, which motivated us to clarify and strengthen the theoretical connection with classical sampling-based planners. We agree that containment alone would be uninformative, since a sufficiently expressive map contains every optimizer. Our new theoretical statement is narrower and runs in the opposite direction: the attention update follows from the Wasserstein proximal step of the free energy, and MPPI and CEM arise as its limit for large step size.

Our newly added Theorem 1(ii) establishes that an entropic relaxation of the Wasserstein proximal step admits an exact representation through cross-attention, with logits combining a transport term scaled by the step size hh and a cost term. As h→∞h\to\infty, the transport-dependent terms vanish, recovering query-independent cost weighting. Corollary 1 (Proposition 1 in the original submission) then explicitly constructs the MPPI and CEM updates: two attention heads compute the cost-weighted and unweighted population means, whose difference defines a shared displacement that recovers the classical mean update in one Euler step. For finite hh, the weights additionally depend on each candidate's position, allowing different members to be drawn toward different promising regions. Conversely, every dot-product attention layer of this form solves a KL-regularized step (Remark 1).

Our revised statement is therefore substantially more informative, as it identifies the restriction that classical planners impose, namely cost weights shared by all members, which our learned attention removes by letting the weights depend on each member. The effect is visible empirically in our new results, where the newly added LGU baseline, which learns the update but retains the Gaussian proposal, attains 86.5% success on Push-T, against 60.4% for MPPI and 99.0% for FlowOpt Policy (Figure 2 and Table 2).


We are grateful to the reviewer for their thoughtful feedback, which motivated us to revisit the foundations of our approach and develop a substantially more principled theoretical formulation. Their suggestions also prompted us to better position our contribution within the literature and strengthen our experimental evaluation. In the resulting revisions, we have addressed all of the reviewer's points.

In light of these improvements, we hope the reviewer will recognize the distinct perspective and contributions of our work, and consider increasing their evaluation.

AuthorsOpenReview ↗

Follow-up on Our Revisions and Responses

As we have entered the final day of the discussion period, we wanted to briefly follow up on our response.

We found your feedback particularly insightful, as it motivated us to significantly strengthen the theoretical foundations of our work, clarify the novelty of FlowOpt Policy within the control and RL as inference literature, as well as expand our experimental evaluation.

We believe these revisions make much clearer the distinct perspective and contributions of our work at the intersection of model predictive control, generative modeling and learned optimization. Following these substantial improvements, we hope that you will consider raising your evaluation.

Reviewer_TcusOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FlowOpt Policy learns to update a population of candidate action sequences using their rollout costs. A conditional flow transports the candidates toward improved solutions, with attention allowing each candidate to use information from the rest of the population. Training targets come from a more expensive optimizer, and flow matching learns the corresponding updates. At deployment, the learned optimizer repeatedly evaluates and refines candidates under the current objective. Experiments cover four simulator-based tasks and two tasks with learned world models, with comparisons to sampling-based optimizers and a directly conditioned flow policy.

Critical strengths
  1. The population update has a clear methodological rationale. Sections 4.2–4.4 combine cost-conditioned attention, candidate-specific transport, and optimizer-generated training targets into a coherent learned planning procedure. Proposition 1 in Section 4.5 shows that restricted attention patterns can reproduce the MPPI/CEM weighted population mean. This is a useful connection between classical cost-weighted aggregation and the proposed richer update rule. Its scope is the mean update, rather than the full CEM distribution update or a convergence guarantee.

  2. The experiments provide meaningful evidence for optimization at deployment. Figure 2 reports benefits across both simulators and learned world models. The Flow Policy baseline uses the same architecture and training procedure, making it a useful control for the value of cost feedback at deployment. Figure 4 further shows that FlowOpt with three iterations can outperform MPPI and CEM with nine iterations on CarNav and Push-T. The obstacle, moving-target, and dynamics-shift experiments in Section 5.3 provide additional evidence that deployment optimization helps beyond the training conditions.

Critical weaknesses

W1. The comparison does not establish the benefit of the proposed update rule over existing learned MPC optimizers. The distinctive design in Section 4 is to learn candidate-specific population transport, whereas earlier methods already learn cost-informed updates to a planning distribution, where examples include Sacks and Boots (ICRA 2022) and DMPO (ICRA 2024). Section 5 compares against hand-designed optimizers and a direct policy, but not a learned optimizer that receives rollout feedback. The Flow Policy control establishes the value of feedback over direct generation, but it does not determine whether population transport improves on learning the proposal-distribution update. This is the main limitation in assessing the paper's contribution relative to its closest methodological alternatives.

W2. The scope of objective generalization is less established than the abstract and contribution statement suggest. The changed obstacle layouts and moving targets in Section 5.3 are useful tests of new objective instances. However, they provide limited evidence about adapting to a changed cost specification. Table 3 tests an added action-change penalty and reports lower mean stage cost than Flow Policy, but compares only against that direct policy. Without a reference planner optimizing the same modified objective, this result leaves the quality of objective adaptation unclear.

Questions for authors

Q1 (W1). How does FlowOpt compare with a learned optimizer that uses the same candidate costs and training targets to update a Gaussian proposal's mean and variance, under a comparable rollout budget? This would test whether transporting individual candidates adds value beyond learning the proposal update.

Q2 (W2). For the modified action-change penalty in Table 3, how does FlowOpt's cost under the new objective compare with a sufficiently optimized CEM or MPPI reference? Does the low completion rate reflect the new objective's preferred tradeoff or difficulty adapting to it? This would clarify the supported scope of the unseen-objective claim.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Tcus

We thank the reviewer for highlighting the strengths of our work and for their valuable suggestions, which have strengthened our experimental evaluation and enabled us to demonstrate the benefits of our approach more clearly.

W1/Q1 (Comparison against learned Gaussian update)

We thank the reviewer for this recommendation. We agree that comparing against a learned Gaussian proposal update is important for establishing the benefit of transporting candidates via a learned flow.

We have addressed this by adding a "Learned Gaussian Update" (LGU) baseline in the spirit of Sacks and Boots (ICRA 2022), which learns to update the mean and variance of a Gaussian proposal. LGU uses the same evaluated candidates, rollout costs, teacher-generated training targets, and rollout budget as FlowOpt Policy. Both methods also share the same network backbone, with the per-candidate velocity output replaced by a pooled prediction of the Gaussian parameters. Therefore, this comparison directly isolates the benefits of learning to transport the population over learning Gaussian proposal updates.

The experiments confirm the advantage of our approach. LGU is a strong baseline, performing competitively across all six tasks. Nevertheless, FlowOpt Policy outperforms LGU on every task (updated performance profiles in Figure 2 and aggregate metrics in Table 2, also shown below). The advantage is particularly pronounced on Push-T, with 99.0% versus 86.5% success, and Humanoid-WM, with 1.00 versus 0.88 normalized return. These results provide direct empirical evidence for the benefits of learning to transport candidates rather than updating a Gaussian sampling distribution.

Task (metric)LGUFlowOpt Policy (Ours)
CarNav (Jnorm↓J_{\mathrm{norm}}\downarrow)1.06 ± 0.371.00 ± 0.33
Cheetah (Rnorm↑R_{\mathrm{norm}}\uparrow)0.90 ± 0.071.00 ± 0.09
Walker (IQM Jnorm↓J_{\mathrm{norm}}\downarrow)1.071.00
Push-T (SR % ↑\uparrow)86.5 ± 11.099.0 ± 0.9
Cheetah-WM (Rnorm↑R_{\mathrm{norm}}\uparrow)0.95 ± 0.161.00 ± 0.09
Humanoid-WM (Rnorm↑R_{\mathrm{norm}}\uparrow)0.88 ± 0.121.00 ± 0.08

W2/Q2 (Generalization to different objectives)

We thank the reviewer for this valuable observation, which motivated us to substantially strengthen our evaluation of generalization to unseen objectives.

First, we have expanded our generalization experiments (Figure 3) to include equal-budget MPPI and CEM, alongside the higher-budget teacher as a reference optimizer. These comparisons cover changes in obstacle configurations on CarNav, and initial-state distributions and friction on Push-T. Across these shifted settings, FlowOpt Policy outperforms the equal-budget planners and approaches the performance of the higher-budget reference.

More importantly, we have added a systematic evaluation of generalization to an unseen objective (Table 5) by introducing a control-rate penalty on Push-T, absent during training, and varying its weight over a broad range. All model-based planners optimize the modified objective, while MPPI and CEM retain their originally tuned hyperparameters. We report the modified objective cost, control-rate term, and success rate separately, allowing us to examine the tradeoff highlighted by the reviewer. We also separate task completion from control effort in the revised evaluation, reporting success based on the final block pose rather than additionally imposing a control-effort cutoff used in the initial submission.

The new results provide strong evidence of generalization to unseen objectives. At wΔu=3×10−3w_{\Delta u}=3\times10^{-3}, FlowOpt Policy achieves 88% success, compared with 98% for the high-budget reference, 54% for MPPI, and 39% for CEM, with a mean cost of 0.35 versus 0.31 for the reference. As the penalty increases, FlowOpt Policy substantially reduces its control-rate term, reaching 1.4 at the largest penalty, compared with 7.1 under the training objective. Even at this largest penalty, it achieves 71% success, compared with 79% for the reference, 25% for MPPI, and 12% for CEM, while attaining a modified-objective cost of 0.51 versus 0.47 for the reference.

These results show that reduced success at larger penalties partly reflects the tradeoff introduced by the objective, as the high-budget reference exhibits a similar trend. Notably, FlowOpt Policy adapts to the modified objective without retraining, achieving costs close to the high-budget reference while substantially outperforming equal-budget MPPI and CEM.


We thank the reviewer for their valuable feedback, which motivated us to substantially strengthen our experimental evaluation with new baselines and generalization results. These additions have directly addressed the reviewer's points and significantly improved the paper.

We hope the reviewer will consider increasing their evaluation in light of these revisions.

Reviewer_TcusOpenReview ↗

Response to authors

Thanks for adding the LGU comparison. It provides a more direct test of the proposed population update and strengthens the case for transporting individual candidates. This addresses W1.

Table 5 also answers my question about the modified objective. FlowOpt reduces the control-rate term as its weight increases and achieves costs close to the higher-budget reference across the tested weights. Success also drops for the reference at larger penalties, supporting the interpretation that part of the decline reflects the changed objective. Separating pose-based success from control effort makes this tradeoff easier to assess. Please document the correction to the success criterion in the paper, since the original and revised success rates are not directly comparable.

I raised my score accordingly.

We really thank the reviewer for their positive reassessment and for raising their score! We are glad that our responses have addressed their concerns, and we will clarify the adjustment to the success criterion in the revised paper, as suggested.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses to Reviewers

We sincerely thank all three reviewers for their thoughtful feedback and valuable suggestions, which have motivated substantial improvements to our paper. We have carefully addressed all the reviewers' points in our responses and revised manuscript. To facilitate the assessment of these revisions and responses, we briefly summarize them here:

  • Methodology framing and principled theoretical foundation (Kg1j and aB5c). We have sharpened the novelty and positioning of our contribution within the control and RL as inference literature and established a theoretical foundation based on the Wasserstein gradient flow of the classical free energy to show how our method connects to this literature and how our algorithmic componencts are justified. Specifically:

    (i) Novelty: While control and RL as inference characterize the optimal distribution, our contribution is to learn the optimization procedure itself via a flow that iteratively transports individual candidate action sequences toward lower cost, conditioned on the population and its rollout costs.

    (ii) Principled theoretical foundation: We have substantially strengthened the theoretical framing of FlowOpt Policy by grounding the method in a heat-proximal splitting of the Wasserstein gradient flow of the free energy, connecting our approach to classical control and RL as inference through a common variational foundation. In addition, this formulation also provides a mathematical basis for the population perturbations, training targets and attention-based flow architecture, while also justifying the fixed scored context used throughout flow integration.

    (iii) Connection to classical planners: We have added Theorem 1, showing that attention arises naturally from both the optimal flow-matching field and an entropic relaxation of the Wasserstein proximal step, further motivating the design choices in our architecture. We also establish that classical cost-based weighting emerges when the transport-dependent terms vanish, and explicitly recover the weighted population-mean updates of MPPI and CEM as restricted attention patterns (Corollary 1), rather than relying on a generic expressiveness argument.

  • Modern learned sampling-based optimizer comparison (Tcus, Kg1j and aB5c). We have added a Learned Gaussian Update (LGU) baseline, in the spirit of learning-based MPC methods such as Learning to Optimize in MPC (Sacks and Boots, ICRA 2022) and DMPO (Sacks et al., ICRA 2024), which learns to update the mean and variance of a Gaussian proposal using the evaluated candidates and their rollout costs. With the same training targets, network backbone, and rollout budget, this directly isolates the benefits of learning to iteratively optimize via population transport over Gaussian proposal updates. FlowOpt Policy outperforms LGU across all six tasks. Together, our comparisons with DIAL-MPC, Flow Policy, and LGU establish the benefits of learned population transport over modern methods in the literature that rely on non-learned diffusion-style updates, direct generative policies, and learned Gaussian proposal updates, respectively.

  • Systematic component ablations (aB5c). We have added extensive ablations on CarNav and Push-T, isolating the contributions of population attention, cost ranking and standardization, flow integration, teacher quality and network capacity. The results identify population attention and multi-step flow integration as the most influential design choices.

  • Generalization to different objectives (Tcus). We have expanded our generalization experiments with equal-budget MPPI/CEM and a higher-budget reference optimizer, and added a systematic evaluation of unseen objectives by varying a control-rate penalty on Push-T that was absent during training. The results demonstrate that FlowOpt Policy adapts very well to changes in the objective without retraining, achieving costs close to the higher-budget reference while substantially outperforming equal-budget planners as well as Flow Policy.

We are really grateful to all three reviewers for their thoughtful feedback, which has motivated us to substantially strengthen our paper, from its methodological positioning and theoretical foundations to its experimental evaluation. We hope these revisions make very clear the distinct perspective and contributions of our work at the intersection of model predictive control, generative modeling and optimization.

We are pleased to see that reviewer Tcus has already provided a positive reassessment and increased their score. In light of these revisions and responses, we look forward to hearing from the other reviewers as well, and hope they will consider reassessing their evaluations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.