acceptodds
Under review as a conference paper at ICLR 2027

SCMO: Stochastic Control for Optimization over Probability Measures on Infinite-Dimensional Spaces

Abstract

We study objective-only optimization of possibly nonconvex and nonsmooth functionals over probability measures on a separable Hilbert space, allowing the optimizer to be intrinsically non-Dirac. We introduce SCMO (Stochastic Control Measure Optimizer), a gradient-free particle method derived from entropy-regularized stochastic control. After finite-particle and Galerkin approximations, a Cole–Hopf transform represents the optimal feedback as a Gibbs-weighted terminal displacement. SCMO approximates this feedback by sampling context clouds, replacing one particle at a time with candidate draws, scoring the resulting empirical measures, and applying exponential reweighting. SCMO uses separate covariances for candidate proposals and particle updates, termed matched when equal and nonmatched otherwise; our analysis covers both settings. In the matched case, we establish PDE-free qualitative convergence and a projection-first quantitative bound separating particle, Galerkin-projection, and entropy-regularization errors. For the practical multi-context implementation with nonmatched covariance, we prove finite-candidate consistency and show that the signed, curvature-dependent effect of nonmatched covariance can reduce the resulting upper bound on the approximation error. Finally, experiments on function-space, trajectory-law, and contact-rich manipulation problems show that SCMO handles nonsmooth and nonconvex objectives, escapes suboptimal local basins, and recovers prescribed multimodal, non-Dirac law structure.

Primary area:
optimization
Keywords:
Optimization over probability measures, stochastic optimal control, derivative-free optimization, infinite-dimensional optimization, particle methods, nonsmooth nonconvex optimization.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EW3VOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SCMO, a stochastic-control-based optimization framework for probability measures on infinite-dimensional Hilbert spaces. Building on an entropy-regularized optimal control formulation, the method employs finite-particle and Galerkin approximations to derive a gradient-free optimization algorithm. The paper also provides extensive theoretical analyses of convergence and approximation errors, together with empirical evaluations across several optimization settings.

Critical strengths
  1. The proposed framework is well motivated and mathematically principled. Its algorithmic design follows naturally from the underlying stochastic optimal control formulation, making the overall approach reasonable and convincing.
  2. The paper provides substantial theoretical analysis, covering convergence of the idealized optimization problem, particle and projection approximation errors, and finite-candidate implementation errors. This theoretical development is a notable strength of the work.
  3. The experimental results demonstrate the effectiveness of the proposed method across different optimization settings, providing further support for its practical applicability.
Critical weaknesses

Overall, I consider this a technically solid and well-developed paper, with meaningful theoretical contributions. My main concerns are related to the positioning of its contributions and the completeness of the experimental comparisons.

  1. The proposed framework is closely related to the work of Qiu (2026), as acknowledged in the introduction. In particular, the fundamental stochastic control formulation and the derivation of gradient-free optimization through optimal feedback appear to build directly upon this prior work. Although the present paper provides meaningful extensions, particularly in its infinite-dimensional formulation and theoretical analysis, I believe the authors could more explicitly clarify the essential technical differences and contributions beyond Qiu (2026).

  2. The current baselines appear somewhat limited for assessing the improvements over the most closely related prior work. In particular, a direct comparison with Qiu (2026) would help demonstrate the practical benefits of the proposed extensions. I encourage the authors to consider including such experiments, ideally under comparable evaluation budgets. A discussion or comparison with relevant mean-field optimization methods would also help better position the proposed approach.

  3. Since the theoretical development relies on mean-field control and value-function approximation, it would be helpful to further discuss how the proposed theoretical framework differs from existing mean-field optimization approaches. In particular, clarifying the distinctive advantages of the stochastic optimal control perspective, both theoretically and algorithmically, would strengthen the contribution.

I would like to note that, given the complexity and length of the theoretical development, I have not independently verified all the proofs in detail. My positive assessment of the theoretical component is therefore primarily based on the overall proof structure, mathematical formulation, and consistency of the proposed analytical approach, rather than a complete verification of every technical argument. Despite these concerns, my overall assessment is positive. I believe the paper makes a meaningful contribution, and addressing the points above would further strengthen its presentation and empirical justification. With these improvements, I would be inclined to support a clear acceptance.

Questions for authors
  1. Could the authors clarify the key technical and theoretical advances over Qiu (2026), particularly which results require substantially new analysis beyond the existing stochastic control framework?
  2. Could the authors provide direct experimental comparisons with Qiu (2026), preferably under comparable computational budgets, to better demonstrate the benefits of the proposed extensions?
  3. Could the authors further discuss the relationship between the proposed theoretical framework and existing mean-field optimization methods, including the specific advantages of using the stochastic optimal control formulation?
5 comments
AuthorsOpenReview ↗

Response to Reviewer EW3V (Part 1/3)

We thank the reviewer for recognizing the theoretical contributions and empirical effectiveness of our work. We have revised the manuscript, with changes highlighted in blue, to clarify the advances beyond Qiu (2026), add direct comparisons, and discuss related mean-field methods. We hope these revisions address the remaining concerns and support a clear acceptance.

Q1. Key technical and theoretical advances over Qiu (2026)

We build on the entropy-regularized control formulation and Cole–Hopf representation of Qiu (2026), but make four further advances:

1. Infinite-dimensional formulation and projection-first analysis. We study P2(H)\mathcal P_2(\mathcal H) on a separable Hilbert space using trace-class QQ-Wiener noise and Galerkin projection onto a KK-dimensional subspace. Theorem 4.2 gives a projection-first error bound separating particle, projection, and entropy-regularization errors. It controls the projection tail, projected Cameron–Martin cost, and dimension-dependent term εKlog⁡(1/ε)\varepsilon K\log(1/\varepsilon), with conditions for joint convergence as K,N→∞K,N\to\infty and ε↓0\varepsilon\downarrow0. These estimates cannot be obtained by directly letting the dimension grow in Qiu's fixed-dimensional bounds.

2. Nonmatched proposal and execution covariances. Motivated by numerical observations, we distinguish the proposal covariance QpropQ_{\rm prop} used to estimate feedback from the execution covariance QdynQ_{\rm dyn} used in particle dynamics. This departure from the matched control problem requires new analysis. Theorem 4.3 and Appendix E.1 derive a signed covariance correction whose leading term involves Tr⁡[(Qdyn−Qprop)D2v]\operatorname{Tr}[(Q_{\rm dyn}-Q_{\rm prop})D^2v]. Under favorable curvature conditions, this sharpens the upper error bound, without implying that nonmatching always helps. See also Appendix A.2 (Table 2).

3. PDE-free qualitative convergence for less regular objectives. Theorem 4.1, under Assumption 4.1 (robust recovery), proves convergence of normalized idealized projected-particle values to the global infimum along suitable joint sequences K,N→∞K,N\to\infty and ε↓0\varepsilon\downarrow0. It requires neither classical HJB/Master-equation solutions, Lions differentiability, nor convexity, and includes certain nonsmooth and discontinuous objectives excluded by the quantitative theory.

4. Multi-context feedback estimation: resolving a frozen-background limitation. Qiu (2026) gives both a full-cloud Monte Carlo scheme (Eq. (3.11)) and a cheaper single frozen-background approximation (Section 3.3, Eqs. (3.14)–(3.17)). For interacting objectives, the latter estimates feedback conditional on a sampled background, not the unconditional full-cloud feedback. Increasing the candidate count SS alone does not remove this discrepancy: with a fixed background, the S→∞S\to\infty limit remains conditional.

Our estimator uses RR independent context clouds and SS candidates per context, with joint Gibbs normalization across all RSRS scores. Independent contexts, rather than more candidates in one context, are the essential improvement. Proposition E.1 establishes almost-sure consistency for the full-cloud feedback as R→∞R\to\infty, even for fixed S≥1S\geq1, and a mean-square error bound O(1/R+1/(RS))O(1/R+1/(RS)) under suitable moment and denominator-stability conditions. Theorem E.5 propagates the estimation error through one rollout, controlling the mean-square difference between the finite-candidate algorithm and the exact-feedback Euler chain.

Appendix A.4 (Table 3 and Figure 3) gives an interacting quadratic example where the ratio of the expected one-context feedback to exact feedback approaches NN as ε↓0\varepsilon\downarrow0. Under the same RS=256RS=256 replacement-score budget, increasing contexts from (R,S)=(1,256)(R,S)=(1,256) to (32,8)(32,8) lowers the mean final objective from 0.182220.18222 to 0.001410.00141. This establishes a concrete benefit of multi-context estimation in this example; our comparison concerns Qiu's localized frozen-background approximation, not its full-cloud scheme.

In summary, beyond the original control formulation, we provide Hilbert-space convergence bounds, nonmatched-covariance analysis, PDE-free qualitative convergence, and a consistent multi-context estimator with implementation guarantees.

AuthorsOpenReview ↗

Response to Reviewer EW3V (Part 2/3)

Q2. Direct experimental comparisons with Qiu (2026)

In response to the reviewer's suggestion, we have added direct experimental comparisons with Qiu (2026), together with a new particlewise random-search baseline, across five trajectory-law optimization tasks (T1–T5). These tasks examine narrow feasibility, multimodal law recovery, discontinuous interacting objectives, and escape from suboptimal local basins. The results are reported in Table 1 (Panel B) and Appendix F.3 (Table 15) of the revised manuscript.

1. Experimental setup and computational fairness.

We implement the localized frozen-background method of Qiu (2026, Section 3.3), using a single context cloud (R=1R=1) and matched proposal and execution covariances. SCMO instead uses multiple contexts (R=4R=4) and allows nonmatched covariances.

All methods use the same initialization protocol and 20 paired evaluation seeds, with N=128N=128 particles representing 42-step trajectories. We impose the same maximum computational budget of 58,256,768 trajectory simulations per method. SCMO and Qiu also use the same number of candidate replacements per particle per update (RS=384RS=384), with (R,S)=(4,96)(R,S)=(4,96) for SCMO and (1,384)(1,384) for Qiu. Hyperparameters are tuned separately on development runs and frozen before evaluation. Actual evaluation counts may differ because of stopping criteria.

2. A new particlewise random-search baseline.

To investigate whether SCMO's performance can be explained by straightforward candidate selection, we introduce a particlewise random-search baseline. This method uses R=4R=4 contexts and S=96S=96 candidates per context, selects the candidate–context pair with the lowest objective for each particle, and simultaneously replaces all particles.

Unlike SCMO, this baseline uses hard candidate selection without Gibbs weighting, feedback relaxation, or execution noise. It therefore provides an informative comparison with a simple derivative-free particlewise search strategy. We also retain the whole-cloud random-search baseline from the original submission, alongside CEM and Mixture MPPI.

3. Main experimental findings.

SCMO achieves a lower mean objective than Qiu (2026) on four of the five tasks, with particularly substantial improvements on the more challenging problems.

T2 (narrow passage): SCMO achieves 1.097±0.0281.097\pm0.028, compared with 5.415±0.1945.415\pm0.194 for Qiu and 2.631±0.1172.631\pm0.117 for particlewise random search. SCMO also achieves successful trajectory mass 0.996±0.0060.996\pm0.006, compared with 0.371±0.0530.371\pm0.053 for Qiu and 0.766±0.0350.766\pm0.035 for particlewise random search.

T3 (multimodal recovery): SCMO achieves 0.0796±0.00580.0796\pm0.0058, compared with 2.156±0.1702.156\pm0.170 for Qiu. SCMO and particlewise random search satisfy the strict full-success and two-route criterion in 20/20 runs, whereas Qiu does so in 0/20 runs. Particlewise random search achieves a slightly lower mean objective (0.06600.0660).

T4 (discontinuous interacting quotas): SCMO achieves −0.4820±0.0084-0.4820\pm0.0084, compared with 1.932±0.1871.932\pm0.187 for Qiu. All three particlewise methods satisfy the structural criterion in 20/20 runs, with SCMO and particlewise random search achieving nearly identical mean objectives.

T5 (escape from a suboptimal basin): SCMO achieves 2.71519±0.000082.71519\pm0.00008, compared with 2.8086±0.00202.8086\pm0.0020 for Qiu and 2.775530±0.0000122.775530\pm0.000012 for particlewise random search. SCMO discovers the better upper route in all 20 runs, whereas Qiu and particlewise random search remain in the inferior lower-route basin in every run.

On the simpler T1 task, both Qiu and particlewise random search outperform SCMO in mean objective, while all three satisfy the structural success criterion in 20/20 runs. We therefore do not claim uniform superiority.

4. Significance of the additional comparisons.

These results provide two complementary findings.

First, SCMO substantially improves upon the localized method of Qiu (2026) on several challenging trajectory-law optimization problems under the same maximum computational budget.

Second, the particlewise random-search comparison demonstrates that hard candidate selection alone does not explain SCMO's performance across all tasks. While random search performs particularly well on T1 and T3 and matches SCMO closely on T4, SCMO achieves substantially higher successful mass on T2 and consistently escapes the inferior local basin on T5.

These comparisons evaluate the combined contributions of multi-context estimation, Gibbs-weighted feedback, and nonmatched execution covariance; they do not isolate each component's causal effect. The roles of multi-context sampling and covariance nonmatching are further investigated in Appendix A.4 (Table 3 and Figure 3) and Appendix A.2 (Table 2), respectively.

We believe these additional experiments directly address the reviewer's concern and provide stronger empirical evidence for the practical benefits of our proposed extensions.

AuthorsOpenReview ↗

Response to Reviewer EW3V (Part 3/3)

Q3. Relationship to existing mean-field optimization methods and advantages of stochastic optimal control

We have expanded Appendix B.1 and added a five-method comparison in Appendices B.2–B.4 (Table 5 and Figure 4). Our main distinction is that stochastic control provides both a principled derivation of the algorithm and mathematical tools for analyzing its approximations.

1. A control-theoretic formulation.

We reformulate optimization over P2(H)\mathcal P_2(\mathcal H) as a finite-horizon stochastic control problem, with GG evaluated at the terminal law and a quadratic drift-energy penalty. This allows us to use established techniques from dynamic programming, HJB equations, Feynman–Kac representations, and FBSDE theory. After particle and Galerkin approximation, the Cole–Hopf transform yields a computable Gibbs-weighted feedback (Equation (7)), leading to the SCMO update (Equation (9)).

2. A theoretically derived, gradient-free algorithm.

Unlike the Wasserstein gradient flow, mean-field Langevin, and EFP implementations considered in Appendix B, SCMO requires only evaluations of GG, not its first variation or spatial derivatives. It can therefore handle nonsmooth, discontinuous, or simulator-based objectives.

The control representation also determines the structure of the update: the remaining-time horizon determines the proposal covariance, Gibbs weights arise from the value-function representation, and weighted terminal displacements determine the feedback drift. For interacting objectives, our multi-context estimator consistently approximates the unconditional full-cloud feedback rather than feedback conditional on one frozen background.

3. A unified framework for convergence and approximation analysis.

The control formulation allows us to distinguish different levels of approximation.

Theorem 4.1 establishes PDE-free qualitative convergence of idealized projected-particle values under robust recovery, including certain nonsmooth and discontinuous objectives.

Theorem 4.2 separates particle, Galerkin-projection, and entropy-regularization errors under stronger classical regularity assumptions.

Theorem 4.3, Proposition E.1, and Theorem E.5 address the signed covariance correction, finite-candidate feedback consistency, and one-rollout approximation to the exact-feedback Euler chain under the stated assumptions. In particular, the HJB formulation explains how nonmatched execution covariance can sharpen the approximation bound through the curvature of the continuation value.

These results do not imply unconditional global convergence of the finite implementation.

4. Relationship to existing mean-field methods.

Wasserstein gradient flow transports particles according to the spatial gradient of the objective's first variation, whereas SCMO constructs feedback from objective evaluations.

Mean-field Langevin and EFP, in our comparison, approximate G(μ)+λKL(μ∥γv)G(\mu)+\lambda\mathrm{KL}(\mu\Vert\gamma_v) and require derivatives. SCMO instead uses a finite-horizon control problem with a quadratic drift-energy penalty, which has a path-space relative-entropy interpretation under suitable conditions. The original objective GG remains the optimization target, while the auxiliary regularization bias is controlled theoretically.

Measure-valued CBO is also derivative-free but moves competing probability laws toward a Gibbs-weighted Wasserstein barycenter, rather than estimating a time-dependent optimal control feedback.

SCMO also avoids the inner Langevin sampling used in our EFP implementation. Its Gaussian proposals and objective evaluations can be batched, although evaluating multiple candidate empirical laws incurs computational costs.

5. Additional experimental comparison.

We compare particle Wasserstein gradient flow, mean-field Langevin, EFP, measure-valued CBO, and SCMO on a two-well objective whose unique minimizer is the non-Dirac law μ⋆=12δ−1+12δ1\mu^\star=\frac12\delta_{-1}+\frac12\delta_1. All methods start near one well and must discover the other while recovering the correct mass distribution.

SCMO attains the lowest mean final objective, 0.00714±0.002490.00714\pm0.00249, compared with 0.06693±0.016310.06693\pm0.01631 for EFP, 0.14546±0.032590.14546\pm0.03259 for Langevin, 0.656850.65685 for gradient flow, and 0.661340.66134 for CBO.

SCMO and EFP satisfy the recovery criterion in 20/20 runs, whereas the other methods do so in 0/20. EFP achieves a slightly smaller mean Wasserstein distance, so we do not claim uniform superiority.

In summary, the distinctive advantage of SCMO is the combination of a control-theoretic derivation, gradient-free implementation, and rigorous approximation analysis. The additional comparison illustrates its effectiveness in recovering a non-Dirac optimal law.

Reviewer_BnQVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors extend the stochastic-control approach to optimization in Qiu (2026), from probability measures on R^d to measures on function spaces. Functions are truncated to finitely many basis coefficients.

As in Qiu (2026): particle cloud moves under drift+noise, a change of variables (via Cole-Hopf transform) is used to write the best drift as a weighted average over random perturbations.

The proposed method: SCMO estimates the drift by trying replacements for each particle. Different from Qiu (2026), SCMO uses context clouds to score each candidate inside perturbed copies, and also allows different noise for candidate proposals and particle motion.

Main contribution: the authors prove that the value of an idealized SCMO version (one using the same noise for proposing candidates and moving particles) approaches the true optimum as the number of particles and basis coefficients grow, while regularization shrinks, all at appropriate rates. Qiu (2026) results are extended to functions. The paper also proves the drift estimate in the practical version of SCMO is consistent.

Experiments: function-space PDE laws, path planning, Push-T task. SCMO got the lowest mean objective on most of these tasks, and returned a distribution with the required structure (e.g. the right proportion of routes/phases) at least as often as the baselines (see Table 1).

Critical strengths
  • Extends Qiu (2026) to optimizing over measures on function spaces; theory seems careful (truncation error bounded separately).

  • Reuses a number of theoretical components/tools from Qiu (2026), but the construction and proof setup seem self-contained without undue reliance on prior work (though I did not verify the proof correctness).

Critical weaknesses
  • Most experiments use very low temperatures (as low as 1e-10). At such temperatures SCMO can behave like a simple search: for each particle, try many random replacements and move toward the one that most improves the whole cloud's objective. A relevant baseline is not provided.

  • The paper criticizes the dimension dependence of Qiu (2026). But the error bound (Theorem 4.2) still grows with the number of basis coefficients. More experiments are needed to show advantage over fixed truncation.

  • Substantial revisions to the writing can improve the readability of this paper for the ICLR audience.

Questions for authors

Introduction motivates SCMO by talking about multimodal robot policies and function-space generative models. But experiments are small: 32 basis coefficients, planar paths, and Push-T with 8 plans from one reset. Since each candidate is scored on the whole cloud, would SCMO stay affordable at realistic scales?

Do the guarantees cover SCMO as it is actually run (multi-context, different noise for proposals and moves)? If only partly, the caveats should be stated clearly.

6 comments
AuthorsOpenReview ↗

Response to Reviewer BnQV (Part 1/2)

We thank the reviewer for the thoughtful assessment and for recognizing the importance of the Hilbert-space extension. We have revised the manuscript to address the concerns, with all changes highlighted in blue. We have also added comparisons with Qiu (2026), random-search and temperature controls, and new sampling and Galerkin-resolution experiments. Our derivations and proofs are self-contained in Appendices A, D, and E.

W1. Low temperatures and hard candidate selection

We agree that very small ε\varepsilon makes Gibbs weights approach hard selection. However, SCMO is not equivalent to directly replacing particles with their best candidates: it uses time-dependent relaxed feedback and independent execution noise.

To examine this concern, we added particlewise random search (Appendix F.3, Table 15). It evaluates R=4R=4 contexts and S=96S=96 candidates per context, but directly replaces each particle with its lowest-scoring candidate, without Gibbs weighting, feedback relaxation, or execution noise. All methods share the same maximum budget of 58,256,768 simulated trajectories, with 20 paired seeds.

SCMO substantially outperforms particlewise random search on the challenging T2 and T5 tasks. On T2, SCMO achieves a mean objective of 1.097±0.0281.097\pm0.028, compared with 2.631±0.1172.631\pm0.117 for random search, and successful mass of 0.996±0.0060.996\pm0.006 versus 0.766±0.0350.766\pm0.035. On T5, SCMO discovers the better upper route in 20/20 runs, whereas random search remains in the inferior lower-route basin in all 20 runs. Conversely, random search achieves lower mean objectives on T1 and T3 and performs comparably on T4. Thus, although SCMO is not uniformly superior, its improvements on challenging tasks can be substantial, particularly in feasibility recovery and escaping suboptimal basins. SCMO also outperforms the localized Qiu (2026) baseline on T2–T5 under the same maximum budget (Table 1, Panel B).

We further added a temperature-sensitivity experiment (Appendix C.1, Table 6). With other update components fixed, ε=10−2\varepsilon=10^{-2} achieves 0.00124±0.001480.00124\pm0.00148, compared with 0.00903±0.009750.00903\pm0.00975 at ε=10−10\varepsilon=10^{-10} and 0.00962±0.009680.00962\pm0.00968 for hard-minimum selection. Thus, finite-temperature weighting can be beneficial even when near-hard selection works well. Together, these experiments show that SCMO can substantially outperform simple hard candidate selection, although they do not isolate Gibbs weighting as the sole source of improvement.

W2. Dimension dependence and Galerkin approximation

We do not claim dimension-free bounds. Theorem 4.2 retains the term εKlog⁡(1/ε)\varepsilon K\log(1/\varepsilon). Our projection-first analysis instead identifies how particle, projection, and regularization errors must be controlled jointly as K,N→∞K,N\to\infty and ε↓0\varepsilon\downarrow0, rather than simply increasing KK in a fixed-dimensional estimate.

We added Galerkin-resolution experiments in Appendix C.3 (Tables 7–8, Figures 6–7). Increasing KK from 4 to 128 with fixed temperature and numerical budget reduces the projection floor, but worsens the mean terminal objective from 0.105860.10586 to 0.130040.13004. With joint refinement of KK, ε\varepsilon, and Euler resolution, the mean improves from 0.062660.06266 (K=4K=4) to 0.032510.03251 (K=17K=17), then levels off. This improvement requires more computation, and the experiment does not isolate individual parameter effects.

Appendix C.2 (Figure 5) further shows that larger NN and KK can require larger RSRS budgets for comparable feedback accuracy; some configurations fail to reach the accuracy target. These findings clarify the remaining dimension and sampling limitations.

W3. Readability

We apologize that the original manuscript was difficult to follow. Our work combines stochastic analysis, optimal control, and variational analysis on Wasserstein space. Connecting these areas is technically challenging and an important part of our contribution. Nevertheless, we recognize that the main ideas and their connections could have been explained more clearly.

We have revised the abstract and Section 3.3 to define context clouds and candidate replacements before their sampling formulas. New Figures 1–2 illustrate the control reformulation and six-stage SCMO update, including repeated rollouts. We have also added the definition of the Lions derivative in Appendix A.1, with a cross-reference from Section 3.1. We have also rewritten the introduction to Section 4 to distinguish idealized-value convergence from finite-candidate implementation analysis and improved the experimental presentation. All revisions are highlighted in blue.

We hope these changes make the manuscript easier to follow, and we would be grateful for any further suggestions on improving its presentation.

AuthorsOpenReview ↗

Response to Reviewer BnQV (Part 2/2)

Q1. Applicability and computational affordability at realistic scales

We acknowledge that our experiments do not yet establish scalability to full robot-policy learning or large-scale function-space generative modeling. SCMO evaluates NRSNRS candidate replacement laws per update, and the computational cost of each law evaluation depends on the objective. Thus, evaluating complete empirical laws may be expensive when simulator calls or particle interactions are costly.

First, our original submission already included a fixed-reset, open-loop Push-T simulator experiment (Appendix F.4, Table 17). Under an exactly matched budget of 4,408 simulator calls per method, evaluated over ten seeds and five deterministic scale perturbations of one fixed reset, SCMO achieves a mean objective of 0.021±0.0040.021\pm0.004, compared with 0.062±0.0140.062\pm0.014 for CEM. All eight trajectories succeed in 36/50 runs, compared with 27/50 for CEM. This demonstrates feasibility in a contact-rich simulator, but does not establish scalability to learned policies or generalization across initial states.

Second, Appendix G reports a preliminary MNIST latent-law matching experiment using a fixed pretrained VAE. With N=64N=64 particles in a 16-dimensional latent space, 512 direct SCMO updates take 65.7 seconds on an Apple M3, excluding VAE training. We also investigate amortizing the cloud transitions through a permutation-equivariant network, avoiding candidate scoring at inference time at the expense of generating teacher trajectories.

These experiments clarify the practical potential of SCMO. We do not claim scalability to realistic large-scale applications has been established. Efficient objective evaluation, scalable sampling, and amortized updates remain important directions for future work.

Q2. Do the guarantees cover the algorithm as run in the experiments?

Our theoretical results justify several components of the implemented algorithm, but they do not provide an unconditional global-convergence guarantee for every finite experimental configuration. We have clarified this distinction in the revised manuscript.

At the optimization-problem level, Theorem 4.1 proves PDE-free qualitative convergence of idealized projected-particle values to the global infimum along suitable joint sequences (K,N,ε)(K,N,\varepsilon) under Assumption 4.1 (robust recovery). It does not require differentiability or convexity and includes certain discontinuous objectives. Theorem 4.2 provides a quantitative particle–projection–regularization bound, but requires stronger classical regularity assumptions, which may not hold for some experimental objectives.

At the implementation level, Proposition E.1 establishes consistency of the multi-context feedback estimator at fixed positive ε\varepsilon as R→∞R\to\infty, even with fixed S≥1S\geq1. Its mean-square error bound requires additional moment and denominator-stability assumptions. Under stronger regularity and stability conditions, Theorem E.5 bounds the mean-square difference between the finite-candidate algorithm and the exact-feedback Euler chain over one rollout, with error O(1/R+1/(RS))O(1/R+1/(RS)). Theorem 4.3 additionally accounts for the Euler residual and signed covariance correction under its quantitative assumptions.

The experiments therefore provide empirical evidence at the reported computational budgets, while the theory establishes qualitative convergence of idealized values under robust recovery and conditional implementation-level approximation guarantees under stronger assumptions. We have made these distinctions clearer in Section 4 and the theoretical appendices.

W1. It seems that the supplied experiment is related to what reviewer 5ULo asked for. I had a different experiment in mind. However, I can now see how my wording could have been interpreted to be closer to 5ULo's request. Hence, for now I will keep my score and wait to see whether the authors' response is satisfactory to 5ULo.

W2. Thank you for adding an experiment to investigate the effect of changing K on practical performance. I assumed that one of the settings discussed in the original submission would be used as the domain. However, the new experiments are run on a new domain that was not included in the original submission. It would also have been useful to compare with the performance of Qiu (2026) directly for each K, rather than only plotting the performance of the proposed method.

W3. Thank you for adding Figure 1 as a high-level overview of the proposed method. The figure improves the clarity of the problem formulation at a glance. The addition of a high-level motivating example at the start of the introduction is also useful. The robotics example provides a clear motivation that should be widely accessible and relatable to a range of readers. Additionally, it would be good to keep the original first sentence you had in the initial submission at the very start, then proceed with the robotics example. The original first sentence was: "Many problems in machine learning and control seek a probability law over high- or infinite-dimensional solutions rather than a single optimizer, often under nonconvex or nonsmooth objectives."

Reviewer_5ULoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes SCMO, a gradient-free particle method for minimizing possibly nonconvex and nonsmooth functionals GG over probability measures on a separable Hilbert space, where the minimizer may be a non-Dirac law. The measure to be optimized is represented as the terminal law of a process Xsθ=ξ+∫0sθrdr+WsX^\theta_s = \xi + \int_0^s \theta_r \mathrm{d}r + W_s, started at a random variable ξ\xi and driven by a control drift θ\theta and a Brownian motion WW with trace-class covariance operator QQ. One then solves

min⁡θ{G(L(X1θ))+ε2E∫01∥θs∥Q2ds},\min_{\theta} \Big\lbrace G\big(\mathcal{L}(X^{\theta}_1)\big) + \frac{\varepsilon}{2} \mathbb{E} \int_0^1 \Vert \theta_s \Vert_Q^2 \mathrm{d}s \Big\rbrace ,

where, by Girsanov's theorem, the control energy amounts to an entropic regularization. After approximating the measure by NN particles and projecting onto a KK-dimensional Galerkin subspace, the Cole-Hopf transform gives an explicit formula for the optimal feedback. This formula is a Gibbs-weighted average of Gaussian perturbations and only requires evaluations of GG. In SCMO itself, the control is not learned, e.g. by a neural network, but estimated by Monte Carlo at every time step, at the current particle positions.

The theory includes a qualitative convergence result, a quantitative bound separating particle, projection and regularization errors, and a bound for the implemented scheme with differing covariances. In the numerical experiments, NN objects (particles) are optimized, and the objective is evaluated on their empirical distribution. The objects are:

  • functions on an interval, scored by phase-field energies whose gradient flows are PDEs such as the Allen-Cahn equation (no PDE is solved; the energies are only evaluated);
  • control sequences that steer a point through openings in a wall;
  • pusher motions in the Push-T robotics benchmark.

Additional terms on how the objects are distributed often force the optimum to have several modes.

Critical strengths
  • The question of optimizing over probability measures using only objective evaluations, where the optimum can be a multimodal law, is interesting. Connecting it to entropy-regularized stochastic control is a promising approach.
  • Nonconvex and nonsmooth objectives can be treated without gradients. The qualitative convergence result requires neither convexity nor differentiability, and in the experiments the method escapes local minima where Adam-based baselines get stuck.
  • The construction is technically careful, e.g. the Cole-Hopf transform is applied only after the particle and Galerkin approximations, since it does not linearize the master equation itself.
  • The mathematical analysis seems sound and is very detailed, with explicit assumptions and complete proofs in the appendix.
  • The method seems to work on the presented examples, including discontinuous objectives such as the three-phase quota task (P3), where all baselines fail.
Critical weaknesses
  • Presentation (main weakness). The paper is very hard to read, in particular for readers who are not already familiar with the topic.

    • The problem is not well motivated. It does not become clear why one wants to optimize over probability measures, or in which concrete situation this is needed.
    • Little intuition is given for the construction: why the control formulation is used, what the feedback formula (7) means, and what the algorithm actually does in each step.
    • Many parts are overly technical. Theorems 4.2 and 4.3 come with heavy notation and little explanation of what they say and why they matter.
    • Notation and terminology are introduced late or not at all (e.g. P2\mathcal{P}_2, L\mathcal{L} in (2), "interacting" GG, Lions and Fréchet derivatives), and the contribution paragraph relies on definitions given only later.
    • The text uses a lot of jargon (e.g. "context-conditioned one-particle replacement scores").
    • There is no high-level overview of the method and no illustration. The main text contains no figure, and the algorithm is only described through equations in running text.

    A short intuitive overview, an illustrative example with a figure, and a condensed algorithm box (a short version of Algorithm 1 in App. A.3) would improve readability a lot.

  • Motivation and relevance. The introduction lists application areas with citations, but does not explain why one would want to optimize over probability measures in these settings, or what is gained compared to optimizing over points. Examples only appear later, in the experiments. It is also hard to understand early on how the control problem (2) relates to problem (1) (minimization over controls vs. over measures).

  • Experiments. From the main text it is hard to understand what is actually optimized in each task, and much terminology is not explained. The tasks are largely synthetic, and in most of them the objective is designed such that the optimum is a prescribed mixture. Results are reported mainly as mean objectives and success counts; optimal values, although known for some tasks, are not reported, and on several tasks the obtained objectives seem far from optimal (e.g. P2, P3, P5). Most derivative-free baselines (CEM, CMA-ES, random search) optimize over the entire particle set at once, which puts them at a structural disadvantage; a particle-wise random search with the same budget (and, for tasks with a single optimal point, consensus-based optimization) would be more informative. Budgets are matched only among the derivative-free methods, the gradient-based baselines use far fewer objective evaluations, and wall-clock times are not reported. Finally, the main text contains no figure (e.g. a plot comparing a computed solution with the ground truth, as in Fig. 2 or Fig. 5 in the appendix).

  • Related work. The paper discusses path-integral control (Theodorou et al., 2010; Williams et al., 2017) and soft-constrained Schrödinger bridges (Garg et al., 2024), but not the closely related sampling literature. The feedback formula (7) is the Föllmer drift of a Schrödinger half-bridge, and Schrödinger-Föllmer samplers likewise estimate this drift by Monte Carlo and use Euler steps (Huang et al., 2021, "Schrödinger-Föllmer Sampler: Sampling without Ergodicity"), also for global optimization (Dai et al., 2021, "Global Optimization via Schrödinger-Föllmer Diffusion"). Diffusion-based samplers instead learn this control, e.g. the Path Integral Sampler (Zhang & Chen, 2022) and its generalizations (Richter & Berner, 2024, "Improved Sampling via Learned Diffusions"), where the log-variance loss does not require derivatives of the target. A discussion of these connections and a comparison with a learned control would strengthen the paper.

  • Minor.

    • Feynman-Kac and Cole-Hopf are classical and should be cited.
    • The correspondence between theorems in the main text and in the appendix (e.g. 4.2 and B.7) should be stated explicitly.
    • References to equations or figures in the appendix should say so (e.g. eq. (35) in Sec. 4.3).
    • ξ\xi denotes both the initial random variable (Sec. 3.1) and the candidate noise (eq. (8), Alg. 1).
    • Writing GG for GKG_K (Sec. 3.2) while using GKG_K elsewhere (e.g. (4) vs. (9)) is confusing.
    • "The proof first compares ..." before Theorem 4.2 does not say which proof is meant.
Questions for authors
  1. Section 1 lists the extensions over Qiu (2026) (Hilbert-space setting, multiple context clouds, separate covariances). Which parts of the theoretical analysis are new, and which are adapted from Qiu (2026)?
  2. Can you make explicit early on how optimizing over controls in (2) relates to optimizing over measures in (1), and which measures can be reached via the controlled process?
  3. Several experiments use very small temperatures (e.g. ε=10−10\varepsilon = 10^{-10}), where the Gibbs weights essentially select the single best candidate. How sensitive are the results to ε\varepsilon, and how does this regime relate to the theory?
  4. The feedback is estimated by self-normalized Monte Carlo, which can have high variance. Did you observe such effects or failed runs, and how do RR and SS need to scale with NN and KK?
  5. How does the method scale with the dimension KK? No experiment varies KK, and the eigenvalues λk\lambda_k used in the PDE suite seem not to be reported.
  6. For several tasks the optimum is known (e.g. the PDE tasks are minimized by mixtures of constant functions, with G=0G = 0 for P1, P2, P4, P5 and G=−4G = -4 for P3, and the target law of the MMD toy is given). Could you report the gap to these optimal values and move a plot comparing a computed solution with the ground truth into the main text?
  7. Theorem 4.3 covers a single rollout from the collapsed initial cloud, under continuity assumptions on GG, while the experiments use many outer loops and several discontinuous objectives (e.g. P3, T4). Could you comment on this gap?
  8. How does SCMO compare to Schrödinger-Föllmer or path-integral samplers, or to methods that learn the control?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 5ULo (Part 1/5)

We thank the reviewer for the careful assessment and for recognizing the interest of the problem and the soundness of our analysis. We appreciate the reviewer's candid feedback regarding the readability of the initial submission. By incorporating the reviewer's comments, questions, and suggestions, we have substantially revised the presentation, with changes highlighted in blue, and added experiments that we hope may address the reviewer's concerns.

Presentation and motivation

Our work connects stochastic analysis, optimal control, and variational analysis on Wasserstein space. Although bringing these frameworks together is technically challenging and part of our contribution, we recognize that the original presentation did not sufficiently explain the underlying intuition.

We have revised the introduction to clarify why optimizing over probability measures differs from optimizing over individual points. For example, robot demonstrations may contain several valid trajectories. A deterministic predictor trained with squared loss can average distinct successful trajectories into an infeasible one, whereas optimizing a probability law allows us to represent multiple strategies and their frequencies.

We have also added two figures to the main text. Figure 1 illustrates how the original measure-valued optimization problem is approached through stochastic control, while Figure 2 provides a six-stage overview of SCMO, including context sampling, candidate replacements, Gibbs weighting, particle updates, and repeated rollouts.

We have added explicit definitions of the Lions derivative and the spatial Fréchet derivative in Appendix A.1, with a cross-reference from Section 3.1. The Lions derivative is introduced through the Fréchet derivative of the lifted functional on square-integrable random variables.

In Section 3.3, we now define context clouds and candidates before presenting the formulas. We have also revised the opening of Section 4 to explain the scope of the convergence results before introducing their technical assumptions.

We hope these changes make the manuscript easier to follow, and we would appreciate any further suggestions for improving its presentation.

Q1. Which theoretical results are new relative to Qiu (2026)?

We acknowledge that the entropy-regularized control formulation, particle approximation, Cole–Hopf transformation, and resulting Gibbs-weighted feedback originate from Qiu (2026). We build upon these ideas rather than claiming them as new.

Our new contributions are:

1. Hilbert-space convergence analysis. We extend the framework to probability measures on a separable Hilbert space using trace-class QQ-Wiener noise and Galerkin projection. Theorem 4.2 provides a projection-first bound separating particle, Galerkin, and regularization errors, including terms that must be controlled jointly as K,N→∞K,N\to\infty.

2. PDE-free qualitative convergence. Theorem 4.1 establishes convergence of idealized projected-particle values under robust recovery, without classical HJB/Master-equation solutions, Lions differentiability, or convexity. This includes certain discontinuous objectives.

3. Nonmatched covariance analysis. Theorem 4.3 and Appendix E.1 identify the signed, curvature-dependent effect of using different proposal and execution covariances, and specify when it sharpens the approximation bound.

4. Multi-context estimation. Proposition E.1 establishes consistency and a mean-square error bound for our RR-context, SS-candidate feedback estimator. Theorem E.5 controls its approximation to the exact-feedback Euler chain over one rollout. Appendix A.4 further demonstrates why increasing the candidate count within a single frozen background does not generally recover the unconditional feedback for interacting objectives.

AuthorsOpenReview ↗

Response to Reviewer 5ULo (Part 2/5)

Q2. How does control optimization relate to measure optimization, and which measures are reachable?

The original objective is V0=inf⁡μ∈P2(H)G(μ)\mathcal V_0=\inf_{\mu\in\mathcal P_2(\mathcal H)}G(\mu). In the control formulation, each admissible drift θ\theta produces a terminal law μ1θ=L(X1θ)\mu_1^\theta=\mathcal L(X_1^\theta). We therefore optimize G(μ1θ)G(\mu_1^\theta) while penalizing the energy needed to generate that law.

For fixed positive ε\varepsilon, the control problem is a regularized surrogate of the original measure optimization problem, not an equivalent unconstrained minimization over all laws.

For a deterministic initial state, a controlled process with finite relative entropy produces a terminal law absolutely continuous with respect to the uncontrolled Gaussian reference law, under the appropriate Girsanov conditions. In the finite-dimensional projected problem, target laws with finite relative entropy relative to this reference can be realized through suitable Föllmer drifts.

Consequently, arbitrary measures, particularly singular measures such as discrete optimal laws, need not be exactly reachable with finite control energy. The role of projection, approximation, and vanishing regularization is therefore essential. Theorem 4.1 shows that the normalized idealized projected-particle values approach the unrestricted infimum under robust recovery, without requiring exact finite-energy reachability of every optimizer.

Q3. Temperature sensitivity and the hard-selection regime

Very small temperatures can yield effective optimization without extensive tuning, although they are not necessarily optimal. They can therefore serve as a practical default.

We agree that very small ε\varepsilon makes Gibbs weights approach hard selection. However, SCMO still uses relaxed, time-dependent feedback and independent execution noise rather than directly replacing each particle with its best candidate.

During our numerical experiments, we often observed that very small temperatures yielded effective optimization without extensive tuning, although they were not necessarily optimal. To further investigate this observation, we added a temperature-sensitivity experiment (Appendix C.1, Table 6), varying only ε\varepsilon while keeping all other update components fixed. The mean final objective is 0.00124±0.001480.00124\pm0.00148 at ε=10−2\varepsilon=10^{-2}, compared with 0.00903±0.009750.00903\pm0.00975 at ε=10−10\varepsilon=10^{-10} and 0.00962±0.009680.00962\pm0.00968 under hard-minimum selection. These results illustrate that very small temperatures can serve as a practical default to reduce tuning effort, while moderate temperatures may achieve better performance.

We also added particlewise random search (Appendix F.3, Table 15), which directly selects the best candidate without Gibbs weighting, feedback relaxation, or execution noise. SCMO substantially outperforms it on T2 and T5 under the same maximum budget. On T2, SCMO achieves 1.097±0.0281.097\pm0.028 versus 2.631±0.1172.631\pm0.117 in mean objective, and 0.996±0.0060.996\pm0.006 versus 0.766±0.0350.766\pm0.035 in successful mass. On T5, SCMO discovers the better upper route in 20/20 runs, whereas random search remains in the inferior basin in all runs. Random search performs better on T1 and T3. These comparisons show that SCMO is not merely hard-minimum random search, even at low temperatures, although they do not isolate each update component's contribution.

The theory distinguishes regularization from sampling error. Theorem 4.1 concerns joint limits of K,N,εK,N,\varepsilon, while Proposition E.1 establishes estimator consistency at fixed positive ε\varepsilon. Its constants need not remain uniform as ε↓0\varepsilon\downarrow0, so smaller temperatures need not improve finite-budget performance.

AuthorsOpenReview ↗

Response to Reviewer 5ULo (Part 3/5)

Q4. Feedback variance and sampling requirements for RR and SS

For interacting objectives, increasing RR controls context-sampling error that increasing SS alone cannot generally remove. Larger NN and KK can require larger sampling budgets, with no universal scaling rule.

Under suitable moment and denominator-stability assumptions, Proposition E.1 gives a mean-square error bound O(1/R+1/(RS))O(1/R+1/(RS)). The terms correspond to context and within-context sampling errors. The constant may depend on N,K,εN,K,\varepsilon.

Appendix A.4 (Table 3 and Figure 3) demonstrates the importance of RR. At the same RS=256RS=256 budget, changing (R,S)(R,S) from (1,256)(1,256) to (32,8)(32,8) reduces feedback RMSE from 1.61391.6139 to 0.15250.1525 and mean final objective from 0.182220.18222 to 0.001410.00141. Despite fewer candidates per context, larger RR performs much better, showing the benefit of distributing samples across independent contexts. The R=1R=1 configuration even performs worse than the initial cloud on average. Figure 3 also confirms the approximately R−1R^{-1} decay of feedback MSE at fixed SS.

Appendix C.2 (Figure 5) examines dependence on NN and KK. Larger N,KN,K can require substantially greater RSRS to achieve comparable accuracy; several configurations fail the 0.250.25 relative-RMSE threshold even at the largest tested budgets. This shows that accurate feedback estimation becomes more expensive as problem size increases, consistent with the dimension-dependent theoretical constants.

Q5. Scaling with the Galerkin dimension KK

Increasing KK reduces projection error but does not necessarily improve the implemented optimizer; effective refinement can require greater numerical effort.

We added two experiments on a fixed infinite-dimensional quadratic objective in Appendix C.3 (Tables 7–8, Figures 6–7). First, increasing KK from 4 to 128 at fixed temperature, sampling counts, and time resolution reduces the projection floor from 7.14×10−37.14\times10^{-3} to 3.14×10−73.14\times10^{-7}, but worsens the mean terminal objective from 0.105860.10586 to 0.130040.13004.

Second, jointly refining KK, ε\varepsilon, and Euler resolution reduces the mean terminal objective from 0.062660.06266 (K=4K=4) to 0.032510.03251 (K=17K=17), before leveling off. This requires substantially more computation and does not isolate each parameter's effect.

These results illustrate why Theorem 4.2 requires joint control of particle, projection, and regularization errors, including εKlog⁡(1/ε)\varepsilon K\log(1/\varepsilon). We claim neither dimension-independent convergence nor improvement from increasing KK alone.

Finally, Appendix F.2 now specifies the PDE covariance eigenvalues: Qek=λkekQe_k=\lambda_k e_k, where λk=(1+π2k2)−1\lambda_k=(1+\pi^2k^2)^{-1}, k=0,…,K−1k=0,\ldots,K-1.

Q6. Optimality gaps and comparison with known optimizers

We agree that reporting the objective alone makes it difficult to judge how close the computed law is to an optimizer.

We have revised Table 1 (Panel A) to report optimality gaps rather than raw objective values. The exact optimal values for P1–P5 are (0,0,−4,0,0)(0,0,-4,0,0). SCMO's corresponding mean gaps are 0.1280.128, 0.9840.984, 1.4591.459, 0.008830.00883, and 0.3420.342.

We acknowledge that the gaps remain substantial for some tasks, particularly P2, P3, and P5. Our experiments are not intended to establish that SCMO reaches the exact global optimum at the reported finite computational budgets. Rather, they also investigate whether the returned empirical law recovers the required structure.

For example, on P3, SCMO satisfies all three occupancy quotas in 20/20 runs, compared with 0/20 for the baselines. On P5, SCMO recovers the prescribed balanced outer-phase structure in 20/20 runs, although Particle Adam achieves a lower mean objective while failing the structural criterion in every run. These results show why reporting both the objective gap and structural recovery is informative.

We have also provided Figure 11 in Appendix F.2, which compares representative computed function laws with the exact optimal phases and masses. The new main-text Figures 1–2 focus on explaining the method, while Table 1 presents the quantitative results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.