- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes SCMO, a stochastic-control-based optimization framework for probability measures on infinite-dimensional Hilbert spaces. Building on an entropy-regularized optimal control formulation, the method employs finite-particle and Galerkin approximations to derive a gradient-free optimization algorithm. The paper also provides extensive theoretical analyses of convergence and approximation errors, together with empirical evaluations across several optimization settings.
Critical strengths
- The proposed framework is well motivated and mathematically principled. Its algorithmic design follows naturally from the underlying stochastic optimal control formulation, making the overall approach reasonable and convincing.
- The paper provides substantial theoretical analysis, covering convergence of the idealized optimization problem, particle and projection approximation errors, and finite-candidate implementation errors. This theoretical development is a notable strength of the work.
- The experimental results demonstrate the effectiveness of the proposed method across different optimization settings, providing further support for its practical applicability.
Critical weaknesses
Overall, I consider this a technically solid and well-developed paper, with meaningful theoretical contributions. My main concerns are related to the positioning of its contributions and the completeness of the experimental comparisons.
-
The proposed framework is closely related to the work of Qiu (2026), as acknowledged in the introduction. In particular, the fundamental stochastic control formulation and the derivation of gradient-free optimization through optimal feedback appear to build directly upon this prior work. Although the present paper provides meaningful extensions, particularly in its infinite-dimensional formulation and theoretical analysis, I believe the authors could more explicitly clarify the essential technical differences and contributions beyond Qiu (2026).
-
The current baselines appear somewhat limited for assessing the improvements over the most closely related prior work. In particular, a direct comparison with Qiu (2026) would help demonstrate the practical benefits of the proposed extensions. I encourage the authors to consider including such experiments, ideally under comparable evaluation budgets. A discussion or comparison with relevant mean-field optimization methods would also help better position the proposed approach.
-
Since the theoretical development relies on mean-field control and value-function approximation, it would be helpful to further discuss how the proposed theoretical framework differs from existing mean-field optimization approaches. In particular, clarifying the distinctive advantages of the stochastic optimal control perspective, both theoretically and algorithmically, would strengthen the contribution.
I would like to note that, given the complexity and length of the theoretical development, I have not independently verified all the proofs in detail. My positive assessment of the theoretical component is therefore primarily based on the overall proof structure, mathematical formulation, and consistency of the proposed analytical approach, rather than a complete verification of every technical argument. Despite these concerns, my overall assessment is positive. I believe the paper makes a meaningful contribution, and addressing the points above would further strengthen its presentation and empirical justification. With these improvements, I would be inclined to support a clear acceptance.
Questions for authors
- Could the authors clarify the key technical and theoretical advances over Qiu (2026), particularly which results require substantially new analysis beyond the existing stochastic control framework?
- Could the authors provide direct experimental comparisons with Qiu (2026), preferably under comparable computational budgets, to better demonstrate the benefits of the proposed extensions?
- Could the authors further discuss the relationship between the proposed theoretical framework and existing mean-field optimization methods, including the specific advantages of using the stochastic optimal control formulation?
5 comments
Response to Reviewer EW3V (Part 2/3)
Q2. Direct experimental comparisons with Qiu (2026)
In response to the reviewer's suggestion, we have added direct experimental comparisons with Qiu (2026), together with a new particlewise random-search baseline, across five trajectory-law optimization tasks (T1–T5). These tasks examine narrow feasibility, multimodal law recovery, discontinuous interacting objectives, and escape from suboptimal local basins. The results are reported in Table 1 (Panel B) and Appendix F.3 (Table 15) of the revised manuscript.
1. Experimental setup and computational fairness.
We implement the localized frozen-background method of Qiu (2026, Section 3.3), using a single context cloud () and matched proposal and execution covariances. SCMO instead uses multiple contexts () and allows nonmatched covariances.
All methods use the same initialization protocol and 20 paired evaluation seeds, with particles representing 42-step trajectories. We impose the same maximum computational budget of 58,256,768 trajectory simulations per method. SCMO and Qiu also use the same number of candidate replacements per particle per update (), with for SCMO and for Qiu. Hyperparameters are tuned separately on development runs and frozen before evaluation. Actual evaluation counts may differ because of stopping criteria.
2. A new particlewise random-search baseline.
To investigate whether SCMO's performance can be explained by straightforward candidate selection, we introduce a particlewise random-search baseline. This method uses contexts and candidates per context, selects the candidate–context pair with the lowest objective for each particle, and simultaneously replaces all particles.
Unlike SCMO, this baseline uses hard candidate selection without Gibbs weighting, feedback relaxation, or execution noise. It therefore provides an informative comparison with a simple derivative-free particlewise search strategy. We also retain the whole-cloud random-search baseline from the original submission, alongside CEM and Mixture MPPI.
3. Main experimental findings.
SCMO achieves a lower mean objective than Qiu (2026) on four of the five tasks, with particularly substantial improvements on the more challenging problems.
T2 (narrow passage): SCMO achieves , compared with for Qiu and for particlewise random search. SCMO also achieves successful trajectory mass , compared with for Qiu and for particlewise random search.
T3 (multimodal recovery): SCMO achieves , compared with for Qiu. SCMO and particlewise random search satisfy the strict full-success and two-route criterion in 20/20 runs, whereas Qiu does so in 0/20 runs. Particlewise random search achieves a slightly lower mean objective ().
T4 (discontinuous interacting quotas): SCMO achieves , compared with for Qiu. All three particlewise methods satisfy the structural criterion in 20/20 runs, with SCMO and particlewise random search achieving nearly identical mean objectives.
T5 (escape from a suboptimal basin): SCMO achieves , compared with for Qiu and for particlewise random search. SCMO discovers the better upper route in all 20 runs, whereas Qiu and particlewise random search remain in the inferior lower-route basin in every run.
On the simpler T1 task, both Qiu and particlewise random search outperform SCMO in mean objective, while all three satisfy the structural success criterion in 20/20 runs. We therefore do not claim uniform superiority.
4. Significance of the additional comparisons.
These results provide two complementary findings.
First, SCMO substantially improves upon the localized method of Qiu (2026) on several challenging trajectory-law optimization problems under the same maximum computational budget.
Second, the particlewise random-search comparison demonstrates that hard candidate selection alone does not explain SCMO's performance across all tasks. While random search performs particularly well on T1 and T3 and matches SCMO closely on T4, SCMO achieves substantially higher successful mass on T2 and consistently escapes the inferior local basin on T5.
These comparisons evaluate the combined contributions of multi-context estimation, Gibbs-weighted feedback, and nonmatched execution covariance; they do not isolate each component's causal effect. The roles of multi-context sampling and covariance nonmatching are further investigated in Appendix A.4 (Table 3 and Figure 3) and Appendix A.2 (Table 2), respectively.
We believe these additional experiments directly address the reviewer's concern and provide stronger empirical evidence for the practical benefits of our proposed extensions.
Response to Reviewer EW3V (Part 3/3)
Q3. Relationship to existing mean-field optimization methods and advantages of stochastic optimal control
We have expanded Appendix B.1 and added a five-method comparison in Appendices B.2–B.4 (Table 5 and Figure 4). Our main distinction is that stochastic control provides both a principled derivation of the algorithm and mathematical tools for analyzing its approximations.
1. A control-theoretic formulation.
We reformulate optimization over as a finite-horizon stochastic control problem, with evaluated at the terminal law and a quadratic drift-energy penalty. This allows us to use established techniques from dynamic programming, HJB equations, Feynman–Kac representations, and FBSDE theory. After particle and Galerkin approximation, the Cole–Hopf transform yields a computable Gibbs-weighted feedback (Equation (7)), leading to the SCMO update (Equation (9)).
2. A theoretically derived, gradient-free algorithm.
Unlike the Wasserstein gradient flow, mean-field Langevin, and EFP implementations considered in Appendix B, SCMO requires only evaluations of , not its first variation or spatial derivatives. It can therefore handle nonsmooth, discontinuous, or simulator-based objectives.
The control representation also determines the structure of the update: the remaining-time horizon determines the proposal covariance, Gibbs weights arise from the value-function representation, and weighted terminal displacements determine the feedback drift. For interacting objectives, our multi-context estimator consistently approximates the unconditional full-cloud feedback rather than feedback conditional on one frozen background.
3. A unified framework for convergence and approximation analysis.
The control formulation allows us to distinguish different levels of approximation.
Theorem 4.1 establishes PDE-free qualitative convergence of idealized projected-particle values under robust recovery, including certain nonsmooth and discontinuous objectives.
Theorem 4.2 separates particle, Galerkin-projection, and entropy-regularization errors under stronger classical regularity assumptions.
Theorem 4.3, Proposition E.1, and Theorem E.5 address the signed covariance correction, finite-candidate feedback consistency, and one-rollout approximation to the exact-feedback Euler chain under the stated assumptions. In particular, the HJB formulation explains how nonmatched execution covariance can sharpen the approximation bound through the curvature of the continuation value.
These results do not imply unconditional global convergence of the finite implementation.
4. Relationship to existing mean-field methods.
Wasserstein gradient flow transports particles according to the spatial gradient of the objective's first variation, whereas SCMO constructs feedback from objective evaluations.
Mean-field Langevin and EFP, in our comparison, approximate and require derivatives. SCMO instead uses a finite-horizon control problem with a quadratic drift-energy penalty, which has a path-space relative-entropy interpretation under suitable conditions. The original objective remains the optimization target, while the auxiliary regularization bias is controlled theoretically.
Measure-valued CBO is also derivative-free but moves competing probability laws toward a Gibbs-weighted Wasserstein barycenter, rather than estimating a time-dependent optimal control feedback.
SCMO also avoids the inner Langevin sampling used in our EFP implementation. Its Gaussian proposals and objective evaluations can be batched, although evaluating multiple candidate empirical laws incurs computational costs.
5. Additional experimental comparison.
We compare particle Wasserstein gradient flow, mean-field Langevin, EFP, measure-valued CBO, and SCMO on a two-well objective whose unique minimizer is the non-Dirac law . All methods start near one well and must discover the other while recovering the correct mass distribution.
SCMO attains the lowest mean final objective, , compared with for EFP, for Langevin, for gradient flow, and for CBO.
SCMO and EFP satisfy the recovery criterion in 20/20 runs, whereas the other methods do so in 0/20. EFP achieves a slightly smaller mean Wasserstein distance, so we do not claim uniform superiority.
In summary, the distinctive advantage of SCMO is the combination of a control-theoretic derivation, gradient-free implementation, and rigorous approximation analysis. The additional comparison illustrates its effectiveness in recovering a non-Dirac optimal law.
Response to Reviewer EW3V (Part 1/3)
We thank the reviewer for recognizing the theoretical contributions and empirical effectiveness of our work. We have revised the manuscript, with changes highlighted in blue, to clarify the advances beyond Qiu (2026), add direct comparisons, and discuss related mean-field methods. We hope these revisions address the remaining concerns and support a clear acceptance.
Q1. Key technical and theoretical advances over Qiu (2026)
We build on the entropy-regularized control formulation and Cole–Hopf representation of Qiu (2026), but make four further advances:
1. Infinite-dimensional formulation and projection-first analysis. We study P2(H) on a separable Hilbert space using trace-class Q-Wiener noise and Galerkin projection onto a K-dimensional subspace. Theorem 4.2 gives a projection-first error bound separating particle, projection, and entropy-regularization errors. It controls the projection tail, projected Cameron–Martin cost, and dimension-dependent term εKlog(1/ε), with conditions for joint convergence as K,N→∞ and ε↓0. These estimates cannot be obtained by directly letting the dimension grow in Qiu's fixed-dimensional bounds.
2. Nonmatched proposal and execution covariances. Motivated by numerical observations, we distinguish the proposal covariance Qprop used to estimate feedback from the execution covariance Qdyn used in particle dynamics. This departure from the matched control problem requires new analysis. Theorem 4.3 and Appendix E.1 derive a signed covariance correction whose leading term involves Tr[(Qdyn−Qprop)D2v]. Under favorable curvature conditions, this sharpens the upper error bound, without implying that nonmatching always helps. See also Appendix A.2 (Table 2).
3. PDE-free qualitative convergence for less regular objectives. Theorem 4.1, under Assumption 4.1 (robust recovery), proves convergence of normalized idealized projected-particle values to the global infimum along suitable joint sequences K,N→∞ and ε↓0. It requires neither classical HJB/Master-equation solutions, Lions differentiability, nor convexity, and includes certain nonsmooth and discontinuous objectives excluded by the quantitative theory.
4. Multi-context feedback estimation: resolving a frozen-background limitation. Qiu (2026) gives both a full-cloud Monte Carlo scheme (Eq. (3.11)) and a cheaper single frozen-background approximation (Section 3.3, Eqs. (3.14)–(3.17)). For interacting objectives, the latter estimates feedback conditional on a sampled background, not the unconditional full-cloud feedback. Increasing the candidate count S alone does not remove this discrepancy: with a fixed background, the S→∞ limit remains conditional.
Our estimator uses R independent context clouds and S candidates per context, with joint Gibbs normalization across all RS scores. Independent contexts, rather than more candidates in one context, are the essential improvement. Proposition E.1 establishes almost-sure consistency for the full-cloud feedback as R→∞, even for fixed S≥1, and a mean-square error bound O(1/R+1/(RS)) under suitable moment and denominator-stability conditions. Theorem E.5 propagates the estimation error through one rollout, controlling the mean-square difference between the finite-candidate algorithm and the exact-feedback Euler chain.
Appendix A.4 (Table 3 and Figure 3) gives an interacting quadratic example where the ratio of the expected one-context feedback to exact feedback approaches N as ε↓0. Under the same RS=256 replacement-score budget, increasing contexts from (R,S)=(1,256) to (32,8) lowers the mean final objective from 0.18222 to 0.00141. This establishes a concrete benefit of multi-context estimation in this example; our comparison concerns Qiu's localized frozen-background approximation, not its full-cloud scheme.
In summary, beyond the original control formulation, we provide Hilbert-space convergence bounds, nonmatched-covariance analysis, PDE-free qualitative convergence, and a consistent multi-context estimator with implementation guarantees.