acceptodds
Under review as a conference paper at ICLR 2027

A Neural Score-Based Probability Flows Approach for Second-Order Mean-Field Games

Abstract

Second-order mean-field games (MFGs) model strategic interactions among large populations with stochastic dynamics, but their coupled forward-backward PDE systems are difficult to solve in high dimensions. We develop a unified probability-flow framework for general and potential second-order MFGs using score-based neural ODEs. Building on the flow-velocity formulation based on the Nelson's approach, we rewrite the Fokker-Planck equation as a continuity equation whose velocity depends on the density score. Under suitable regularity conditions, the resulting deterministic probability flow and the controlled diffusion have identical time marginals, yielding a grid-free Lagrangian formulation that avoids spatial discretization. We study an analytical interbank benchmark, synthetic transport including a 1000-dimensional case, nonlinear 12D quadcopter control, and unpaired CelebA image translation task at latent space. The experiments recover analytical solutions in benchmark settings and demonstrate the framework's feasibility for high-dimensional transport and nonlinear control, without claiming that finite training necessarily converges to an MFG equilibrium.

Primary area:
optimization
Keywords:
Second-Order Mean-Field Games, Probability Flow, Neural Network

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_9eByOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper uses machine learning methods, namely neural ODEs and score learning, to solve mean-field games (MFGs). The mean field game can be solved iteratively: An individual agent’s movements is described via an SDE with a drift that changes in response to a cost function that depends on the entire population. Hence, given the distribution of the population an agent minimises the cost to find the drift. Given the best drift, this defines a new distribution of the population (via the marginal distribution of the SDE). The aim is to find an equilibrium This paper reformulates the SDE/Fokker-Planck point of view with probability flows/continuity equations and then learns a vector field vθv_{\theta} for an ODE to have the same marginals as the SDE. It then learns the score of the marginals of the ODE/SDE, so that the drift is given by b=vθ+νsϕb = v_{\theta} + \nu s_{\phi}. This is used to define the new distribution, and this is iterated over until convergence.

Critical strengths

I find using machine learning methods to solve MFG problems to be an interesting research direction. Moreover, upon checking the mathematical details everything seemed correct and in that sense, sound. The method applies to both potential and non-potential MFGs and is therefore quite general.

Critical weaknesses

I found this paper very hard to read. For context, I have background in SDEs, score matching, flow matching and stochastic optimal control but not in mean-field games (MFGs).

  1. Background on MFGs: a self-contained section on the MFG would be useful. Although some background is given in the introduction and key equations are defined in Section 2, they are defined without much motivation or explanation for what they are. I believe adding this would make the paper much clearer. Currently I would argue it is inaccessible to an ML audience who have an appropriate background to understand MFGs but are unfamiliar with them.


  2. Structure: I find the structure of the paper quite unclear in places, making it hard to follow. Some terms are not defined, others are used and then defined later in the text. A non-exhaustive list of examples includes: 


    • non-potential MFGs are defined before MFGs. Moreover, Eq. (2) seems to come under the heading of non-potential MFGs making it unclear whether this is a general definition or only meant for non-potential MFGs. 

    • In line 114 it’s stated “The best response formulation applies to…”, but the best response is not defined until line 135. 

    • In line 142 the paper states “…the same fixed flow problem…” however, it is not clear which fixed flow problem it is referring to; include a reference to Eq. (2) and state that fixed-flow means ρ\rho is fixed.

    • Admissible feedback in line 137 is undefined (it is mentioned in line 126 that they must be square integrable, but it’s not clear whether this is the definition).
  3. Motivation: Somewhat related to the structure, I found that a lot of the definitions and propositions lacked an explanation for why they were introduced or why they were interesting. For example, in Section 3, some explanation around vv would be useful. Currently, it is just defined as v:=b−νsρv:= b - \nu s_{\rho}, but explaining why it is being defined would be of value, e.g. by adding a sentence about the marginals of the ODE with vector field vv.


  4. Probability flows: It is not clear to me the advantage of using probability flows instead of directly using the SDE parametrisation given in Eq (2). Introducing the flow formulation means both a velocity field and the score of the corresponding ODE need to be learned. However, instead one could parametrise the drift as bθb_{\theta} directly, negating the need to learn a score. Can the authors explain what is gained in using the probability flow formulation?

Minor: I saw some spelling mistakes throughout and from page 6 onwards, the paper is in italics.

Summary: I think the paper presentation needs significant improvement in order to be published. Although sometimes paper presentation issues can be more minor, here I find it significantly impacts the readability of the paper to a point where it is hard to evaluate the method. For this reason I have given both 1 for presentation and informativeness. As for soundness, I did not find any issues in the proofs. However, given the lack of motivation for using probability flows over directly using the SDE (see point 4.) leads me to question the utility of the method, therefore I also give a 2 for soundness. If this is addressed I am willing to raise the soundness.

Questions for authors

What is the advantage of using the probability flow formulation and learning b=vθ+νsϕb = v_{\theta} +\nu s_{\phi} over directly learning a drift bθb_{\theta}?

1 comment

We thank the reviewer 9eBy for the carefully reading and presentation comments. We introduce the non-potential-MFG first because it is the general MFG formulation, and then we present potential MFGs as a special case. We plan to define the representative-agent's optimal problem w.r.t. the population, the best response, admissible feedback, and population consistency before introducing potential MFGs. We will also clarify that Eq. (2) describes the fixed-population control problem and correct the overall typography.

The motivation for defining v=b−ν∇log⁡ρv=b-\nu\nabla\log\rho is to express the controlled response law through deterministic trajectories. Here, ρ\rho denotes the response density induced by the controlled drift bb, while the population flow entering the agent's cost remains fixed during each best response. With this definition, the Fokker-Planck equation becomes ∂tρ+∇⋅(ρv)=0.\partial_t\rho+\nabla\cdot(\rho v)=0. Under our regularity assumptions and with the exact response score, the deterministic characteristics of vv have the same time marginals as the controlled SDE. This allows us to optimize the best-response objective through deterministic particle trajectories. The continuity-equation viewpoint also connects our formulation to dynamic optimal transport [K]. [D] in the response to reviewer 1 also uses SDE as dynamcs, but it introduces a saddle-point structure and need the adverserial training. We can solve the problem directly using optimization structure. We will add this motivation immediately before the definition of vv.

The reviewer is correct that the SDE drift can also be parameterized and trained directly. Direct SDE policy training [I] and FBSDE-based training [J] are valid alternatives, however, these methods are restricted at low dimensions. We will add the comparisons in the supplementary. Our choice provides a deterministic particle representation of the evolving response law: the inner policy solve requires no Brownian increments or backward network, but it requires response-score estimation and ODE backpropagation. Thus, the benefit comes with a computational trade-off.

To examine this trade-off, we compared probability flow with pathwise SDE training on an 8D frozen Gaussian LQ best response: dXt=αt dt+2ν dWt,dX_t=\alpha_t\,dt+\sqrt{2\nu}\,dW_t, with ν=0.2\nu=0.2, X0∼N(0,0.752I)X_0\sim\mathcal N(0,0.75^2I), mt=tym_t=ty, and y=(2,−2,…)y=(2,-2,\ldots). Per coordinate, the running cost was 0.5α2+0.1(X−mt)20.5\alpha^2+0.1(X-m_t)^2, and the terminal cost was 4(X1−y)24(X_1-y)^2. Both methods used 384-parameter controls, 512 training particles, 24 time steps, and 150 updates. In this single-seed experiment with an exact Gaussian score, probability flow achieved a held-out original-SDE objective of 2.8202.820, compared with 3.3233.323 for pathwise SDE training, evaluated using 2,048 common particles. CPU training times were 1.1321.132 and 0.7950.795 seconds, respectively. Probability flow therefore achieved a lower objective at greater training cost in this instance. This comparison covers one frozen best response and excludes ScoreNet training and the outer MFG iteration.

[I] Ming Min and Ruimeng Hu. Signatured Deep Fictitious Play for Mean Field Games with Common Noise. ICML, 2021.

[J] Jiequn Han, Ruimeng Hu, and Jihao Long. Learning High-Dimensional McKean-Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependence. SIAM Journal on Numerical Analysis, 2024.

[K] Jean-David Benamou and Yann Brenier. A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numerische Mathematik, 2000.

Reviewer_dJqwOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper develops a neural score-based probability-flow solver for second-order mean-field games, whose Nash equilibria are fixed points of a backward Hamilton–Jacobi–Bellman equation coupled to a forward Fokker–Planck equation. Using Nelson’s flow velocity, v=b−ν∇log⁡ρv = b - \nu \nabla \log \rho, the Fokker–Planck constraint is rewritten as a continuity equation with the same time marginals, so the response law can be represented by a grid-free Neural ODE. For the quadratic effort cost, the probability-flow action keeps the diffusion through a Fisher-information running term and an endpoint entropy, and the stochastic drift is recovered by b=v+ν∇log⁡ρb = v + \nu \nabla \log \rho. Under the stated regularity, this map preserves best responses and therefore Nash densities. A general game uses an inner alternation between a velocity network and a ScoreNet fitted to the current particles, then a relaxed outer population update. A potential game drops the outer loop and optimizes the population action directly. For control-affine dynamics, a prescribed velocity is physically realizable only when the score-corrected residual lies in the range of the control matrix. The method matches the analytical interbank feedback, transports synthetic laws through obstacles and in 1000 dimensions, and produces feasible 12-dimensional quadcopter swarms and unpaired CelebA translation in a 128-dimensional latent space. The CelebA change is modest, with FID moving from 101.11 to about 97.4, and the paper does not claim that finite training converges to an equilibrium.

Critical strengths

The central strength is the equivalence between the second-order game and its probability-flow form. Proposition 1 shows that the Nelson velocity v=b−ν∇log⁡ρv = b - \nu \nabla \log \rho converts a Fokker–Planck pair into a continuity-equation pair with the same time marginals, and the inverse map recovers an admissible drift. For quadratic effort, Proposition 2 gives an equivalent action whose diffusion appears as Fisher information and an endpoint-entropy correction. Theorem 1 and Corollary 1 then identify regular best responses and Nash densities across the two formulations, and Proposition 3 states when a learned velocity is realizable for control-affine dynamics. The argument is explicit about its regularity assumptions, and the paper does not claim that a finite training run converges to an equilibrium.

Critical weaknesses

The experiments do not compare the solver with existing methods, including the simulation-free neural conservation-law approach of Hua, Vanden-Eijnden, and Chen (2025), which enforces the Fokker–Planck equation by construction and covers stochastic optimal control and mean-field objectives. Training here integrates an augmented Neural ODE and differentiates through that solve, but the paper does not justify this choice against simulation-free alternatives: the original conservation-law parameterization of Richter-Powell, Lipman, and Chen (2022), the Girsanov-based on-policy gradients of Hua, Laurière, and Vanden-Eijnden (2024), which avoid backpropagation through the controlled dynamics, and conditional flow matching (Lipman et al., 2023), which replaces an ODE solve by a regression target. The reported interbank, transport, quadcopter, and CelebA results are therefore self-contained accuracy and feasibility checks rather than evidence that the Neural ODE formulation is preferable.

  • Mengjian Hua, Eric Vanden-Eijnden, and Ricky T. Q. Chen. "Simulation-free differential dynamics through neural conservation laws." arXiv:2506.18604, 2025.
  • Jack Richter-Powell, Yaron Lipman, and Ricky T. Q. Chen. "Neural conservation laws: A divergence-free perspective." Advances in Neural Information Processing Systems, 2022.
  • Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. "Flow matching for generative modeling." International Conference on Learning Representations, 2023.
Questions for authors

In Algorithm 1, the score update is delegated to "Algorithm 4 in D." In the manuscript I reviewed, that reference does not resolve. What does the hyperref point to, and can the authors include that procedure in the paper?

1 comment
AuthorsOpenReview ↗

Our work focus on designing an unified particle framework for high-dimensional second-order MFGs. Too much comparison on solvers distract the main focus. We agree that combinations of these solvers are interesting future directions..

We thank Reviewer dJqw for suggesting ways to train controlled dynamics without simulating trajectories or differentiating through a simulator. We agree that our experiments do not establish that our Neural ODE solver is the most efficient. Our focus is a unified particle framework for high-dimensional second-order MFGs, including potential and non-potential games. This requires a Nash fixed point: agents optimize against an anticipated population, which must then agree with their optimal response. Hua, Vanden-Eijnden, and Chen [E], building on Richter-Powell, Lipman, and Chen [F], learn a normalized density and flux that satisfy Fokker--Planck without trajectory integration during training. Their mean-field experiment concerns stochastic mean-field control, rather than an mean-field game equilibrium. They also note that their likelihood-based model is difficult to scale to high dimension. Adapting this simulation-free approach to games is an interesting future direction.

The on-policy Girsanov method of Hua, Lauri`ere, and Vanden-Eijnden [G] still simulates SDE paths, but avoids differentiating through them. Conditional flow matching by Lipman et al. [H] trains a velocity by regression for a chosen probability path. Because the equilibrium best-response path is unknown, [H] is not a direct solver for our problem without an additional path-update step. These ideas could inform the frozen best-response solve, after which population consistency must still be enforced. Our method starts from samples of the initial law and follows the response with an ODE. It avoids choosing a normalized density family, while requiring response-score fitting and ODE backpropagation. We will add these citations and explain the trade-offs.

We compared three training approaches on one frozen linear-quadratic best response in 20D and 100D. The physical control is α=2ν u\alpha=\sqrt{2\nu}\,u in dXt=αtdt+2ν dWtdX_t=\alpha_tdt+\sqrt{2\nu}\,dW_t, so the shared control cost 0.5u2=α2/(4ν)0.5u^2=\alpha^2/(4\nu) differs from the paper's 0.5α20.5\alpha^2. We used X0∼N(0,0.752I)X_0\sim\mathcal N(0,0.75^2I), a target with alternating coordinates ±2\pm2, and a frozen mean moving linearly between them. Per coordinate, the running cost is 0.5u2+0.1(X−m)20.5u^2+0.1(X-m)^2 and the terminal cost is 4(XT−y)24(X_T-y)^2. With ν=0.2\nu=0.2, T=1T=1, 24 steps, 256 training and 1024 common held-out particles, each approach had the same parameter count and received 120 Adam updates at learning rate 0.030.03. The table shows the held-out SDE objective gap to the exact discrete Riccati solution and CPU seconds for seed 2027.

MethodObjective gap (20D)Objective gap (100D)Seconds (20D)Seconds (100D)
Probability flow, exact score0.1980.2041.041.77
Pathwise SDE gradients1.1231.1310.711.51
Girsanov gradient [G], adapted6.56610.2830.711.38

Probability flow had a smaller gap in this single best response. The score was exact, so its time excludes ScoreNet training. The [G] row is our adaptation of that gradient method. No row represents the published conservation-law method [E], and none measures MFG convergence.

The reference to ``Algorithm 4 in D'' is broken because the appendix was compiled separately. The intended procedure, present in the earlier manuscript, samples (t,Xt)(t,X_t) from the \emph{current candidate response} and fits sϕs_\phi by \begin{equation*} \mathbb E\left[\nabla\cdot s_\phi(X_t,t)+\tfrac12|s_\phi(X_t,t)|^2\right], \end{equation*} using ξ⊤(∇sϕ)ξ\xi^\top(\nabla s_\phi)\xi with ξ∼N(0,I)\xi\sim\mathcal N(0,I) to estimate the divergence in high dimension. ScoreNet is then frozen during the control update. This is the current-response update already described in Remark 2. We will state these steps directly in Algorithm 1 and remove the broken cross-file reference.

Reviewer_zoaoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a neural score-based probability-flow framework for solving second-order mean-field games (MFGs). The key idea is to transform the Fokker–Planck equation into a continuity equation using the probability-flow velocity, enabling a deterministic Neural ODE representation of the population dynamics while retaining the effects of diffusion through the density score. The paper establishes equivalence between the stochastic and probability-flow formulations at the level of best responses and Nash equilibrium distributions under suitable regularity assumptions. Numerical experiments show the proposed method applied to several problems, but critically does not compare against existing methods.

Critical strengths
  • The paper provides a coherent formulation that connects stochastic MFGs with deterministic probability-flow dynamics, enables a grid-free, particle-based representation of population evolution through Neural ODEs.
  • Empirical results show the proposed method can be applied to diverse list of different tasks.
Critical weaknesses
  • The central probability-flow transformation, which converts the Fokker–Planck equation into a continuity equation using the density score, is well established. Closely related methods have already employed score-based Neural ODEs for second-order mean-field control [A] and variational normalizing flows with learned scores [B]. Moreover, particle-based flow methods have been proposed for high-dimensional general MFGs [C]. The main novelty of this paper appears to lie in applying these established techniques to the best-response formulation of general MFGs, combined with a standard fixed-point iteration. While useful, the authors do not sufficiently clarify the methodological advances over prior work or demonstrate the benefits of the proposed combination.
  • The experiments lack direct comparisons with closely related solvers, particularly score-based Neural ODE approaches [A], particle-based flow matching [C], and existing neural methods for stochastic MFGs [D]. Such comparisons are necessary to substantiate claims of computational efficiency and scalability, and give the readers a better understanding of how the proposed method is situated within the existing methods.

[A] Zhou, M., Osher, S., and Li, W. (2025). Score-based Neural Ordinary Differential Equations for Computing Mean Field Control Problems. Journal of Computational Physics

[B] Zhao, J., Zhou, M., Zuo, X., and Li, W. (2025). Variational Conditional Normalizing Flows for Computing Second-Order Mean Field Control Problems. arXiv

[C] Yu, J., Lee, J., Xie, Y., and Cheng, X. (2026). High-dimensional Mean-Field Games by Particle-based Flow Matching. ICLR 2026.

[D] Lin, A. T. Q., Fung, S. W., Li, W., Nurbekyan, L., and Osher, S. J. (2021). Alternating the Population and Control Neural Networks to Solve High-Dimensional Stochastic Mean-Field Games. PNAS

Questions for authors
  • What is the precise methodological novelty relative to existing score-based Neural ODE approaches for mean-field control?
  • How does the proposed method compare empirically to existing methods?
1 comment

We thanks for reviewer zoao's comment, and for giving these closely relative literatures. We agree that the probability-flow identity and fixed-point iteration are established. Our contribution is to apply probability flow to each frozen best response of the high-dimensional second-order MFGs, propose the unified framework to solve it, and apply this to diverse areas. Under our assumptions, the SDE and ODE yield the same agent costs and best-response laws, hence the same Nash fixed points, without a population potential. Zhou et al. [A] and Zhao et al. [B] solve second-order MFC. Yu et al. [C] solve first-order MFG, while ours extend the idea to second-order. APAC-Net [D] only solves potential MFG through a population/value saddle formulation, while we include the general MFGs (non-potential). We optimize each frozen agent problem over probability-flow particles and fit the response score.

MethodProblemMain representation
Zhou et al. [A]Second-order MFCNeural ODE with transported score
Zhao et al. [B]Second-order MFCTime-conditioned invertible density map
Yu et al. [C]First-order general MFGParticle flow matching and outer iteration
APAC-Net [D]Second-order Potential MFGPopulation and value networks in a saddle problem
OursSecond-order MFG (MFGs and potential MFGs)Probability-flow best responses with a response ScoreNet

Relative to [A], we learn the score from response particles rather than transporting it through the ODE. This avoids higher spatial derivatives in the score dynamics but adds score-estimation work. Unlike [B], we learn a velocity rather than an invertible density map. APAC-Net [D] starts from a population minimum constrained by Fokker--Planck and introduces a value-function multiplier to obtain its saddle problem (Eqs.~(2.3) to (2.5)), which require an adverserial training and might introduce unstable as the nature. For each frozen population in our method, ODE trajectories satisfy the continuity equation, and the transformed agent cost is minimized without that value-function maximizer. Score fitting and the outer best-response iteration remain. This is a difference in optimization structure, not a guarantee of faster convergence. The completed external comparisons are below.

SettingMetric (lower is better)ProposedCited method
Yu [C], 2D, ν=0\nu=0Agent objective15.4718.26
Error to target line x2=−1x_2=-10.5720.752
Stationarity residual2.340.54
Training time (s)1943
APAC [D], 12D, ν=0.01\nu=0.015E∥e6∥5\mathbb{E}\lVert e_6\rVert20.7542135.81
Training time (s)127221132

The 2D experiment comparison with [C] is a one-seed reimplementation. We obtain lower agent cost but worse stationarity and longer training time. The 12D quadcopter experiment comparing with [D] is setted as ν=0.01\nu=0.01 and γ=20\gamma=20: its terminal norm is lower for our solver, but APAC was diverging at its 20k-update checkpoint, the training losses and budgets differ, and our run is slower. We will add complete comparison to the experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.