acceptodds
Under review as a conference paper at ICLR 2027

Neural Solvers for Mean-Field Games with Equilibrium Certificates

Abstract

Large-scale multi-agent systems can be modeled by mean-field games (MFGs), in which individual decision-making and population evolution are coupled through an equilibrium between individual optimality, where the representative agent's policy is a best response to the population distribution, and population consistency, where the distribution is generated by the resulting policy. Solving such an equilibrium typically requires repeated forward–backward iterations to a fixed point for each new environment, objective, or population state. Neural solvers can reduce per-instance computation by learn a mapping from instances to solutions, but their predicted policies and populations need not form an equilibrium. We introduce Certified Equilibrium Neural Operator (CENO), a value-only neural MFG solver that couples solution construction with equilibrium certification. CENO predicts only candidate value functions. A Gibbs policy is derived from the predicted values and the population is then propagated through the policy-induced dynamics, ensuring population consistency by construction. To assess individual optimality, we derive an equilibrium certificate from the Bellman defect of the predicted values. The certificate upper-bounds the exploitability of the resulting policy-population pair, without computing a reference equilibrium or an additional best-response solve. The certificate also provides a label-free training objective, enabling the neural operator to learn across MFG instances without equilibrium solution labels. Under monotone population interactions, the certificate further bounds the error of the induced population trajectory relative to equilibrium, extending the guarantee from individual optimality to population-level accuracy. Experiments on a game-theoretic benchmark and three multi-agent population tasks evaluate solution quality and illustrate population-level coordination behaviors.

Primary area:
optimization
Keywords:
Mean Field, multi-agent systems, optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_K3RFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a neural solver for solving MFGs with equilibrium certification. Instead of predicting the policy and population distribution independently, the authors predict only the value function and derive the Gibbs policy and corresponding population from it, ensuring forward population consistency. By minimizing the certified Bellman defect, the trained neural-network predictor can guarantee an upper bound on expected exploitability without equilibrium labels or best-response targets. Experiments on different tasks show that CENO achieves better performance in terms of equilibrium quality, online solution time, and certification.

Critical strengths
  • Predicting only the continuation values and constructing the Gibbs policy and population from them is novel and reasonable, naturally guaranteeing forward consistency while also providing a guarantee of backward optimality for the derived policy.

  • The authors use the Bellman defect-based certificate to measure the prediction quality, which can be computed from the solver output and the known MFG model alone without requiring either equilibrium labels or an additional best-response computation, making the training and evaluation across different MFG instances easier.

  • The proposed method solves a single forward pass without per-instance fixed-point iteration, and experiments show competitive or better performance in terms of equilibrium quality, online solution time, and certification.

Critical weaknesses
  • This paper is restricted to finite-state, entropy-regularized MFGs with population-independent transitions. Though this is a common type of MFG, it would be good to provide more discussion on how the current method can be adapted to a broader class of MFGs. Besides, the population-level guarantees additionally require monotone interactions, and the accumulated-error bound requires strong monotonicity, which may not hold in some cases.

  • Experimental environments seem quite simple with a deterministic transition kernel; thus, the real-world validity remains unclear.

Questions for authors

See weaknesses.

1 comment

W1. How the current method can be adapted to a broader class of MFGs

Than you for this suggestion. These assumptions (finite-state, population-independent transition, entropy regularization, and monotonicity) serve different roles in CENO. We discuss their purpose, the challenges of relaxing them, and possible extensions below.

Entropy regularization provides a unique and smooth Gibbs policy from predicted values, enabling end-to-end training of a shared neural solver across MFG instances. For unregularized MFGs, we can instead construct through Bellman or Hamiltonian minimization and evaluate optimality using the corresponding residuals. The main challenge is that the optimal control may be non-unique or non-smooth, requiring the solution computed instance by instance rather than one network.

Population-independent transitions allow the population to evolve through a policy-induced Markov transition and simplify the comparison between predicted and equilibrium populations for population-level guarantees. For population-dependent transitions, the forward rollout can still use the transition kernel evaluated at the current population. However, changes in population affect the transitions, introducing additional errors that must be controlled in the certificate and population-level analysis, for example, under Lipschitz dependence of the kernel on the population, which we leave for future work.

Monotonicity is required only for the population-level guarantees, not for the individual exploitability certificate. It makes the equilibrium population unique, so the population-level bounds are well defined. Monotonicity holds for tasks in which the interaction cost responds to changes in population distribution, such as crowd evacuation, traffic routing, and multi-robot navigation. When monotonicity fails, as with herding or coordination, multiple equilibria exist, the certificate still bounds exploitability, but there is no unique equilibrium population to compare against. We believe the population-level guarantee is still meaningful, maybe not the distance to one equilibrium, but the averaged distance to the equilibrium set. This is an important direction for future work.

Finite states allow exact population propagation and direct evaluation of the maximum Bellman defect. For continuous-state MFGs, a natural extension is spatial discretization or continuous population dynamics. The main challenge comes from numerical approximation errors which cause forward consistency and residual evaluation are no longer exact.

W2. Scenario with stochastic transition kernel and additional traffic experiment.

This you for this comment. Our formulation allows stochastic transition kernel. To evaluate this and broaden the application setting, we add a stochastic Beach Bar experiment and a vehicle velocity-control experiment with physical motion constraints.

Stochastic transition experiment. In the Beach Bar experiment, with probability (p=0.2p=0.2), the intended transition is replaced by a move to a uniformly sampled valid neighboring state. The deterministic and stochastic experiments use same initial populations and cost parameters. We train and evaluate all methods, including DeepONet and FNO, under the corresponding transition kernel. At (p=0.2p=0.2), CENO achieves a mean exploitability of 3.919e-5 compared to 1.588e-4 in deterministic transition kernel. Their time cost is 0.69 ms and 0.67 ms respectively. It achieves a better exploitability than Master Policy (8.932e-4 at 2.5 ms), DeepONet (3.365e-2 at 0.38 ms), FNO (3.480e-4 at 0.71 ms). Fictitious play and OMD can reach a better exploitability with a time cost of more than 120 ms, but worse than CENO under the same time cost.

Vehicle velocity-control experiment. The conservativeness of our method is the cost of the generality for all instances. We conduct a vehicle experiment using fixed initial conditions and speed limited motion. These constraints allow us to bound cumulative residuals along admissible trajectories, avoiding the sum of per-time maxima that no single trajectory can realize. Evaluating both bounds for the same candidate policy, the trajectory-constrained certificate is 88%~92% lower than the original per-time bound. This experiment illustrates how physical constraints can make the certificate more informative in a traffic simulation. Therefore, our method has wide potential for specific validation.

We will include both experiments and their setup details in revised version.

Reviewer_SmKnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This manuscript introduces Certified Equilibrium Neural Operator (CENO), a neural solver for finite-horizon entropy-regularized mean-field games (MFGs). CENO predicts only the candidate value function, which helps derive a Gibbs policy to propagate the population through the policy-induced dynamics, ensuring population consistency by construction. The manuscript constructs an equilibrium certificate based on the Bellman defect, which also serves as a label-free training objective, and derives bounds on the deviation of the predicted population trajectory from an equilibrium trajectory under a monotone population.

The evaluation of the Beach Bar benchmark focuses on exploitability, certification, and computational efficiency against the classical iterative methods and the Master Policy. CENO is additionally evaluated on three population-level coordination tasks—spatial allocation, temporal adaptation, and counterflow coordination—against multi-agent learning baselines. The reported results demonstrate low exploitability and fast online computation on the Beach Bar benchmark, alongside improved exit rates and reduced congestion-related costs across the three population-level tasks.

Critical strengths
  1. This manuscript constructs Cθ=∑h=0H−1max⁡x∈Rhrh(x)−∑h=0H−1∑x∈Rhm^h(x)rh(x)C_\theta = \sum_{h=0}^{H-1} \max_{x \in \mathcal{R}_h} r_h (x) - \sum_{h=0}^{H-1} \sum_{x \in \mathcal{R}_h} \hat{m}_h(x) r_h(x) and establishes 0≤Explτ(π^,m^)≤Cθ0 \le \text{Expl}_\tau(\hat{\pi}, \hat{m}) \le C_\theta. This bound can be directly evaluated without an additional best-response solve.
  2. The training objective L(θ)=Eξ∼D[Cθ(ξ)]\mathcal{L}(\theta) = \mathbb{E}_{\xi \sim \mathcal{D}}[C_\theta(\xi)] requires no equilibrium labels or best-response targets.
  3. On the tested MFG benchmark, CENO can produce low-exploitability solutions efficiently, and its certificate provides useful information about equilibrium quality in that setting.
  4. On the population-level benchmarks—spatial allocation, temporal adaptation, and counterflow coordination—CENO achieves the highest final exit rate and the lowest barrier cost among the evaluated methods.
Critical weaknesses

1. Finite-horizon guarantees against continuing-horizon experiments.

The theoretical formulation in this manuscript, especially in sections 3 to 4, considers finite-horizon, HH, Mean Field Games with terminal costs gg. Looking at the implementation of the algorithm in Appendix B.2, CENO repeatedly solves H=24H=24 MGFs across the open-system simulator, which is used for all three population-level tasks. The method computes a policy over the full planning horizon, executes only its first-step policy, and then replans after observing the updated population. The manuscript does not address whether the proposed equilibrium certificate under repeated planning provides any guarantees for the overall behavior.


2. Incomplete reporting of agent populations.

Table 1 on page 9 reports the cumulative number of agents released by the evaluated methods. The manuscript, however, does not specify the number of agents simultaneously active in each task. The active population changes over time as the simulation introduces new batches depending on whether the source area has cleared. Reporting initial population sizes, batch sizes, and active agent counts or distributions throughout the simulations should strengthen the reproducibility and interpretation of the experimental results.


3. Limited experimental scope and baseline comparisons.

The experimental evaluation consists of a single finite-horizon Beach Bar benchmark and three synthetic grid-based population coordination tasks. This evaluation is narrow, and the chosen tasks share similar underlying dynamics and congestion-based interaction structures. On the Beach Bar benchmark, CENO is primarily evaluated against classical iterative methods and the Master Policy, but the evaluation omits direct comparison against existing neural MFG solution operators discussed in the related work. MF-PSRO is included in the population-level experiments; however, other PSRO variants are not investigated. Finally, the population-level tasks are evaluated only on exit rates and barrier costs, without corresponding measurements of equilibrium exploitability or the certificate tightness. These limitations make it difficult to assess the proposed method's comparative advantage and scalability across diverse MFG settings.

Questions for authors

I seek clarification for the practical significance of the theoretical results, the scale and difficulty of the experiments, and CENO's empirical advantages through my questions below:

  1. Scope of the equilibrium certificate: Appendix B.2 describes a receding-horizon implementation in which CENO repeatedly computes an H=24H=24 policy, executes only its first-step policy, and replans using the updated population. Do the authors intend the equilibrium certificate to characterize only each finite-horizon planning problem, or does it provide any guarantee for the resulting closed-loop behavior across the full simulation?

  2. Agent population sizes: Table 1 reports the cumulative number of agents released, but the number of simultaneously active agents is unclear. Could the authors specify the initial population sizes, batch release sizes, and typical or maximum numbers of concurrently active agents for each population-level task?

  3. Comparison with existing neural MFG solvers: The manuscript discusses existing neural MFG solution operators in the related work, but these methods are not included in the equilibrium-quality comparisons. Would it be possible for the authors to provide comparative results for the Beach Bar benchmark using the same exploitability and computational cost metrics?

6 comments

Q1. Scope of equilibrium certificate. The manuscript does not address whether the proposed equilibrium certificate under repeated planning provides any guarantees for the overall behavior.

Thank you for this question. Our implementation is applying the certificate to characterize each finite-horizon planning problem exactly and using a receding-horizon to preserve this finite-horizon formulation at every planning. Within each planning problem, the current active population defines the initial distribution. Agents are injected before the planning, no new agents are injected during the predicted horizon. The population evolves, and mass reaching an exit is retained. Thus, the total mass remains constant through the planned rollout. The certificate bounds the exploitability of the policy-population constructed for this finite-horizon problem. Between planning problem, only the first-step policy is applied to the simulator. Absorbing and releasing are processed between the planning. These changes only enter through initial conditions to form a new planning problem, rather than through external injections within a planned rollout. We do not claim an equilibrium guarantee for the entire closed-loop simulation. We will make this scope in the revised manuscript.

Q2. Agent population size.

Thank you for pointing out this omission. We randomize the number of agents in each source cell to generate different population instances, varying both the total number of agents and their spatial distribution. This design evaluates CENO's generability to solve different instances using the same trained network, without retraining for each population.

Taking spatial allocation and temporal adaptation as examples, each source region contains 64 cells. Each cell is initialized independently with a uniformly sampled count from ({5,6,7}), giving an initial population of 320-448 agents and an expected size of 384. At subsequent releases, the same distribution is used to sample a target count for each source cell, and only the additional agents needed to reach the target are admitted. A subsequent batch is released only when at least 30 simulation steps have conducted and the mean source-cell occupancy is at most 30% of capacity. At most six batches are released. This regulates inflow while allowing release timing to reflect how effectively each method clears the source region. Counterflow uses the same sampling and occupancy rules, with two 70-cell source regions, an initial population of 350–490 agents per group, a minimum release interval of 35 steps, and at most five batches per group. The number of active agents varies across time and methods as new batches enter and existing agents exit. Figures 4(c) and 4(d) illustrate this variation between a new released batch and agents almost exited. CENO replans from the updated population at each simulation step, therefore guarantee a MFG in each step. We will additionally report the time-averaged and peak active-agent counts for each method and task in revised version.

Q3. Comparison with existing neural MFG solvers discussed in the related work.

Thank you for this suggestion. We add DeepONet and FNO baselines in a supplementary Beach Bar experiments, alongside CENO, Master Policy, and the classical solvers. They are adopted to the same setting in original manuscript. The result shows that CENO achieves a mean exploitability of 1.588e-4 at a time cost of 0.69 ms, DeepONet reaches of 6.994e-2 at a time cost of 0.39 ms, while FNO reaches of 1.351e-3 at a time cost of 0.73 ms. These results support that our method achieves a low exploitability with fast solving. We also repeat the complete comparison with stochastic transition. The full results will be updated in revised version.

Dear reviewer,

We have updated the revised manuscript. This manuscript includes the clarification on the experiments and new experiments according to your suggestions. The extended Beach Bar experiment includes two more neural solver baselines.

Best, Authors.

Dear Authors,

I'm glad the new manuscript addresses my concerns, leaving no further concerns. I've updated my score to a weak accept. I'll refrain from increasing it to a clear accept because I think the new experimental evaluation in the revised manuscript does not yet establish that the certificate provides sufficiently actionable information in the actual settings where CENO is deployed. As I pointed out in my review, the current experimental setups share a similar dynamic. Results from MAgent2 Battle or a StarCraft-style SMAX scenario would help establish whether the proposed algorithm, CENO, remains computationally effective.

Best,

Reviewer SmKn

Reviewer_dcKMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper contributes to the study of mean field games (MFGs) and more specifically to methods based on neural networks for such games. It introduces the Certified Equilibrium Neural Operator (CENO), a value-only neural solver for finite-state, entropy-regularized MFGs. It then analyzes the method with theoretical bounds, and provide a numerical example.

Critical strengths

(1) Idea of replacing exploitability by a certificate

(2) Theoretical analysis

(3) Some numerical experiments in relatively realistic environments

Critical weaknesses

(1) It is not clear how non-tight the certificate is.

(2) The scope of the theoretical analysis also seems relatively limited.

Questions for authors

(1) How tight is the certificate? In other words, how big can the gap between exploitability an CθC_\theta be?

(2) Is it possible that CθC_\theta is non-zero even at a Nash equilibrium?

(3) Could you please provide a few examples satisfying all the assumptions you require?

2 comments

Q1. How big can the gap between exploitability an CθC_\theta be?

Thank you for this question. On Beach Bar, the mean absolute gap Cθ−ExplC_\theta-\text{Expl} is 0.0968 for certificate training, compared with 0.1859 for exact exploitability training, a reduction of 48%.

The gap Cθ−ExplC_\theta-\text{Expl} mainly comes from the per-step maximization in Eq.20. The exact gap requires solving for the best response, whose trajectory is unknown. To certify any instance directly from the solver output, CθC_\theta instead takes the largest Bellman defect over the reachable states independently at each time step, as if a deviating agent could reach the worst state at every step. A real deviator follows its own trajectory and cannot attain all these worst state, so this relaxation leads to the gap. On Beach Bar, this relaxation accounts for 93% of the gap. Reducing this gap in specific scenario is simple: replacing the per-step maximum by maximum along agent's own trajectory. In an additional traffic experiment with fixed initial conditions and speed limits, the gap is reduced by 88%-92%. We will clarify these results in the revised manuscript.

Q2. Is it possible that CθC_\theta is non-zero even at a Nash equilibrium?

Yes, As noted after Theorem 1, Cθ=0C_\theta=0 is sufficient for equilibrium, but not necessary. This is because the Gibbs policy depends on the relative action values, while CθC_\theta also measures the consistency of the predicted values with the Bellman equation. Start from exact equilibrium values (Expl=CθC_\theta=0), modify the predicted values so that, at every state, all action values in Eq.13 shift by the same amount. Since the Gibbs policy in Eq.14 depends only on the differences between action values at a state, so this modification makes the policy unchanged. The following population, the forward consistent trajectory and the exploitability stay unchanged. Therefore, this is still an equilibrium. However, the Bellman values in Eq.5 shift due to this modification. So the Bellman defect changes. Therefore, if the shift differs across states, CθC_\theta becomes non-zero, although it's still an equilibrium.

This does not affect the guarantee Expl<Cθ\text{Expl}<C_\theta. It means that an equilibrium policy does not necessarily correspond to a zero certificate. Minimizing CθC_\theta to the smallest non-zero value also equals to an equilibrium. We will clarify this distinction after Theorem 1.

Q3. Could you please provide a few examples satisfying all the assumptions you require?

All four experimental settings in our paper (the Beach Bar benchmark and three multi-agent tasks) satisfy the stated assumptions. They are formulated as finite-state, entropy-regularized MFGs with population-independent transitions. As for monotonicity, it is required for the population-level guarantees. Under monotonicity, the interaction cost responds to changes in population distribution, so that the equilibrium population is unique and the equilibrium gap can be defined. Beach Bar uses a positive linear congestion cost; spatial allocation task and temporal adaptation task include increasing congestion penalty; and counterflow task considers two populations whose self-interaction is stronger than their cross-interaction. The proof of monotonicity in these tasks is provided in Appendix A.6 and B.2.

The same monotone interaction mechanisms arise in many practical tasks, such as crowd evacuation (Lachapelle & Wolfram, 2011; Djehiche et al., 2017), traffic routing (Huang et al., 2021; Cabannes et al., 2022), and multi-robot navigation (Shiri et al., 2020; Kang et al., 2020), where agents balance their individual objectives against increasing costs in crowded areas. When population dependence enters through these costs, such tasks fit our assumptions. We will clarify these examples in the revision.

Dear reviewer,

We have updated the revised manuscript. This manuscript includes the clarification, supplemental explanation and new experiments according to your suggestions. The extended Beach Bar experiment includes more baselines and ablations. And we further design a traffic velocity-control experiment to investigate the tightness of CθC_\theta in specific scenario. Hope these results can address your concerns.

Best, Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.