- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a neural solver for solving MFGs with equilibrium certification. Instead of predicting the policy and population distribution independently, the authors predict only the value function and derive the Gibbs policy and corresponding population from it, ensuring forward population consistency. By minimizing the certified Bellman defect, the trained neural-network predictor can guarantee an upper bound on expected exploitability without equilibrium labels or best-response targets. Experiments on different tasks show that CENO achieves better performance in terms of equilibrium quality, online solution time, and certification.
Critical strengths
-
Predicting only the continuation values and constructing the Gibbs policy and population from them is novel and reasonable, naturally guaranteeing forward consistency while also providing a guarantee of backward optimality for the derived policy.
-
The authors use the Bellman defect-based certificate to measure the prediction quality, which can be computed from the solver output and the known MFG model alone without requiring either equilibrium labels or an additional best-response computation, making the training and evaluation across different MFG instances easier.
-
The proposed method solves a single forward pass without per-instance fixed-point iteration, and experiments show competitive or better performance in terms of equilibrium quality, online solution time, and certification.
Critical weaknesses
-
This paper is restricted to finite-state, entropy-regularized MFGs with population-independent transitions. Though this is a common type of MFG, it would be good to provide more discussion on how the current method can be adapted to a broader class of MFGs. Besides, the population-level guarantees additionally require monotone interactions, and the accumulated-error bound requires strong monotonicity, which may not hold in some cases.
-
Experimental environments seem quite simple with a deterministic transition kernel; thus, the real-world validity remains unclear.
Questions for authors
See weaknesses.
W1. How the current method can be adapted to a broader class of MFGs
Than you for this suggestion. These assumptions (finite-state, population-independent transition, entropy regularization, and monotonicity) serve different roles in CENO. We discuss their purpose, the challenges of relaxing them, and possible extensions below.
Entropy regularization provides a unique and smooth Gibbs policy from predicted values, enabling end-to-end training of a shared neural solver across MFG instances. For unregularized MFGs, we can instead construct through Bellman or Hamiltonian minimization and evaluate optimality using the corresponding residuals. The main challenge is that the optimal control may be non-unique or non-smooth, requiring the solution computed instance by instance rather than one network.
Population-independent transitions allow the population to evolve through a policy-induced Markov transition and simplify the comparison between predicted and equilibrium populations for population-level guarantees. For population-dependent transitions, the forward rollout can still use the transition kernel evaluated at the current population. However, changes in population affect the transitions, introducing additional errors that must be controlled in the certificate and population-level analysis, for example, under Lipschitz dependence of the kernel on the population, which we leave for future work.
Monotonicity is required only for the population-level guarantees, not for the individual exploitability certificate. It makes the equilibrium population unique, so the population-level bounds are well defined. Monotonicity holds for tasks in which the interaction cost responds to changes in population distribution, such as crowd evacuation, traffic routing, and multi-robot navigation. When monotonicity fails, as with herding or coordination, multiple equilibria exist, the certificate still bounds exploitability, but there is no unique equilibrium population to compare against. We believe the population-level guarantee is still meaningful, maybe not the distance to one equilibrium, but the averaged distance to the equilibrium set. This is an important direction for future work.
Finite states allow exact population propagation and direct evaluation of the maximum Bellman defect. For continuous-state MFGs, a natural extension is spatial discretization or continuous population dynamics. The main challenge comes from numerical approximation errors which cause forward consistency and residual evaluation are no longer exact.
W2. Scenario with stochastic transition kernel and additional traffic experiment.
This you for this comment. Our formulation allows stochastic transition kernel. To evaluate this and broaden the application setting, we add a stochastic Beach Bar experiment and a vehicle velocity-control experiment with physical motion constraints.
Stochastic transition experiment. In the Beach Bar experiment, with probability (p=0.2), the intended transition is replaced by a move to a uniformly sampled valid neighboring state. The deterministic and stochastic experiments use same initial populations and cost parameters. We train and evaluate all methods, including DeepONet and FNO, under the corresponding transition kernel. At (p=0.2), CENO achieves a mean exploitability of 3.919e-5 compared to 1.588e-4 in deterministic transition kernel. Their time cost is 0.69 ms and 0.67 ms respectively. It achieves a better exploitability than Master Policy (8.932e-4 at 2.5 ms), DeepONet (3.365e-2 at 0.38 ms), FNO (3.480e-4 at 0.71 ms). Fictitious play and OMD can reach a better exploitability with a time cost of more than 120 ms, but worse than CENO under the same time cost.
Vehicle velocity-control experiment. The conservativeness of our method is the cost of the generality for all instances. We conduct a vehicle experiment using fixed initial conditions and speed limited motion. These constraints allow us to bound cumulative residuals along admissible trajectories, avoiding the sum of per-time maxima that no single trajectory can realize. Evaluating both bounds for the same candidate policy, the trajectory-constrained certificate is 88%~92% lower than the original per-time bound. This experiment illustrates how physical constraints can make the certificate more informative in a traffic simulation. Therefore, our method has wide potential for specific validation.
We will include both experiments and their setup details in revised version.