acceptodds
Under review as a conference paper at ICLR 2027

Neural Parameter Calibration for Finite-State Mean Field Games

Abstract

Mean field games efficiently approximate a very large population of strategic agents. While these games can aid the understanding of complex systems, their deployment in real-world settings is challenged by the specification of their parameters: mean field games (MFGs) often involve hidden preferences, constraints, and interactions that can rarely be theoretically derived or directly observed. To address this gap, we present a neural network-based framework for learning parametric, finite-state MFGs from observed population dynamics. To do so, we formulate the parameter calibration as an inverse problem and use implicit differentiation to backpropagate through the games' equilibrium. The resulting approach is fully differentiable and enables us to estimate flexible trajectory-wise parameter paths, including state- and time-dependent specifications without requiring observations of the individual agents' actions or rewards. We provide a proof for the exactness of the gradient computation in a discrete-time formulation. We validate our framework through numerical experiments across four systems of increasing complexity, ranging from synthetic linear-quadratic benchmarks to real-world urban mobility datasets.

Primary area:
learning on time series and dynamical systems
Keywords:
Finite-state mean field games, parameter calibration, neural networks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CtScOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies a parameter calibration problem: we would like to estimate parameters of a mean field game (MFG), observing the population trajectories. It proposes learning by a neural network the mapping from the population distribution to the parameter and establishes a way to compute the gradient through implicit differentiation. A range of experimental studies are performed, illustrating that the MFG parameters can be accurately learned and the proposed method outperforms simple baselines.

Critical strengths

The paper approaches the parameter calibration in MFG from an interesting, novel (at least to my knowledge) perspective. The method introduced implicitly has a bi-level structure: at the lower level, we solve the MFG under a given γ\gamma; at the upper level, we optimize γ\gamma to minimize the gap between observed and prescribed population trajectories. The gradient with respect to γ\gamma is calculated through implicit differentiation.

Critical weaknesses

While I believe the technical content of the paper may be of interest to the MFG community, the writing quality and presentation clarity need substantial improvement before it should be published. Some important technical results of the paper include how MFGs are solved in the forward direction, how to exactly perform back-propagation based on the principle of implicit differentiation, and the theoretical analysis. Currently they are only described very briefly in Section 3. I believe deferring these results to the appendix significantly disrupts the natural flow of the paper and makes it difficult to understand the authors’ key contributions.

It is not clear to me how significant the theoretical results are and whether any novel analytical tools are introduced by the authors to make the analysis happen. Furthermore, Assumption B.1 seems rather restrictive, though I understand this assumption may have appeared in the existing literature. Regarding Remark B.1 -- it is true that the contraction holds when TT and/or the Lipschitz constants are sufficiently small, but they are dictated by the problem itself which we have no control over.

Questions for authors

In most practical applications, knowing the model exactly (up to certain parameters) seems highly unrealistic. The authors should consider studying the performance of the learning method (from a mathematical perspective and/or empirically) under model misspecification.

Minor:

  1. While conceptually I understand that the Law operator here defines a consistency equation, the exact definition seems missing from the paper.

  2. The authors should differentiate the use of citet and citep.

4 comments

We thank the reviewer for their feedback, and we are glad to hear the reviewer found the paper and interesting, novel perspective. We agree with the reviewer that the paper would be improved with some restructuring, and we hope that our proposal clarifies the work.


W1 (Paper structure) We are glad to hear that the reviewer considers the technical content of interest to the MFG community. We chose to move the theoretical support of our paper to the appendices due to the page limit, but we realise that this has been a mistake. As we are already allowed to extend our submission to 10 pages for the updated version, we propose to absorb the theoretical support into the main text where possible. In case the full proofs are too long, we opt for describing the proofs informally, and referring to the full version in the appendices. We have added the informal proofs for now, but we will extend them if space allows.


W2 (Novelty of tools and theoretical results) We thank the reviewer for this assessment, with which we largely agree.

We do not claim new analytical tools: the analysis combines the Banach fixed-point theorem, implicit differentiation and a covering argument. However, the paper also states that this is not the main focus. We aim to find the model that best fits the data, where our purpose is to certify our proposed algorithm. Specifically, it shows that

  • (i) the adjoint iteration returns the gradient of the equilibrium of the coupled system, in which the network is evaluated inside the fixed point;
  • (ii) our experiments show that this remains true, in the sense of conservative gradients and hence for almost every parameter, in the presence of clipping, projection and ReLU activations;
  • and (iii) the equilibrium map is Lipschitz, which gives the M−1/2M^{-1/2} generalisation rate that we verify empirically in Appendix C.

Q1 (Analysis under misspecifications) We thank the reviewer for raising this point. We agree that exact model knowledge is unrealistic in practice, but studying a phenomenon without having the slightest intuition of a what a model would look like is equally unrealistic. We cannot hope to learn from data without any structure.

Our two real-world data experiments (influenza surveillance and bike sharing) are misspecified by construction. The games considered are too simple and we do not believe that they reflect the true data-generating process. As such, the reported errors already measure the predictive performance under misspecification.

To go deeper into the question, we note that the generalisation bound (proposition B.3) does not assume a well-specified model. It is generally defined and holds for any data distribution. As such, we have added a corollary (Corollary B.1) that quantifies the effect of the misspecification: if the true flow is an equilibrium of an operator within ϵ\epsilon of our Picard operator, the predicted flow is within ϵ/(1−κ)\epsilon / (1-\kappa) of the ground truth. Here, κ\kappa is the contraction constant of our Picard operator Ξ(⋅,γ)\Xi(\cdot, \gamma) from Assumption B.1.

We agree that the real-data results do not quantify how the performance degrades with the degree of misspecification, for it's not known how far from the ground truth our models are. Therefore, we have run an experiment in which the data are generated from a perturbed LQ game and fitted with an unperturbed model. In particular, we add a misspecification ε∈[0.0,0.1,0.25,0.5,1.0]\varepsilon \in [0.0, 0.1, 0.25, 0.5, 1.0]. In our perturbed game, the control cost towards state yy is multiplied by 1+ε μ(y)1+\varepsilon\,\mu(y), a destination-dependent congestion effect that the fitted model does not contain and that its scalar parameter bb cannot reproduce. We then fitted the unperturbed model exactly as in the main experiments (5 seeds, d=3d=3, no observation noise).

Held-out error ∥μobs−μˉ∥∞\|\mu^{\mathrm{obs}}-\bar\mu\|_\infty (×103\times 10^{3}, mean ± SD over seeds):

ε\varepsilonconstant bbtime-dependent bbmean-field-dependent bb
00.18 ± 0.091.80 ± 1.530.76 ± 0.74
0.10.70 ± 0.111.20 ± 0.383.01 ± 3.67
0.251.88 ± 0.202.93 ± 1.772.22 ± 0.41
0.53.11 ± 0.133.96 ± 0.883.65 ± 0.16
14.79 ± 0.236.18 ± 0.297.58 ± 0.19

Mean squared error of the learned parameter bb:

ε\varepsilonconstant bbtime-dependent bbmean-field-dependent bb
02.7e-40.164.7e-3
0.12.9e-30.206.6e-3
0.251.2e-20.261.3e-2
0.55.7e-20.573.5e-2
10.220.970.11

We draw three conclusions based on these results.

  1. Predictions degrade gracefully. Even at ε=1\varepsilon=1, where the control cost is misspecified by up to a factor of two, the held-out error remains below 8×10−38\times10^{-3}, i.e. under one percentage point of population mass. For constant bb the error grows monotonically with ε\varepsilon. For the two other cases, the well-specified error varies considerably across seeds, and the effect of misspecification becomes visible only for ε≥0.25\varepsilon \ge 0.25.
  2. The learned parameter loses its interpretation. The parameter error grows steadily with ε\varepsilon in all cases: the fitted bb compensates for the missing term. Under misspecification, the learned parameters should therefore be read as effective parameters and not as estimates of the true ones. We now state this limitation explicitly.
  3. The derived a-posteriori bound holds. Across all cases, perturbation strengths, seeds and test trajectories, the ratio of prediction error to residual lies in [0.93,1.02][0.93, 1.02], consistent with Corollary B.1 for κ≈0.07\kappa \approx 0.07, which agrees with the contraction factor measured in App. D. The residual is thus a reliable, computable indicator of prediction error under misspecification.

Even though findings are limited to the LQ example, we hope that they illustrate the method's performance under misspecification.


Minor remarks We thank the reviewer for the minor remarks, we have fixed them in the updated (and uploaded) version.


We hope this elaboration on the presentation and the analysis under misspecifications resolves your concerns, and we are happy to elaborate further. If our response has addressed your concerns, we would be grateful if you would consider raising your score accordingly.

The authors have added meaningful results and discussions, especially regarding Q1, and this deserves a score raise. I have adjusted my rating to reflect the improvement in the overall quality of the paper. The reason I am not further increasing my rating is that I still feel overwhelmed reading Section 3 (though I recognize the constraint due to the page limit).

We thank the reviewer for their swift reply. We are grateful for the score increase!

As there was still about half a page left in the manuscript, we have made the following changes to clarify Section 3. We hope that this aids the section's readability, but please let us know if there is anything else we can change.

  1. We added a paragraph on the bi-level structure of the optimization loop, connecting the methods with Figure 1.

  2. We have tried to steamline the notation of the parameters:

    • γθ\gamma_\theta and ϕθ\phi_\theta were used interchangibly. We now define γθ=ϕθ\gamma_\theta = \phi_\theta once. After that, we consistently use γθ\gamma_\theta wherever we mean the parameters, and use ϕθ\phi_\theta only where we want to refer to the network itself.
    • We copied the notation γθ[μ]\gamma_\theta[\mu] from the updated Appendix B, indicating the parameter path along a trajectory. Here, we write the network in continuous time as ϕθ(t,μt)\phi_\theta(t, \mu_t) subscripted with tt. This replaces γtk=φθ(t,μt(k))\gamma^k_t = \varphi_\theta(t,\mu_t^{(k)}), φθ(⋅,μ)\varphi_\theta(\cdot,\mu) and Ξ(μ,φθ(⋅,μ))\Xi(\mu,\varphi_\theta(\cdot,\mu)) .
    • The coupled operator is now written as Ξθ(μ):=Ξ(μ,γθ[μ])\Xi_\theta(\mu) := \Xi(\mu,\gamma_\theta[\mu]) in both the method and the informal assumption.
    • For the ground truth, we now use γ∗∈Γ\gamma^* \in \Gamma, which is consistent with Sec. 2, where Γ\Gamma is a space of functions. The previous "Γ⊂Rp\Gamma \subset \mathbb{R}^p" was removed, and Rp\mathbb{R}^p now refers only to the network weights, θ∈Θ⊂Rp\theta \in \Theta \subset \mathbb{R}^p, as in Appendix B.
  3. We also clarified the equilibrium and solver notation, were we

    • reduced the equilibrium flow notation to μˉγ,μ0\bar\mu^{\gamma,\mu_0} (as in Sec. 2), μˉi,θ:=μˉγθ,μ0i\bar\mu^{i,\theta} := \bar\mu^{\gamma_\theta,\mu^i_0}, and μˉθ\bar\mu^\theta for a fixed μ0\mu_0 (as in Appendix B and Fig. 1). We removed μˉγ\bar\mu^{\gamma}, μ(k),γ\mu^{(k),\gamma} and μˉθ,μ0\bar\mu^{\theta,\mu_0}.
    • introduced a single notation for the equilibrium map Φ(γ,μ0):=μˉγ,μ0\Phi(\gamma,\mu_0) := \bar\mu^{\gamma,\mu_0}, defined at its first use in the calibration problem. Algorithm 1 and the Lipschitz stability proposition now use Φ(γθ,⋅)\Phi(\gamma_\theta,\cdot) instead of Φ(φθ,⋅)\Phi(\varphi_\theta,\cdot).

Thanks again for your reply! We know that we are getting close to the end of the dicusssion period, but we would greatly appreciate it if you could reconsider the updated manuscript.

Reviewer_BPefOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a neural framework for calibrating finite-state mean field games from macroscopic population trajectories, without observing individual actions or rewards. Neural networks represent time- and population-dependent model parameters, and training uses implicit differentiation through a coupled HJB–FPK equilibrium solver. The paper provides conditional results on gradient correctness and generalization, and evaluates the approach on several representative models. The proposed method offers some novelty in addressing an important calibration problem. However, the manuscript in its current form does not meet the standard for publication at ICLR.

Critical strengths
  1. Novel and practically relevant problem formulation. Rather than using neural networks only to solve a forward MFG problem, the paper places a neural parameterization on top of a classical HJB–FPK MFG solver to calibrate unknown model parameters from aggregate population trajectories, without observing individual actions or rewards. This is a meaningful and relatively underexplored use of neural methods, and the problem has clear practical value.

  2. Broad empirical coverage. The experiments span controlled synthetic settings and several representative real-world models, and examine multiple aspects of the proposed approach, including gradient behavior, robustness to noise, distributional changes, and counterfactual behavior. The experiments help illustrate the method’s behavior and potential practical value.

Critical weaknesses
  1. The current experiments do not adequately support the paper’s central claims. The paper claims to predict population trajectories from initial conditions, but the supplied cybersecurity and influenza evaluation code feeds the observed test trajectories into the parameter networks. Such reconstruction results do not establish prediction from initial conditions alone. In the bike-sharing comparisons, the MFG models have additional learned costs and, in the general-cost variant, a flexible neural parameterization. In at least one comparison, the baseline is restricted to fixed parameters while the proposed model is allowed time-varying parameters. This confounds the MFG structure with the ability to model temporal variation, so the comparison cannot establish that an MFG model outperforms a non-MFG model because of the game structure itself. The supplied SIR implementation also appears inconsistent with the action-dependent cost in Eq. E.7 and uses initial states that are not normalized probability distributions. It is unclear whether the reported influenza results were generated with this implementation. Together, these concerns leave insufficient evidence for the claimed predictive capability and benefits of the game structure; addressing them may require substantial changes to the core experiments.

  2. The presentation and organization substantially impede technical assessment. Important components of the method are described inconsistently across the overview figure, Section 3.1, and Algorithm 1. Experimental settings are repeatedly introduced in the main text and appendices, while the four cases are split across several objectives and qualitatively different types of evidence are discussed together. Key quantitative comparisons and limitations are also separated from the corresponding claims. As a result, the relationship between model inputs, evaluation protocols, results, and claimed conclusions is difficult to follow. This is a substantive clarity problem because it prevents reliable assessment of the scope and validity of the reported evidence.

Questions for authors
  1. Can the authors provide cybersecurity and influenza test results that use only the initial state, without the observed test trajectory, to support the claimed prediction ability?

  2. Can the authors provide a fair comparison in which the baseline has the same parameter flexibility, including dynamic parameters, so that any improvement can be attributed to the MFG structure?

3 comments

We thank the reviewer for recognizing the relevance of the problem considered in our work. In addition, we are glad that the reviewer appreciates the broad coverage of our experiments. The reviewer's concerns focus on the experimental support of the central claims made in the paper, which has helped us identify a few inconsistencies in the description of our methods. We hope that the elaboration below helps to resolve the reviewer's concerns.


W1 (On the experimental support of the central claims) In its current form, the paper is indeed imprecise on the input data for the network. We apologise for this inconsistency, which comes from an earlier formulation of our method, where the network φθ\varphi_\theta received the whole observed mean field flow to predict γ\gamma. We used to refer to this step as the pre-evaluation. It turns out that this pre-evaluation was unnecessary: the model can accurately predict the parameter estimates during the Picard solve, such that the predictions can dynamically be based on μt\mu_t and time. In addition, we have often used the notation φθ(⋅,μˉγ,μ0)\varphi_\theta(\cdot, \bar\mu^{\gamma, \mu_0}) to indicate that we use φθ\varphi_\theta at each time step to predict γ(t)\gamma(t) based on μˉγ,μ0\bar\mu^{\gamma, \mu_0} (Incorporating the Neural Network, line 229). We have elaborated on this notation in the updated version of the paper (lines 229-235).

We have reviewed our code on the reviewer's directions, and we found that the SIR experiment uses the full trajectories for calculating the training loss only, which is in accordance with the paper's central claims. However, the cybersecurity experiment was indeed still based on the pre-evaluation. To correct this matter, we have rerun all cybersecurity experiments, and updated the figures and experimental outcomes (Appendix G) in the new version of the paper.

  • On the inconsistency of the bikeshare experiment, we corrected our code have rerun the comparison. We have updated Figure 3, 6 and Appendix H accordingly.

  • For the SIR experiment, the reviewer is correct that the initial distribution is not a normalized probability distribution. The CDC data we used for this experiment reports the percentage of infected individuals that have been considered by the testing centre. We turn this signal into a probability by dividing it by 100. We consider it a proxy for the fraction of infected individuals in the total population (experiments/SIR/nrevss.py, line 106). Because we cannot know what fraction of the population is susceptible and what fraction is recovered, we have filled the rest of the array with dummy values. Albeit this is not a valid probability distribution, this does not affect our experiments, for we only consider the data at index 1 (experiments/SIR/nrevss.py, line 48, 123).

  • Regarding the inconsistencies with the action dependent cost, we have indeed replaced the action dependent cost function with a learned cost in our code (src/mean_field_games/SIR.py, line 37). We have changed the code to match Equation 7, and we have updated Figure 5 accordingly.


W2 (Inconsistent presentation) We thank the reviewer for this remark, and we apologise for the inconsistencies that still point to the old pre-evaluation method. We have carefully reread the paper, have clarified our approach in the Methods (including Algorithm 1), Appendix A and Appendix B. We have also reviewed the claims made in the Introduction and Conclusion to match the updated formalization.


Q1 (Rerun cybersecurity and influenza test results) We have rerun both experiments with the fixed code, and we have updated the paper accordingly (Figure 5, Appendix G, K.3 and K.5). The cybersecurity results did not notably change. However, with the updated definition of the loss function, the out-of-distribution predictive performance of the SIR model has improved to a mean L2L2 of 4.326e−024.326e-02 with a standard deviation of 5.806e−025.806e-02 over three seeds.


Q2 (Bikeshare comparison) We thank the reviewer for pointing this out. We have extended the existing forward dynamics baseline with a time-dependent network that has the same architecture as the cost network from the MFG. We refer to our answer to Q3 for reviewer 7WmA for an elaboration on this extension, as well as the results.


We hope this elaboration on the presentation the reformulation of the forward dynamics baseline resolves your concerns, and we are happy to elaborate further. If our response has addressed your concerns, we would be grateful if you would consider raising your score accordingly.

Thank you for the additional experiments and detailed explanations. I appreciate the effort, but I will maintain my score because the presentation is still not good enough.

We thank the reviewer for their kind reply, and we are happy to hear that the reviewer appreciates the additional experiments and elaboration. We're sorry that the presentation and organisation of the paper still impedes its quality. We have therefore restructured Section 4 and 5. We think that the paper has been improved, and we hope that these efforts improve your assessment as well. We've reuploaded the latest version of the manuscript.


We have reviewed the notation in Figure 1, Section 3.1 and Algorithm 1. The parameter function is written γθ:=φθ\gamma_\theta := \varphi_\theta throughout. Evaluating it along the solver's current iterate is denoted γθ[μ]\gamma_\theta[\mu], matching the label in Fig. 1 and the notation in App. B. The solver map Φ(γ,μ0)\Phi(\gamma,\mu_0) now has a single signature in the text, Algorithm 1 and the theory.

  • Each experimental setting is now introduced once in the main text, and only elaborated upon in the appendix.

    • We included a new overview table (Table 1) at the start of Sec. 4. For every test case it lists the data, the state dimension, the learned parameters, the train/test split, the objective it addresses, and where the results are.
    • Urban-mobility models in one place. The baseline and MFG models are now defined only in Sec. 4; the duplicated descriptions and equations were removed from Sec. 5.
  • Model inputs and evaluation protocol stated explicitly

    • New opening paragraph in Sec. 5. It states that, at test time, every model receives only the initial distribution μ0\mu_0, and that the observed test trajectory is used only to compute the error. It also gives the number of seeds per experiment and notes that the real-data experiments go beyond the i.i.d. setting of the generalization bound.
  • Influenza setup stated in the main text:
    • the chronological split (training 2015–2022, testing 2023–2026);
    • that the loss uses only the observed infected coordinate;
    • that the susceptible and recovered trajectories are latent quantities implied by the model.
  • We only added a single type of evidence per paragraph. Objective 3 split into three labelled parts: noise robustness (LQ), out-of-distribution forecasts (influenza), and novel scenarios (urban mobility). We've added these labels in italic.

  • We've placed comparisons and limitations next to the claims

    • Objective 2 rewritten around a capacity-matched comparison. The baseline and the MFG are compared at three matched levels of capacity, and the results appear in a new main-text table (Table 2) instead of only in App. H.
    • The result is stated directly:
      • The game structure reduces the in-distribution error when the base rates are restricted.
      • Both models perform equally well when they have matched neural capacity.
      • Neither model generalizes to weekends.
      • Fig. 3 now shows only the trajectories; the error comparison is in Table 2.
      • Limitations now point to results found: the weekend shift (Table 2, Test OOD), and the interpretation of the learned parameters as effective parameters under misspecification (Cor. B.1).

As such, we are grateful for your reply! We know that we are getting close to the end of the dicusssion period, but we would greatly appreciate it if you could reconsider the updated manuscript.

Reviewer_7WmAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes a method for estimating unknown parameters in finite-state MFGs from observed trajectories, without observing individual actions or rewards. A neural network produces parameter paths and a forward-backward solver computes the equilibrium. Training differentiates through the equilibrium condition instead of differentiating through every solver iteration. The paper provides theoretical guarantees under contraction and smoothness assumptions (however, these are in the appendix) and evaluates the method on various experiments.

Critical strengths
  • I found the problem to be well motivated.
  • The evaluation on several experiments is substantial.
  • I found Appendix B to be useful, although it seems the paper's focus is not theoretical. In particular, Proposition B.2 shows that the equilibrium’s sensitivity depends on the contraction constant, so this assumption seems to be quite crucial. Also, Remark B.2 distinguishes the exact fixed-point result from the approximate computation used in practice.
Critical weaknesses
  • The theory assumes smooth, contractive Picard map. It is acknowledged in Appendix B that the implementation includes clipping, projections and discrete optimization, but these operations can violate the smoothness assumption.
  • I am confused about the generalization claims. As far as I understand, the method evaluates the network on observed trajectories to obtain the parameters used by the MFG solver. This makes sense for calibration, but I don't see how the method would predict a new trajectory when only the initial population distribution is available. In particular, it seems that in Proposition B.3 the bound uses a loss function based on the full observed trajectory as input, so it's not predicting from the initial distribution alone.
Questions for authors
  • Can the authors provide any ideas on how to weaken the assumptions so that they are better aligned with the experimental regimes?
  • When you evaluate on a new trajectory, do you provide the entire observed trajectory to the network or do you generate the prediction using only the initial distribution?
  • This is more of an exploratory question: is is possible to compare the method with a mean field dynamics baseline that uses a neural network of similar size? I'm curious because one of the objectives of the paper is to demonstrate the benefits of leveraging strategic behavior through MFGs.
2 comments

We thank the reviewer for appreciating our problem motivation, experimental setup and theoretical foundations. In what follows, we address each of the reviewer's concerns individually. The concerns focused on the smoothness assumption, Picard contractivity and the generalization of our claims. Engaging with them has led us to weakening our assumptions on the Lipschitz continuity of the equilibrium map, which supports the use nonsmooth operations in the experimental setup.


W1 and Q1 (Smoothness assumption and Picard contractivity) We thank the reviewer for this remark and related question. We have split our answer into two perspectives. We first review the relation between the theory and our method's empirical performance. Next, we make a proposal for reframing Proposition B.1 to allow for nonsmooth operations in the experimental setup.

  • (i) First, we see as a positive outcome the strong empirical performance of our method in cases that are not covered by the theory: it demonstrates the method's robustness beyond the theoretical conditions. As such, we consider the assumption on smoothness a sufficient condition for our theoretical analysis, rather than a required one.

  • (ii) Nonetheless, we agree that we can better accommodate our experimental design by weakening our assumptions, which sharpens our work. Smoothness and contractivity. We first identify what theoretical results use the smoothness and contractivity of the Picard map. The Lipschitz continuity of the equilibrium map (Proposition B.2) and the generalisation bound (Proposition B.3) rely only on contractivity and Lipschitz dependence on γ\gamma. Clipping and projection are Lipschitz, so these results apply to the implemented operator without modification. All statements are made for the time-discretised operator used in our code, so in this case, discretisation introduces no gap between theory and implementation. Weakened assumption for the adjoint gradient. Only Proposition 1 used C1C^1 regularity (the continuous differentiability of the Picard map, Assumption B.2). We can replace it by local Lipschitz continuity plus definability, which covers clipping, projection and max/min operations. The revised Proposition 1 states: (a) whenever no nonsmooth operation is exactly at its kink along the equilibrium trajectory, the original result holds verbatim, since the implicit function theorem is local; (b) in general, the adjoint iteration still converges, because every element of the Clarke Jacobian inherits the contraction bound, and its output is a conservative gradient of the loss in the sense of Bolte and Pauwels (2021), by the nonsmooth implicit function theorem of Bolte et al. (2021). It therefore coincides with the true gradient for almost every γ\gamma. We did not add this proposition to the paper yet, but we are happy to do so if you think that this approach improves the work.


W2 and Q2 (Generalization claims) There are several ways to generate a new trajectory from μ0\mu_0 only.

  • In the special case that γ\gamma is a constant, we don't need a neural network φθ\varphi_\theta to estimate the rates. Instead, we can train a single parameter θ\theta.
  • In the general case, if γ\gamma is a function of time, we learn the neural network φθ\varphi_\theta as a function of time. The network may have μt\mu_t as an input as well, although this, depending on the problem at hand, may not be necessary.

Once the model is trained, we have a candidate for γ\gamma that we can use for inference. Given γ\gamma and the initial value μ0\mu_0, we can use our forward-backward solver to compute the MFG solution and the corresponding, predicted mean field flow (the μˉ\bar \mu associated with the equilibrium).

We train our calibration method on different initializations to improve the generalization of the method (Section 3.1). Here, we acknowledge that for some problems, the network may learn the "perfect" solution for the initial training distributions, but may not generalize to unseen trajectories. We quantify the gap between the expected loss on new data and the average loss on the training data in Proposition B.3.

Q3 (Mean field dynamics baseline) We thank the reviewer for this question. The paper contains a mean field dynamics baseline for the urban mobility example (Objective 2 and Appendix H), but the baseline does not have the same size as the MFG for all tests. We clarify the model architectures below and add the comparison that was missing.

All models use the same forward equation, data, optimiser and seeds, and differ only in what is learned.

ComparisonMF dynamics baselineMFG
Test 1 (constant base rates)3038
Test 2 (affine base rates)6068
Test 3 (affine base rates + cost network)606,689
Test 3, matched baseline (new)6,6226,689

Tests 1 and 2 are matched in size. The baseline and the MFG share the same parameterisation of the base transition rates. The MFG adds only two cost scalars and a terminal cost vector, i.e. 8 parameters. The reduction in held-out error in these tests (Table 3) is therefore attributable to the game structure and not to model capacity.

Test 3 was not matched, as the reviewer's question correctly implies. There, the MFG has a neural cost network while the baseline remains affine in time. We have added a baseline in which the transition rates are produced by a network with the same architecture and inputs as the MFG's cost network, q_{xy}(t,\mu_t) = \mathrm{softplus}\big(\mathrm{NN}_{xy}(t,\mu_t)\big), with no value function and no equilibrium condition. It is trained with the same data, seeds, optimiser and number of epochs as the MFG.

Absolute L2L_2 error (×103\times 10^{3}), mean ± SD over 5 seeds:

SplitOriginal baselineMatched NN baselineMFG (Test 3)
Train1.02 ± 0.010.85 ± 0.010.84 ± 0.01
Test, in-distribution (weekdays)0.84 ± 0.010.67 ± 0.020.67 ± 0.01
Test, out-of-distribution (weekends)3.10 ± 0.053.25 ± 0.103.27 ± 0.04

The performance of the matched baseline and the MFG is similar on all splits. The improvement of the Test 3 MFG over the affine baseline is therefore explained by model capacity and not by the game structure, and we have corrected the corresponding statements in the paper.

Taken together, the three tests show that the benefit of the game structure for predictive accuracy decreases as the dynamics model becomes more flexible: it is substantial for constant rates, smaller for affine rates, and absent for a neural rate network. This is consistent with theory: when the equilibrium is unique, the equilibrium transition rates are a function of time and the current distribution, which a sufficiently flexible forward model can learn directly. The MFG formulation matches the accuracy of the flexible forward model while additionally providing interpretable cost parameters and the ability to evaluate announced interventions without retraining (Fig. 5), for which a forward-only model has no mechanism.


We hope this elaboration on the smoothness of the solver and the reformulation of the forward dynamics baseline resolves your concerns, and we are happy to elaborate further. If our response has addressed your concerns, we would be grateful if you would consider raising your score accordingly.

References

  • Bolte, J., and Pauwels, E. (2021). Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning. Mathematical Programming, 188(1), 19-51.
  • Bolte, J., Le, T., Pauwels, E., and Silveti-Falls, T. (2021). Nonsmooth implicit differentiation for machine-learning and optimization. Advances in neural information processing systems, 34, 13537-13549.`

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.