- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies a parameter calibration problem: we would like to estimate parameters of a mean field game (MFG), observing the population trajectories. It proposes learning by a neural network the mapping from the population distribution to the parameter and establishes a way to compute the gradient through implicit differentiation. A range of experimental studies are performed, illustrating that the MFG parameters can be accurately learned and the proposed method outperforms simple baselines.
Critical strengths
The paper approaches the parameter calibration in MFG from an interesting, novel (at least to my knowledge) perspective. The method introduced implicitly has a bi-level structure: at the lower level, we solve the MFG under a given ; at the upper level, we optimize to minimize the gap between observed and prescribed population trajectories. The gradient with respect to is calculated through implicit differentiation.
Critical weaknesses
While I believe the technical content of the paper may be of interest to the MFG community, the writing quality and presentation clarity need substantial improvement before it should be published. Some important technical results of the paper include how MFGs are solved in the forward direction, how to exactly perform back-propagation based on the principle of implicit differentiation, and the theoretical analysis. Currently they are only described very briefly in Section 3. I believe deferring these results to the appendix significantly disrupts the natural flow of the paper and makes it difficult to understand the authors’ key contributions.
It is not clear to me how significant the theoretical results are and whether any novel analytical tools are introduced by the authors to make the analysis happen. Furthermore, Assumption B.1 seems rather restrictive, though I understand this assumption may have appeared in the existing literature. Regarding Remark B.1 -- it is true that the contraction holds when and/or the Lipschitz constants are sufficiently small, but they are dictated by the problem itself which we have no control over.
Questions for authors
In most practical applications, knowing the model exactly (up to certain parameters) seems highly unrealistic. The authors should consider studying the performance of the learning method (from a mathematical perspective and/or empirically) under model misspecification.
Minor:
-
While conceptually I understand that the Law operator here defines a consistency equation, the exact definition seems missing from the paper.
-
The authors should differentiate the use of citet and citep.
4 comments
Q1 (Analysis under misspecifications) We thank the reviewer for raising this point. We agree that exact model knowledge is unrealistic in practice, but studying a phenomenon without having the slightest intuition of a what a model would look like is equally unrealistic. We cannot hope to learn from data without any structure.
Our two real-world data experiments (influenza surveillance and bike sharing) are misspecified by construction. The games considered are too simple and we do not believe that they reflect the true data-generating process. As such, the reported errors already measure the predictive performance under misspecification.
To go deeper into the question, we note that the generalisation bound (proposition B.3) does not assume a well-specified model. It is generally defined and holds for any data distribution. As such, we have added a corollary (Corollary B.1) that quantifies the effect of the misspecification: if the true flow is an equilibrium of an operator within of our Picard operator, the predicted flow is within of the ground truth. Here, is the contraction constant of our Picard operator from Assumption B.1.
We agree that the real-data results do not quantify how the performance degrades with the degree of misspecification, for it's not known how far from the ground truth our models are. Therefore, we have run an experiment in which the data are generated from a perturbed LQ game and fitted with an unperturbed model. In particular, we add a misspecification . In our perturbed game, the control cost towards state is multiplied by , a destination-dependent congestion effect that the fitted model does not contain and that its scalar parameter cannot reproduce. We then fitted the unperturbed model exactly as in the main experiments (5 seeds, , no observation noise).
Held-out error (, mean ± SD over seeds):
| constant | time-dependent | mean-field-dependent | |
|---|---|---|---|
| 0 | 0.18 ± 0.09 | 1.80 ± 1.53 | 0.76 ± 0.74 |
| 0.1 | 0.70 ± 0.11 | 1.20 ± 0.38 | 3.01 ± 3.67 |
| 0.25 | 1.88 ± 0.20 | 2.93 ± 1.77 | 2.22 ± 0.41 |
| 0.5 | 3.11 ± 0.13 | 3.96 ± 0.88 | 3.65 ± 0.16 |
| 1 | 4.79 ± 0.23 | 6.18 ± 0.29 | 7.58 ± 0.19 |
Mean squared error of the learned parameter :
| constant | time-dependent | mean-field-dependent | |
|---|---|---|---|
| 0 | 2.7e-4 | 0.16 | 4.7e-3 |
| 0.1 | 2.9e-3 | 0.20 | 6.6e-3 |
| 0.25 | 1.2e-2 | 0.26 | 1.3e-2 |
| 0.5 | 5.7e-2 | 0.57 | 3.5e-2 |
| 1 | 0.22 | 0.97 | 0.11 |
We draw three conclusions based on these results.
- Predictions degrade gracefully. Even at , where the control cost is misspecified by up to a factor of two, the held-out error remains below , i.e. under one percentage point of population mass. For constant the error grows monotonically with . For the two other cases, the well-specified error varies considerably across seeds, and the effect of misspecification becomes visible only for .
- The learned parameter loses its interpretation. The parameter error grows steadily with in all cases: the fitted compensates for the missing term. Under misspecification, the learned parameters should therefore be read as effective parameters and not as estimates of the true ones. We now state this limitation explicitly.
- The derived a-posteriori bound holds. Across all cases, perturbation strengths, seeds and test trajectories, the ratio of prediction error to residual lies in , consistent with Corollary B.1 for , which agrees with the contraction factor measured in App. D. The residual is thus a reliable, computable indicator of prediction error under misspecification.
Even though findings are limited to the LQ example, we hope that they illustrate the method's performance under misspecification.
Minor remarks We thank the reviewer for the minor remarks, we have fixed them in the updated (and uploaded) version.
We hope this elaboration on the presentation and the analysis under misspecifications resolves your concerns, and we are happy to elaborate further. If our response has addressed your concerns, we would be grateful if you would consider raising your score accordingly.
The authors have added meaningful results and discussions, especially regarding Q1, and this deserves a score raise. I have adjusted my rating to reflect the improvement in the overall quality of the paper. The reason I am not further increasing my rating is that I still feel overwhelmed reading Section 3 (though I recognize the constraint due to the page limit).
We thank the reviewer for their swift reply. We are grateful for the score increase!
As there was still about half a page left in the manuscript, we have made the following changes to clarify Section 3. We hope that this aids the section's readability, but please let us know if there is anything else we can change.
-
We added a paragraph on the bi-level structure of the optimization loop, connecting the methods with Figure 1.
-
We have tried to steamline the notation of the parameters:
- and were used interchangibly. We now define once. After that, we consistently use wherever we mean the parameters, and use only where we want to refer to the network itself.
- We copied the notation from the updated Appendix B, indicating the parameter path along a trajectory. Here, we write the network in continuous time as subscripted with . This replaces , and .
- The coupled operator is now written as in both the method and the informal assumption.
- For the ground truth, we now use , which is consistent with Sec. 2, where is a space of functions. The previous "" was removed, and now refers only to the network weights, , as in Appendix B.
-
We also clarified the equilibrium and solver notation, were we
- reduced the equilibrium flow notation to (as in Sec. 2), , and for a fixed (as in Appendix B and Fig. 1). We removed , and .
- introduced a single notation for the equilibrium map , defined at its first use in the calibration problem. Algorithm 1 and the Lipschitz stability proposition now use instead of .
Thanks again for your reply! We know that we are getting close to the end of the dicusssion period, but we would greatly appreciate it if you could reconsider the updated manuscript.
We thank the reviewer for their feedback, and we are glad to hear the reviewer found the paper and interesting, novel perspective. We agree with the reviewer that the paper would be improved with some restructuring, and we hope that our proposal clarifies the work.
W1 (Paper structure) We are glad to hear that the reviewer considers the technical content of interest to the MFG community. We chose to move the theoretical support of our paper to the appendices due to the page limit, but we realise that this has been a mistake. As we are already allowed to extend our submission to 10 pages for the updated version, we propose to absorb the theoretical support into the main text where possible. In case the full proofs are too long, we opt for describing the proofs informally, and referring to the full version in the appendices. We have added the informal proofs for now, but we will extend them if space allows.
W2 (Novelty of tools and theoretical results) We thank the reviewer for this assessment, with which we largely agree.
We do not claim new analytical tools: the analysis combines the Banach fixed-point theorem, implicit differentiation and a covering argument. However, the paper also states that this is not the main focus. We aim to find the model that best fits the data, where our purpose is to certify our proposed algorithm. Specifically, it shows that