acceptodds
Under review as a conference paper at ICLR 2027

Bayesian Learning of Parametric Mean Field Games with Common Noise

Abstract

We study the inverse problem of recovering unknown parameters governing the dynamics and rewards of Mean Field Games (MFGs) from partial observations of population trajectories, made stochastic by common noise. The observer seeks to recover a consistent parameter via Bayesian inference, facing the challenge that evaluating the likelihood requires solving a new Nash equilibrium (NE) for each candidate . Due to common noise, the mean-field evolution is stochastic and equilibrium policies are population dependent, which makes solving for each parameter computationally expensive. To overcome this, we introduce the *Bayesian-Extended Mean Field Game* (BE-MFG), whose single NE simultaneously encodes equilibrium policies for all parameters, enabling efficient learning of the conditional density of mean-field trajectories. We formalize parameter consistency and identifiability in MFGs with common noise and investigate conditions under which consistent parameter recovery is guaranteed. On three synthetic environments of various complexity, we recover the ground-truth parameter accurately from a small number of partial, stochastic observations, outperforming an adaptation of inverse reinforcement learning for MFGs. A fourth experiment, on public electricity market data, suggests that the approach can also be applied to real-world data.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Mean field games, Inverse problem, Bayesian learning, Common noise, Nash equilibrium

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_KvhgOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Bayesian parameter inference in mean field games (MFGs) with common noise, where unknown parameters govern the reward functions and transition dynamics, and only partial population-level trajectories are observed. The authors propose a Bayesian-Extended Mean Field Game (BE-MFG) formulation that embeds the parameter space into an augmented game, allowing a single Nash equilibrium to represent equilibria across different parameter values. They develop a computational pipeline combining fictitious play, FiLM-conditioned neural policies, and conditional density estimation using normalizing flows or Gaussian processes to perform Bayesian inference. Theoretical results concerning parameter identifiability and equilibrium equivalence are established. Experiments on three synthetic environments and real-world electricity market data demonstrate parameter recovery and compare the proposed method against an adapted mean-field inverse reinforcement learning baseline.

Critical strengths

Interesting formulation: The proposed BE-MFG provides a unified framework for learning parameter-dependent equilibrium policies, avoiding repeated Nash equilibrium computations during Bayesian inference.

Practical methodology: The combination of fictitious play, parameter-conditioned neural networks, and conditional density estimation offers a flexible approach to recovering unknown reward and transition parameters from partial, stochastic population observations.

Diverse experiments: The paper evaluates the method on several synthetic environments with different parameter dimensions and observation settings, as well as a real-world electricity market dataset, demonstrating its potential applicability.

Critical weaknesses

1. Methodological novelty

The proposed BE-MFG formulation is closely related to amortized equilibrium learning, where a parameter-conditioned policy is trained to represent equilibria across different parameter values. Moreover, the computational pipeline primarily combines existing techniques, including fictitious play, FiLM, and normalizing flows. The authors could better clarify the methodological novelty relative to existing parameter-conditioned equilibrium solvers and explain which components are essential to the proposed approach.

2. Theoretical contributions

Although the paper presents several theoretical results, most are structural properties of the proposed formulation rather than substantive learning guarantees. In particular, Theorem 3.6 follows almost immediately from the definitions, while Theorem 4.2 primarily exploits the decomposition of the augmented game into parameter-specific MFGs. The latter provides useful justification for the proposed construction, but the overall technical depth appears limited.

More importantly, despite the paper's emphasis on Bayesian parameter learning and identifiability, it provides neither asymptotic consistency results for the proposed MAP estimator nor finite-sample estimation bounds or posterior concentration guarantees. The effects of approximate equilibrium computation, policy distillation, and conditional density estimation on the final parameter recovery are also not analyzed. Consequently, the current theoretical results do not establish when or how accurately the proposed pipeline can recover the true parameter from finitely many observations.

Additionally, the applicability of Assumption 2 to the experimental environments deserves clarification. Since the synthetic experiments involve continuous unknown parameters but discrete common noise, the conditional laws of exact mean-field trajectories may be atomic and need not be absolutely continuous with respect to the prior predictive distribution. It is therefore unclear whether the Radon–Nikodym likelihood assumed in Lemma 4.1 exists in the experimental settings.

3. Experimental validation

The experiments cover several settings, but additional comparisons would help establish the advantages of the proposed method. In particular, the MF-IRL baseline is not originally designed for common noise, and its adaptation averages out common-noise effects. Comparisons with stronger parameter-conditioned equilibrium learning or inverse-MFG baselines would be informative. The grid-search comparison could also be made more balanced by accounting for differences in parameter resolution. Furthermore, reporting end-to-end computational costs, evaluating more diverse high-dimensional parameter configurations, and including ablation studies and posterior calibration metrics would provide stronger evidence of the method's effectiveness and robustness.

Questions for authors
  1. Methodological novelty: How does the BE-MFG formulation differ from directly training a parameter-conditioned equilibrium policy using fictitious play? Beyond theoretical interpretation, what specific advantages does the augmented game formulation provide?

  2. Validity of theoretical assumptions: Can the authors clarify whether Assumptions 1 and 2 hold in the experimental environments? In particular, given continuous parameter spaces and discrete common noise, how is the absolute continuity required for the likelihood in Lemma 4.1 guaranteed?

  3. Statistical guarantees: Can the authors provide further insight into the statistical properties of the proposed estimator? In particular, under what conditions can parameter consistency or posterior concentration be expected, and how might equilibrium approximation and density estimation errors affect parameter recovery? (I know this is hard, but will strengthens this work substantially if established)

  4. Computational efficiency: Since avoiding repeated equilibrium computations is a central motivation, how does the end-to-end computational cost compare with alternative approaches when policy distillation and conditional density estimation are included? Under what conditions does the amortized approach offer a substantial computational advantage?

  5. Experiments: Please address my concern in the "Weakness-Experimental validation" part.

5 comments

We thank the reviewer for their review and for acknowledging the interest of the formulation, the practical methodology, and the diversity of the experiments. We respectfully believe that some of the concerns do not fully reflect the content of the paper, and we clarify each of them below.

W1:

We are sorry but we did not find any reference corresponding to the exact expression “amortized equilibrium learning” so we are not sure what you mean. Can you please clarify precisely with respect to which paper(s) you would like us to compare our contributions?

Our contribution is not a new neural architecture, and we do not claim it is. It is:

  • The augmented game. The BE-MFG is a standard MFG with common noise whose equilibrium encodes, simultaneously, an equilibrium of every PMFG of the family. This is proved (Theorem 4.2).
  • Its use for the inverse problem under common noise. Since the BE-NE gives the law of the observations for every θ\theta, the likelihood of the observations can be learned once, and the inference no longer requires any equilibrium computation.
  • The well-posedness analysis of the inverse problem under common noise: consistency and identifiability (Section 3), which, to our knowledge, have not been studied before for MFGs.

As for the components: fictitious play, FiLM and normalizing flows can be replaced by other methods. What is essential is the augmented game: any MFG solver can compute the BE-NE, and any conditional density estimator can learn the likelihood. Relying on standard, well-understood components for each step is precisely what makes the pipeline credible.

W2:

We address the three paragraphs in turn.

Technical depth. Theorem 4.2 is the result that makes the method correct: without it, there is no reason why a policy trained by fictitious play on a single augmented game should induce an equilibrium of each PMFG. Its proof requires handling the mean field on the product space Θˉ×X\bar\Theta \times \mathcal X under common noise (the lemmas of App. B). The identifiability results are, to our knowledge, the first for inverse problems in discrete-time MFGs with common noise. We claim that they are the right notions for this problem, which had not been formalized before. The main contribution of our paper is the formulation of the problem.

Statistical guarantees. Please see our answer to Q3.

Assumption 2 and the atomic laws. Assumption 2 is not required to run the method, and none of our experiments relies on it. One must distinguish the mathematical assumptions, which are needed to treat the theory rigorously, from the numerical experiments, which, as is common practice, are designed to go beyond them. Assumption 2 is used in a single place: to guarantee that the likelihood of Lemma 4.1 exists as a Radon-Nikodym derivative, so that the posterior is well defined mathematically. The pipeline itself never uses it: the normalizing flow learns a continuous conditional density from simulated trajectories, and the posterior is computed from this learned density, whatever the nature of the exact laws.

W3:

We respectfully disagree with several points.

  • Baselines with common noise. As stated in the paper (related work), there is no existing method for inverse problems in MFGs with common noise. This is precisely the gap that our work fills. MF-IRL is the closest existing inverse-MFG method, and we adapted it because it is the only available baseline; we do not see how a method that does not exist can be compared. We are not aware of "stronger parameter-conditioned equilibrium learning or inverse-MFG baselines" either (see W1); we would be glad to include any specific reference.
  • Grid-search resolution. The purpose of the grid-search comparison is to show that grid search is not a credible baseline for inference in MFGs. Even at a coarse resolution, the grid is already much more expensive than the BE-MFG, and despite this much larger computational budget, it is not more accurate (App. J, Tables 8 and 10).
  • End-to-end cost. See Q4.
  • High-dimensional parameters. District has a 10-dimensional parameter, a setting in which we show that grid search cannot even be run (see App. J). We believe this is convincingly a high-dimensional setting for inverse problems in MFGs.
  • Ablations and calibration. The number of observations NN, the observation window I\mathcal I, the type of observation (ρ\rho or μ\mu) and the common-noise intensity are all varied in the paper, which already constitutes an ablation study. If the reviewer has another specific ablation in mind, we would be happy to hear it.

Q1/Q2:

Please see our answer to W1/W2.

Q3:

We agree this question is important.

  • Posterior consistency. Our NN observations are i.i.d. episodes given θ0\theta_0, and identifiability (Section 3) is exactly the injectivity of θ↦Pθ,I\theta \mapsto \mathbb P_{\theta,\mathcal I}. Doob's consistency theorem (Doob, 1949) then applies: the posterior concentrates at θ0\theta_0 for qq-almost every θ0\theta_0, without any further regularity. This is precisely why identifiability is the right notion, and Theorem 3.6 gives a sufficient condition for it. Under the classical Wald conditions (Wald, 1949; compact Θˉ\bar\Theta, continuity of the likelihood in θ\theta, identifiability), the MAP is also consistent.
  • Effect of the approximation errors. The approximate BE-NE and the density estimator give an approximate likelihood Γ~\tilde\Gamma, so the posterior we compute is that of a misspecified model. For misspecified models, under regularity conditions on the model and the prior, the posterior is known to concentrate on the parameters that minimize the Kullback-Leibler divergence from the true law of the observations (Kleijn and van der Vaart, 2006, 2012). Checking these conditions for a learned likelihood, and quantifying the resulting displacement of the parameter, would require a stability estimate of the equilibrium laws with respect to θ\theta, which is an open question for MFGs with common noise.

We will add the first two points to the paper. Finite-sample bounds would require new results on the stability of MFG equilibria under common noise, which we believe are beyond the scope of this paper.

We would also like to emphasize that theoretical guarantees for Bayesian inference are an extremely challenging topic, (Ghosh and Ramamoorthi, 2003; Ghosal and van der Vaart, 2017). Asking for such derivations in a conference paper limited to 9 pages seems excessive, especially since the paper already formalizes the main concepts that make such derivations possible. Our paper lays the theoretical and empirical groundwork for such studies, and should be assessed as such.

Q4:

Could you please let us know specifically what you would to like know, which is not already in our paper? We would like to draw your attention to the following elements:

  • Section 4.1 (after the definition of the MAP and BMA estimators) explains why a grid of dedicated solves is not viable, and refers to App. J for the quantitative comparison.
  • Section 4.2 (end of "Link between the models") states that the BE-MFG is a standard MFG with common noise, solved with the same algorithms as a single PMFG and at a comparable cost (App. B and App. J).
  • App. J, "Computational cost", Table 8 (Beach Bar): at each θ\theta, a dedicated PMFG reaches 5%5\% relative exploitability in a time comparable to the single BE-MFG solve. A grid of 88 nodes costs 3,3803{,}380 s, whereas a single BE-MFG solve reaches 5%5\% at all the nodes in 1,082±5901{,}082 \pm 590 s.
  • App. J, Table 9 (District, d=10d = 10): a single BE-MFG solve costs about as much as one dedicated solve, whereas a grid with only three points per axis would require 310=59,0493^{10} = 59{,}049 dedicated solves.
  • App. J, "Accuracy", Table 10: at a lower cost, the BE-MFG recovers the parameter about 1.71.7 times more accurately than the grid.

The answer to the second question follows from these tables: the amortized approach offers a substantial advantage as soon as the equilibrium is needed at more than a few parameters, which is the case already in dimension 11, and its advantage grows exponentially with the dimension of the parameter, since the cost of a grid does, while the cost of the BE-MFG stays comparable to one dedicated solve.

Q5:

Please see our answer to W3.

In light of these answers, we kindly ask the reviewer to reassess the value of our work and to consider raising the score. If any concern remains, we would be glad to answer further questions during the discussion period.

References

  • Doob, J. L. (1949). Application of the theory of martingales. Le Calcul des Probabilités et ses Applications, Colloques Internationaux du CNRS 13, 23--27.
  • Ghosal, S. and van der Vaart, A. (2017). Fundamentals of Nonparametric Bayesian Inference. Cambridge University Press.
  • Ghosh, J. K. and Ramamoorthi, R. V. (2003). Bayesian Nonparametrics. Springer.
  • Kleijn, B. J. K. and van der Vaart, A. W. (2006). Misspecification in infinite-dimensional Bayesian statistics. The Annals of Statistics, 34(2), 837--877.
  • Kleijn, B. J. K. and van der Vaart, A. W. (2012). The Bernstein--von Mises theorem under misspecification. Electronic Journal of Statistics, 6, 354--381.
  • Wald, A. (1949). Note on the consistency of the maximum likelihood estimate. The Annals of Mathematical Statistics, 20(4), 595--601.

Thank you for the response and clarifications. I unterstand that the primary contribution is the BE-MFG formulation rather than the individual computational components, and I appreciate the additional discussion of the theoretical and computational aspects. However, I still have several concerns.

  1. Methodological novelty. By 'amortized equilibrium learning', I refer to learning a parameter-conditioned policy that represents equilibria across a family of games, rather than solving each equilibrium independently. Related examples include Generalization in Mean Field Games by Learning Master Policies (Perrin et al., AAAI 2022) and Neural Parameter Calibration for Finite-State Mean Field Games (Thöni et al., 2026). Although these works address different settings, I believe a clearer comparison would help establish the distinctive contribution of BE-MFG beyond parameter-conditioned equilibrium learning.

  2. Theoretical guarantees. I agree that Doob's theorem provides a general posterior consistency result under appropriate assumptions. However, this concerns the exact Bayesian posterior, whereas the proposed method relies on approximate equilibria and learned likelihoods. The connection between these two remains unclear. In particular, while Assumption 2 may not be necessary to run the numerical pipeline. I would appreciate clarification of how the continuous density estimator relates to the exact conditional observation laws when these laws are potentially atomic.

  3. Computational efficiency. I acknowledge that Appendix J demonstrates the computational advantage of amortizing equilibrium solves compared with grid search. My concern is specifically about the end-to-end inference pipeline, including policy distillation and conditional density estimation, which are excluded from the reported timing comparisons. Since computational efficiency is a central contribution, clarifying these additional costs and the circumstances under which amortization becomes beneficial would strengthen the evidence.

Overall, I appreciate the usefulness of the proposed formulation, but I remain concerned about the extent of its methodological novelty and the connection between the theoretical formulation and the practical algorithm. I therefore raised my score to 2.

Reviewer_xkuiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies parameter inference for finite-state MFGs with common noise. An observer sees N realizations of the MF trajectory at equilibrium and wants the parameter theta of the dynamics and/or reward. Evaluating the likelihood at theta would require computing the NE of M_theta so the authors make theta a public type (so that fictitious play gives eq policies for all theta). Trajectories simulated from that policy train a MAF as a surrogate likelihood which then yield MAP and BMA estimates.

Critical strengths
  • I think the structure of the solution method generally fits the problem. The computation of each likelihood is an equilibrium problem so learning the map from theta to pi^* so it can be used (read from) at inference time seems logical

  • Appendix J contain a lot of good information (comparison in Table 8 and the parameter recovery results in Table 10); I do think that this should be moved to the main body of the paper though given its strength

  • I like the formalization of consistency and identifiability under common noise

Critical weaknesses
  • As far as I can tell, theta is drawn once at the beginning of the game and observed by everyone. Lemmas B.3 and B.4 show that the mean-field always factors, meaning that agents of a given type theta don’t interact with any other type. My concern is that this significantly waters down the game theoretic framing of the problem. Also, from a theoretical perspective, this decoupling means that the BE-MFG is a mixture over M_theta (without any coupling between them). The only benefit of this setup is the parameter sharing argument (stated in the “Dimension of the BE-MFG” section in the appendix) but this is the master-policy idea of Perrin et al. (2022). I see that the authors already cite Perrin but only for the Beach Bar example and for the cost of the computation of the Nash eq. Some other relevant work also appears to be missing (step 2 of Alg. 7 is the neural likelihood estimation from Papamakarios, Sterrat, Murray (2019)).

  • The statement from the paper “uniquely bypasses the inner-loop equilibrium computation” (lines 113-116) is not accurate. Table 8 shows that the BE-MFG solution is just as expensive as PMFG which means that the approach is just one expensive computation rather than many smaller ones (i.e., not a bypass).

  • The name “Bayesian-extended” is a bit misleading since the agents in the game don’t appear to do anything Bayesian (theta is drawn once and observed by everyone so there is no belief over theta)

  • I don’t think the claim in Section 5.3 of accurate reconstruction of the “shape of theta^e … with remarkably low variance across seeds” is accurate. The congestion error STDs are about the same size as the means in many of the cells (compare 13.8 with 15.2 for Bullseye at eta=0.3)

  • The authors need to be a bit more clear about the disagreement between the theory and experiments. Theorem 3.3 requires that the common noise is recoverable from the observation but the Beach Bar variant in the paper seems to have the bar permanently open (with a random shift as noise). I don’t believe any of the three synthetic environments satisfy theorems 3.3 or 3.6. This disagreement is fine but it should be stated more explicitly.

Questions for authors
  • The claim that MAP beats BMA “across all settings” and attributing this to the learned densities being “insufficiently sharp” needs more quantification. My main reason for this is that just because BMA underperforms MAP does not necessarily mean that it is miscalibrated (an average of eq policies across theta is probably not an equilibrium or any of them)

  • How does the inference error vary with BE-NE’s exploitability? I’m also not sure what the BE-NE’s exploitability is at each theta_0 (is this reported?). Can you confirm that some ground-truth observations come from solutions of Appendix C.2?

2 comments

We thank the reviewer for the detailed and constructive review, for finding that the structure of the method fits the problem, and for appreciating the formalization of consistency and identifiability under common noise. We also agree that Appendix J deserves more visibility: if the paper is accepted, we will use the additional page allowed for the camera-ready version to move it to the main body.

We detail each point below.

W1:

Thank you for this comment.

  • Since Θ\Theta is drawn once and shared by all the agents, the population never mixes agents of different parameters. This is what allows a single BE-NE to induce a NE of Mθ\mathcal M_\theta for every θ\theta (Theorem 4.2), and hence to simulate the law Pθ,I\mathbb P_{\theta,\mathcal I}. If agents of different parameters interacted, the conditional game would no longer be Mθ\mathcal M_\theta, and its equilibrium would not give the laws needed for the inference.
  • No agent ever plays the BE-MFG. The data are generated by a population playing a NE of Mθ0\mathcal M_{\theta_0}, which is a genuine MFG with common noise, and the game-theoretic content of the problem is in each Mθ\mathcal M_\theta. The BE-MFG is used offline, by the observer, to compute all these equilibria at once.
  • On Master policies (Perrin et al., 2022).. Perrin et al. (2022) learn a master policy as a function of the current mean field distribution (see their Section~4), which yields a policy that can generalize across the initial conditions of a fixed game. The BE-MFG learns a policy conditioned on a parameter drawn once at the start, to the parameters of the model, which may enter both the reward and the dynamics. The only common point is the idea of extending the state space. Our contributions are not the parameter sharing itself, but (i) the formalization of the augmented game and its link with the family of PMFGs (Theorem 4.2 and the existence result), (ii) its use as a simulator of the law of the observations for the inverse problem under common noise, and (iii) the consistency and identifiability analysis. Note also that their main theoretical contribution (Theorem 1) differs from ours: they do not prove that a Nash equilibrium of the augmented game induces an equilibrium of the original game for each conditioning initial distribution. We prove the corresponding statement for our augmented game (Theorem 4.2, almost surely). We will add this discussion to the related work.
  • Thank you for noticing that step 2 of Algorithm 7 is neural likelihood estimation (Papamakarios, Sterratt and Murray, 2019). We will cite it there. This omission was not intended.

W2:

Thank you for this comment, which shows that the sentence needs to be clarified. By "inner loop", we mean the equilibrium computations performed during the inference, for each candidate parameter. Methods such as MF-IRL, or a grid search, compute a NE of Mθ\mathcal M_\theta for every θ\theta they evaluate. In the BE method of our pipeline, the neural network is trained only once and then can be quickly evaluated for any θ\theta.

Table 8 shows exactly this: the single BE-MFG solve costs about as much as one dedicated PMFG solve. A grid of 88 nodes on Beach Bar costs 3,3803{,}380 s, against 1,082±5901{,}082 \pm 590 s for the BE-MFG, and on District, a grid with three points per axis would require 310=59,0493^{10} = 59{,}049 dedicated solves. So our method avoids repeatedly computing equilibria.

We will rephrase the sentence as: "our method replaces the equilibrium computations of the inner loop by a single offline solve, whose cost is comparable to that of one dedicated solve".

W3:

We agree. The word "Bayesian" was meant to refer to the prior qq from which the parameter is drawn, and to the Bayesian inference that the augmented game enables, not to any belief of the agents, who observe the parameter. We propose to rename it as Parametric-Extended Mean Field Game (PE-MFG), and its equilibrium as PE-NE, throughout the paper.

W4:

We agree, thank you for checking. The claim of low variance is not supported for the congestion parameter. The sentence was meant for the attraction profile θe\theta^e. We will make that accurate in the corrected manuscript.

W5:

Theorems 3.3 and 3.6 give sufficient conditions under which the inverse problem is well posed. Among our experiments, only the electricity model of App. I satisfies the assumptions of Theorem 3.3. The inference pipeline itself (Theorem 4.2 and the posterior) does not rely on these theorems.

This was deliberate: our objective was to design environments and experiments that go beyond the theory we establish, in order to test whether the parameter can be recovered when the sufficient conditions are not met. We will add a paragraph at the beginning of Section 5 that states, for each environment, which assumptions hold.

Q1:

Thank you for this question. Our point is different.

The BMA policy is the natural Bayesian way to account for the uncertainty on the parameter: instead of committing to a single estimate, it averages the equilibria over the posterior. One could expect this hedging to make the downstream task more robust. Our experiments show that, in our settings, it actually degrades the downstream task of the inference, namely finding a policy with low exploitability or good performance against the expert, compared with plugging in the MAP.

Q2:

  • Ground truth. Yes, we confirm it: in all experiments, the observations are generated by dedicated equilibria of Mθ0\mathcal M_{\theta_0}, never by the BE-NE.
  • Exploitability of the BE-NE at each θ\theta. It is reported in App. J. Table 8 gives, for each θ\theta of Beach Bar, the number of FP rounds needed by the BE-MFG to reach 5%5\% relative exploitability at θ\theta. Table 9 gives the relative exploitability of the BE-NE at each District scenario at the end of the FP budget.
  • Inference error as a function of the exploitability of the BE-NE. This is a very good question, and we ran the experiment on Beach Bar. We reuse the Bayesian FP runs of App. J (5 seeds). For a snapshot kk, we take the FP average of the rounds 0,…,k0, \dots, k, measure its relative exploitability (averaged over θ∈{0.5,1,1.5,2}\theta \in \{0.5, 1, 1.5, 2\}, as in App. J), distill it into one Bayesian policy, train the ensemble of flows on it with the settings of the paper, and infer θ0\theta_0 from N=10N = 10 observations of each of the 100100 dedicated ground-truth equilibria of App. J, with θ0∈[0.5,2]\theta_0 \in [0.5, 2]. Mean ±\pm std over the 5 seeds:
FP round kkRelative exploitability of the BE policyMean error ∣θ^−θ0∣\lvert \hat\theta - \theta_0 \rvert (N=10N = 10)
124.0±0.6%24.0 \pm 0.6\%1.10±0.071.10 \pm 0.07
211.8±0.6%11.8 \pm 0.6\%0.25±0.030.25 \pm 0.03
310.9±0.4%10.9 \pm 0.4\%0.23±0.030.23 \pm 0.03
54.2±0.3%4.2 \pm 0.3\%0.09±0.010.09 \pm 0.01
106.2±1.0%6.2 \pm 1.0\%0.15±0.040.15 \pm 0.04
203.8±1.2%3.8 \pm 1.2\%0.11±0.020.11 \pm 0.02

The inference error closely follows the exploitability of the BE policy. At round 1, the policy carries no information on θ\theta and the inferred θ\theta does not depend on θ0\theta_0. Around 11%11\% relative exploitability, θ0\theta_0 is recovered with a bias. Below 5%5\%, the error reaches its final level. The error is not monotone in kk but in the exploitability. We will add this experiment to our manuscript, and we believe it strenght its results.

We hope these answers address the reviewer's concerns and that the reviewer will consider raising the score; we would be happy to clarify anything that remains unclear during the discussion period.

Reviewer_VAacOpenReview ↗
Rating
3(+1)
Confidence
4(+2)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the inverse problem of recovering unknown parameters in MFGs with common noise from partial observations of population trajectories. The authors propose the Bayesian-Extended Mean Field Game (BE-MFG), which embeds the parameter space into an augmented game so that a single equilibrium solve provides equilibrium policies across parameter values, avoiding repeated equilibrium computation during Bayesian inference. The paper further examines parameter identifiability and establishes a theoretical link between BE-MFG equilibria and the underlying parametric MFG equilibria. The proposed pipeline combines Fictitious Play, FiLM, and Masked Autoregressive Flows, and is evaluated on three synthetic environments and real electricity market data.

Critical strengths
  1. The BE-MFG formulation is an interesting theoretical contribution. In particular, Theorem 4.2 establishes a connection between the equilibrium of the augmented game and those of the underlying parameterized MFGs, thereby providing a mathematical foundation for solving a family of equilibrium problems within a unified framework.

  2. The paper is well organized, with detailed mathematical formulations and proofs. The experiments cover different parameter dimensions and observation settings, and the results demonstrate the potential of the proposed approach for parameter inference under common noise.

Critical weaknesses

My main concern is the theoretical foundation of the proposed BE-MFG equilibrium. Although Theorem 4.2 establishes an equilibrium correspondence, it assumes that a BE-MFG Nash equilibrium already exists. I could not find an explicit existence result for the BE-MFG under the stated assumptions. Assumption 1 concerns the uniqueness of equilibrium-induced laws in the underlying PMFGs, but it does not establish equilibrium existence. Since the proposed framework fundamentally relies on computing a BE-MFG equilibrium, its existence conditions should be established more rigorously, or at least explicitly justified by citing appropriate existence results. This point is particularly important because the augmented state space includes the continuous parameter space Θˉ\bar{\Theta}, so existence cannot be directly imported from standard finite-state MFG results without further clarification.

Questions for authors
  1. Existence and uniqueness: Could the authors establish sufficient conditions for the existence of a BE-MFG Nash equilibrium, rather than assuming it in Theorem 4.2? It would also be helpful to clarify whether uniqueness is required at the policy level or only at the level of equilibrium-induced laws, and whether the qq-a.s. statement under condition (C2) is sufficient for the inference procedure.

  2. Practical significance: The BE-MFG construction assumes that the sampled parameter is fixed and commonly observed by all agents. Is this intended primarily as a computational device for Bayesian inference, or as a behavioral assumption about agents? Clarifying this distinction would help assess the practical significance of the formulation, especially in real-world settings where agents may have heterogeneous or unknown preferences.

  3. Scalability: The experiments primarily operate on population-level mean-field distributions in finite state and action spaces. How well does the proposed method scale to continuous or substantially larger state and action spaces, higher-dimensional parameter spaces, and finite-population multi-agent simulations? A discussion of approximation error and sample complexity would further strengthen the practical relevance of the approach.

3 comments

We thank the reviewer for the careful reading of the paper, for recognizing the BE-MFG formulation as an interesting theoretical contribution, and for the precise questions, which helped us strengthen the paper. In summary:

  1. Existence of a BE-NE. We prove that a BE-NE exists as soon as every Mθ\mathcal M_\theta admits a NE.
  2. Scalability. We add a new experiment in continuous state and action spaces (R3\mathbb R^3), with a 9-dimensional parameter and observations of finite populations of 10 to 1000 agents.

We detail each point below.

W1:

You are right that Theorem 4.2 takes the existence of a BE-NE as given. To fix this gap, we propose the following theorem:

Theorem. Assume every Mθ\mathcal M_\theta, θ∈Θˉ\theta\in\bar\Theta, admits a NE, and that either (C1) holds or (C3) the common noise space E0\mathcal E^0 is finite. Then Mqˉ\bar{\mathcal M_q} admits a BE-NE.

Proof:

Reduction to a measurable selection. Let (θ,i,x,ρ)↦πθ,i(a∣x,ρ)(\theta,i,x,\rho)\mapsto\pi_{\theta,i}(a\mid x,\rho) be jointly measurable, with πθ\pi_\theta a NE of Mθ\mathcal M_\theta for qq-a.e. θ\theta. Set πˉi(a∣θ,x,ρˉ)=πθ,i(a∣x,ρˉ(Θˉ×⋅))\bar\pi_i(a\mid\theta,x,\bar\rho)=\pi_{\theta,i}(a\mid x,\bar\rho(\bar\Theta\times\cdot)). Then πˉ\bar\pi is clearly a BE-NE. The goal is therefore only to choose a NE πθ\pi_\theta for each θ\theta in a jointly measurable way.

  • Under (C1) this is trivial, since every map on a countable discrete Θˉ\bar\Theta is measurable.
  • Under (C3) we use the proof below.

Proof under (C3).

1. Tree encoding.

  • Histories are Hi=(E0)i\mathcal H_i=(\mathcal E^0)^i, with ν(h)=∏kν0(ek)\nu(h)=\prod_k\nu^0(e_k).
  • A policy is encoded by σ=(σi(h))\sigma=(\sigma_i(h)), indexed by ii and h∈Hih\in\mathcal H_i, which lives in the compact metric space Y=∏i,h(ΔA)XY=\prod_{i,h}(\Delta_\mathcal A)^\mathcal X.
  • The tree flow is ρθ,0σ=ρ0\rho^\sigma_{\theta,0}=\rho_0 and ρθ,i+1σ(h,e)=∑x,aρθ,iσ(h)(x)σi(h)(a∣x)Pθ(⋅∣x,a,ρθ,iσ(h),e)\rho^\sigma_{\theta,i+1}(h,e)=\sum_{x,a}\rho^\sigma_{\theta,i}(h)(x)\sigma_i(h)(a\mid x)P_\theta(\cdot\mid x,a,\rho^\sigma_{\theta,i}(h),e).
  • Each coordinate of the flow is Borel in (θ,σ)(\theta,\sigma).

2. Realizability. A Markov policy plays the same strategy at nodes with equal mean fields. Define

R={(θ,σ):ρθ,iσ(h)=ρθ,iσ(h′)⇒σi(h)=σi(h′)∀i,h,h′}.R=\lbrace(\theta,\sigma):\quad \rho^\sigma_{\theta,i}(h)=\rho^\sigma_{\theta,i}(h')\Rightarrow\sigma_i(h)=\sigma_i(h')\quad \forall i,h,h'\rbrace.

RR is a finite intersection of sets of the form (open) ∪\cup (closed) in the Borel variables, hence Borel.

  • Every π∈Π\pi\in\Pi induces σθ,iπ(h)=πi(⋅∣⋅,ρθ,iπ(h))\sigma^\pi_{\theta,i}(h)=\pi_i(\cdot\mid\cdot,\rho^\pi_{\theta,i}(h)), with (θ,σθπ)∈R(\theta,\sigma^\pi_\theta)\in R and the same flow.
  • Conversely, (θ,σ)∈R(\theta,\sigma)\in R defines a policy by reading σi(h)\sigma_i(h) at ρ=ρθ,iσ(h)\rho=\rho^\sigma_{\theta,i}(h), and the uniform strategy elsewhere.

3. Equilibrium graph. Define the on-policy value

Jθ(σ)=∑i,hν(h)∑x,aρθ,iσ(h)(x)σi(h)(a∣x)rθ(x,a,ρθ,iσ(h)),J_\theta(\sigma)=\sum_{i,h}\nu(h)\sum_{x,a}\rho^\sigma_{\theta,i}(h)(x)\sigma_i(h)(a\mid x)r_\theta(x,a,\rho^\sigma_{\theta,i}(h)),

and let Jθ⋆(σ)=∑xρ0(x)u0(∅,x)J^\star_\theta(\sigma)=\sum_x\rho_0(x)u_0(\emptyset,x), where uu solves the backward recursion

ui(h,x)=max⁡a[rθ(x,a,ρi(h))+∑eν0(e)∑x′Pθ(x′∣x,a,ρi(h),e)ui+1((h,e),x′)],u_i(h,x)=\max_a\Big[r_\theta(x,a,\rho_i(h))+\sum_{e}\nu^0(e)\sum_{x'}P_\theta(x'\mid x,a,\rho_i(h),e)u_{i+1}((h,e),x')\Big],

with uH≡0u_H\equiv0. Both JJ and J⋆J^\star are Borel.

On RR, equal mean fields at a node imply identical subtrees, so ui(h,⋅)u_i(h,\cdot) depends on hh only through ρi(h)\rho_i(h). Hence Jθ⋆(σθπ)=max⁡π′Vθ(π′;π)J^\star_\theta(\sigma^\pi_\theta)=\max_{\pi'}V_\theta(\pi';\pi). Also Jθ(σθπ)=Vθ(π;π)J_\theta(\sigma^\pi_\theta)=V_\theta(\pi;\pi). Therefore

π is a NE of Mθ  ⟺  (θ,σθπ)∈G={(θ,σ)∈R:Jθ(σ)=Jθ⋆(σ)}.\pi \text{ is a NE of } \mathcal M_\theta \iff (\theta,\sigma^\pi_\theta)\in G=\lbrace(\theta,\sigma)\in R:\quad J_\theta(\sigma)=J^\star_\theta(\sigma)\rbrace.

GG is Borel, and each section GθG_\theta is nonempty by assumption.

4. Selection.

  • By the Jankov--von Neumann theorem [Bertsekas and Shreve, Stochastic Optimal Control: The Discrete-Time Case, Proposition 7.49, p.189], there is a universally measurable s:Θˉ→Ys:\bar\Theta\to Y with (θ,s(θ))∈G(\theta,s(\theta))\in G for every θ\theta.
  • YY is a compact subset of a Euclidean space. Applying [Bertsekas and Shreve, Proposition 7.27, p.173] gives a Borel map s~:Θˉ→Y\tilde s: \bar \Theta \to Y equal to ss qq-a.e. Write σ~θ=s~(θ)\tilde \sigma^\theta = \tilde s(\theta) and ρ~θ,i\tilde \rho_{\theta, i} for its tree flow.
  • Enumerate Hi={h1,…,hK}\mathcal H_i=\lbrace h_1,\dots,h_K\rbrace. Set πθ,i(⋅∣x,ρ)=σ~iθ(hk)(⋅∣x)\pi_{\theta,i}(\cdot\mid x,\rho)=\tilde\sigma^\theta_i(h_k)(\cdot\mid x) for the first kk with ρ=ρ~θ,i(hk)\rho=\tilde\rho_{\theta,i}(h_k), and the uniform strategy if there is none.
  • This defines π\pi on a finite Borel partition of (θ,ρ)(\theta,\rho), so it is jointly Borel, and πθ\pi_\theta is a NE for qq-a.e. θ\theta.

The reduction above then yields a BE-NE. ■\blacksquare.

Q1:

For the existence and uniqueness, please see our answer to W1. The inference procedure does not require in practice any theoretical results, and can always be implemented. The qq-a.s. statement has to be seen as a theoretical justification for our method.

Q2:

Thank you for this question, which lets us clarify an important point. The BE-MFG is purely a computational device for the inference. It is not a behavioral assumption, and it does not describe any real-world system.

  • Since Θ\Theta is drawn once and shared by all the agents, the population never mixes agents of different parameters. This is what allows a single BE-NE to induce a NE of Mθ\mathcal M_\theta for every θ\theta (Theorem 4.2, and its converse in our answer to W1), and hence to simulate the law Pθ,I\mathbb P_{\theta,\mathcal I}. If the parameter changed over time, or were private to each agent, the conditional game would no longer be Mθ\mathcal M_\theta, and its equilibrium would not give the laws needed for the inference.
  • No agent ever plays the BE-MFG. The data are generated by a population playing a NE of Mθ0\mathcal M_{\theta_0}, for the true parameter θ0\theta_0. The BE-MFG is only used offline, by the observer.
  • A model in which each agent samples its own parameter would be a different forward model, whose equilibria are not those of the family {Mθ}\lbrace \mathcal M_\theta \rbrace. This is an interesting direction, but it concerns another problem.
  • MFGs can describe agents with heterogeneous preferences either through multi-population MFGs, or by assuming that each agent’s parameter is drawn independently of the parameters of the other agents. In the first case, our method can be applied directly to learn the parameter of each population. In the second case, our method can be used to learn a description of the distribution of the parameter, such as its moments.
  • You also point out that, in practice, the agents’ parameters may be unknown. Our work directly addresses this challenge, since we show how to learn the unknown parameter.

Q3:

Thank you for this question. To address the continuous state and action spaces as well as the discussion on sample complexity, we propose a new experiment. This new experiment has a continuous state and action spaces of dimension 33, and a parameter of dimension 99, that is generally consider as high dimensional for such setting (note that the parameter dimension in the district example of the paper is 1010). We believe that the results strength the paper, and propose to add them to the manuscript. Overall, the experiment shows that the method is scalable to continuous spaces, and that the inferred parameter of the method remains accurate when the observations come from finite populations as a few as 1010 agents, with the error decreasing as the number of observed agents grows.

New experiment: a 3-dimensional MFG with common noise and a 99-dimensional parameter.

  • Model. $ \mathcal{X} = \mathcal{A} = \mathbb R^3$, T=20T = 20 time steps. Each agent follows Xt+1=Xt+at+σεt+σ0εt0,X_{t+1} = X_t + a_t + \sigma \varepsilon_t + \sigma_0 \varepsilon^0_t, where εt∼N(0,I3)\varepsilon_t \sim \mathcal N(0, I_3) is idiosyncratic, εt0∼N(0,I3)\varepsilon^0_t \sim \mathcal N(0, I_3) is common to all agents, σ=σ0=0.2\sigma = \sigma_0 = 0.2, and X0∼N(m0,0.25 I3)X_0 \sim \mathcal N(m_0, 0.25\, I_3). The reward is r(x,a,xˉ)=−ca2∣a∣2−12(x−xˉ)⊤Θ1(x−xˉ)−12∑k=13θ2,k xk2,r(x, a, \bar x) = -\frac{c_a}{2} |a|^2 - \frac12 (x - \bar x)^\top \Theta_1 (x - \bar x) - \frac12 \sum_{k=1}^{3} \theta_{2,k}\, x_k^2,, where xˉ\bar x is the mean of the population and ca=5c_a = 5. The agents are attracted to the mean of the population, and to the origin. The unknown parameter is θ=(Θ1,θ2)\theta = (\Theta_1, \theta_2), of dimension 6+3=96 + 3 = 9, with a uniform prior: diagonal of Θ1\Theta_1 in [1,5][1, 5], off-diagonal terms in [−0.5,0.5][-0.5, 0.5], θ2∈[0,2]3\theta_2 \in [0, 2]^3.
  • Mean field. The state space is continuous, so the mean field is represented by its mean and covariance (9 features), which are the input of the policies together with (t,x)(t, x). The populations are simulated with particles.
  • Equilibria. The BE-MFG and the dedicated PMFGs are all solved by fictitious play (120 iterations).
  • Observations: finite populations. The data are generated by the dedicated PMFG at the true parameter θ0\theta_0. We consider two values of number of observations N∈{10,100}N\in \{10, 100\}, made of Nagents∈{10,100,1000}N_{\text{agents}} \in \{10, 100, 1000\} trajectories of agents, and observe only the empirical mean and covariance of these agents at each time step. The observation is therefore a noisy, finite-population version of the mean field.
  • Protocol. 6 test values θ0\theta_0, 5 seeds. The error is ∣θ^−θ0∣|\hat\theta - \theta_0| divided by the width of the prior, averaged over the 9 coordinates.
NagentsN_{\text{agents}}NNParameter error (%)Performance vs expert (%)Exploitability (%)
101013.6±1.913.6 \pm 1.9−0.93±0.21-0.93 \pm 0.219.0±0.19.0 \pm 0.1
101008.0±0.78.0 \pm 0.7−0.02±0.05-0.02 \pm 0.058.4±0.18.4 \pm 0.1
100108.4±1.38.4 \pm 1.3−0.57±0.19-0.57 \pm 0.198.7±0.08.7 \pm 0.0
1001004.6±0.44.6 \pm 0.4−0.00±0.03-0.00 \pm 0.038.4±0.18.4 \pm 0.1
1000105.0±0.65.0 \pm 0.6−0.27±0.11-0.27 \pm 0.118.5±0.18.5 \pm 0.1
10001002.6±0.22.6 \pm 0.2−0.03±0.02-0.03 \pm 0.028.2±0.18.2 \pm 0.1

(Result of the BMA method). The 9 parameters are recovered and the error decreases steadily both with the number of observed episodes NN and with the number of observed agents, down to 2.6%2.6\% of the prior range. A single BE-MFG solve covers the whole 9-dimensional parameter space, whereas a grid search with only three points per axis would require 393^9.

We hope we answered your questions and we would be grateful if you considered raising your score. Otherwise, we will be happy to answer any follow-up questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the discussion and main revisions

We sincerely thank all reviewers for their careful reviews. Each of them has led to new experiments or new theoretical results, and we hope we have addressed all their concerns. We summarize the discussion below. All the associated changes appear in green in the revised version.

Naming and numbering.

Following Reviewer xkui (W3), the Bayesian-Extended MFG (BE-MFG) is renamed the Parameter-Extended MFG (PE-MFG), and its equilibrium the PE-NE. Since new results were added in Sec. 4, Theorem 4.2 of the submission is now Theorem 4.4.

Theory

  • Existence of a PE-NE (VAac, W1/Q1). New Theorem 4.5: if every Mθ\mathcal{M}_\theta admits a NE, and either Θˉ\bar\Theta is countable or the common noise space is finite, then the PE-MFG admits a PE-NE.
  • Statistical guarantees (Kvhg, W2/Q3). New paragraph in Sec. 4.1 on posterior consistency (Doob), consistency of the MAP (Wald) and misspecification (Kleijn & van der Vaart). New Proposition 4.3, bounds the effect of the density estimation error on the posterior and on the MAP.

Experiments

  • Continuous spaces, high-dimensional parameter, finite populations (VAac, Q3). New Experiment 5 (App. K, summarized in Sec. 5.5): state and action in R3\mathbb{R}^3, a 9-dimensional parameter, and observations of finite populations of 10 to 1000 agents. The error decreases with the number of observations and of agents, down to 2.6% of the prior range.
  • Inference error vs. exploitability of the PE-NE (xkui, Q2; Kvhg). New experiment in App. J (Table 12): the inference error follows the exploitability of the PE policy.
  • End-to-end computational cost (Kvhg, W3/Q4). New table in App. J (Table 11), including policy distillation and density estimation.
  • Comparison with a grid search (xkui, strengths). Moved from App. J to the main text: new Sec. 5.3, Tables 1–2.

We regret that the discussion could not go further during the discussion period, as several of our answers have not received a reply yet. We believe that the answers and the revised version address the main concerns raised in the reviews, and we hope the reviewers will be able to read them, even after the discussion period. If they find that their concerns have been addressed, we would be grateful if they could consider updating their scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.