- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
MF-CAPA audits a supplied MFE claim (pi, L) on two axes: the response gap against L and the consistency of L with Gamma(pi). It returns CERTIFY, REFUTE or ABSTAIN with risk delta, using adaptive trajectory experiments. For a finite realizable model class, the paper shows that on-policy data cannot detect unplayed deviations, that auditing can be much cheaper than identification, and gives a change-of-measure lower bound and an anytime-valid mixture-likelihood Track-and-Stop. Experiments cover a nine-model routing MFG, small MFGLib games with neural policy-flow candidates, and LP-based bounders.
Critical strengths
Treating a learned equilibrium as a claim to be audited, with abstention, is a sensible framing. Separating response optimality from flow consistency is correct.
The statistical treatment is careful and the limitations are stated honestly.
Critical weaknesses
The theory is a specialization of known pure-exploration results, and the mean-field structure plays no role. The "broken equivalences" are simply the definition of an MFE as a fixed point. Prop. 4.1 is immediate. Prop. 4.2 follows because identification minimizes over a superset. Thm. 4.3 is Garivier-Kaufmann. Thm. 4.4 is Ville plus Track-and-Stop. Multiple-answer pure exploration (Degenne-Koolen) already covers "only the verdict must separate." Nothing in Section 4 uses the population coupling. The intervention study is explicitly population-independent, i.e., an MDP. Standard solvers induce L = Gamma(pi) by construction, so the flow axis does no work for them.
The useful setting is excluded and the tested setting does not need the method. The theory needs a finite, realizable model class. The experiments are known-model simulator audits, where G and C are computed exactly by dynamic programming, so no sequential audit is needed. With an unknown model, no guarantee is given. The environments are tiny (3 states, a 9x9 grid, 9 models). The 57x gain is against a strawman identity design. The gain over uniform allocation is about 2x, and over a fixed maximin design 1.03-1.10x for certification. All validated formal backends abstain.
Minor points: "CAPA" is never defined or cited; if it is the authors' prior work, this raises an anonymity question. The jargon is heavy and the citation style is nonstandard. No finite-population (N-player) analysis.
Overall: Correct but incremental. Known sequential-testing theory is relabeled for MFGs without using the mean-field coupling, and it is demonstrated in settings where the audited quantities are computable exactly.
Questions for authors
Give a realistic setting where G and C cannot be computed exactly, yet a finite realizable model class is available.
Which result in Section 4 uses mean-field structure?
Which widely used solvers declare an L that is not Gamma(pi)?
What happens under misspecification, when the true model is outside the class?