- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper relaxes the two assumptions in the classical mean-field games: the homogeneity of agents and the stochasticity of their policies. Instead, the authors assume that the agents can take heterogeneous deterministic policies and theoretically prove the equivalence of the equilibria between the classical MFG system and the one established in this paper. Then, the authors establish the novel algorithms based on this new perspective and integrate RL to handle the large state and action spaces. Experiments demonstrate that the new system shows the same macro behavior as the classical MFG system while allowing heterogeneous individual policies for each agent.
Critical strengths
-
The motivation of this paper is clear and sensible. The authors clearly explain how classical assumptions limit the applicability of classical MFGs. Allowing the agents to have heterogeneous individual policies has practical implications.
-
The authors provide theoretical contributions by proving the equivalence of the equilibria between the classical MFG system and the one established in this paper, which is sound and provides a fundamental understanding of the field of MFGs.
-
The experiments also demonstrate the effectiveness of the established system.
-
Also, this paper is well-written and easy to follow, presenting a well-executed research plan with sound theoretical results and convincing experimental results.
Critical weaknesses
I see no significant weaknesses, but only one minor question: the agents' heterogeneity is defined in terms of their individual pure deterministic strategies, yet they appear to share the same population-level mixed strategy . Does that mean the agents are still somewhat homogeneous?
Questions for authors
See weaknesses.
Response to Reviewer PbPR
We thank the reviewer for the positive assessment and for the precise questions: the agents are heterogeneous in their individual pure strategies, yet they share the population-level mixed strategy σ. Does that mean they are still somewhat homogeneous?
We have two answers.
Part 1: In Section 3.2, agents are homogeneous ex ante. Ex ante, agents are exchangeable and draw from the same σ similar to standard MFG (however, see also Answer 2). Ex post, each agent holds one deterministic policy, so the population is heterogeneous. In a homogeneous behavioral MFG, every agent randomizes at every decision. Here, no individual agent randomizes at all and the randomization is only "ex ante" at the population level.
Part 2: Theorem 3 gives a deterministic finite-N profile without a shared behavioral policy. The shared σ is not necessary in the finite game of actual interest. For any finite N, Theorem 3 finally constructs a tuple of deterministic pure policies (π1,…,πN), one per agent (e.g., by proportional assignment from σ∗), that is an ϵ(N)-Nash equilibrium of the N-agent game in expectation, with ϵ(N)→0. Each agent has its own fixed deterministic policy, and no agent randomizes. The construction may assign the same pure policy to several agents. When σ∗ has more than one policy in its support, the population is heterogeneous. The population-level σ is only the limiting mean field solution used to choose the finite list. So the finite-N statement does not require a shared behavioral policy; it is deterministic ex post. Rock–Paper–Scissors makes this concrete: the unique MFNE is (1/3,1/3,1/3), but a homogeneous pure population of e.g. 100 agents is never an exact equilibrium, since 100 is not divisible by 3. But an approximate equilibrium exists with around 1/3 of agents playing each pure policy.
We have added the ex ante/ex post paragraph after the SMFNE definition in Section 3.2, including the sentence on the Theorem 3 finite-N construction, and the abstract now says "behavioral homogeneity (a single policy is shared by all agents)" to avoid the same ambiguity.