- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies symmetry reductions in finite discounted Markov games. State symmetries produce a quotient game. Agent permutations produce one shared table over pointed-state orbits. Each entry records the player's own local state and the multiset of the other players' states. The two reductions can be combined when their group actions commute. The paper transfers existing convergence bounds to state quotients. It also gives a finite-time bound for shared log-barrier softmax policy gradient. With a generative model, it pools symmetric transition rows. Experiments test the benefits in cooperative and zero-sum tabular games.
Critical strengths
- The pointed-state and stabilizer construction keeps the deviator's identity explicit. It explains why agent sharing differs from a state quotient of the original game.
- Corollary 1 studies finite-time convergence with tied parameters. It exposes both orbit counts and sharing multiplicity, with an explicit condition for improving the bound.
- The experiments measure the true unrestricted Nash gap. The discussion also separates row savings, per-row estimation costs, and planning costs.
Critical weaknesses
The account of prior theory needs more care. The paper acknowledges that the reductions are classical. However, Section 1.1 places some earlier work under priors and empirical validation. This can understate its theory. van der Pol et al. (2022) already handle global symmetries and agent permutations together. Theorem 4.4 of Chen and Zhang (2024) gives symmetric-value and optimal-policy guarantees for cooperative Markov games. Theorem 4.0.2 of Emmons et al. (2022) connects symmetric local optima to Nash equilibria in common-payoff normal-form games. Their Section 4.1 also discusses shared-policy MARL. Please focus the distinction on policy classes, deviation classes, exact per-policy gap identities, and finite-time bounds. These precedents do not establish redundancy. They do show why the gap cannot be described broadly as a lack of prior theory or equilibrium guarantees. The best-response argument should also be linked more directly to classical results such as Zinkevich and Balch (2001).
Questions for authors
none
2 comments
Thank you for the clarification and revisions. The more careful treatment of prior theory and the clearer distinction of your contributions address my concerns. I maintain my weak acceptance recommendation.
Rebuttal
A more careful account of prior theory
We thank the reviewer for identifying where our discussion understated prior theory. Our revision has incorporated the suggestions with highlighted text, no longer presenting the following prior works as mainly empirical uses of symmetry: