- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper empirically studies what sort of Nash equilibria do popular families of zero-sum game solvers tend to converge towards. The paper considers popular last-iterate and regret averaging algorithms applied to normal form (matrix) and extensive form zero-sum games with known Nash solution sets, and compares the behavior of the converged methods, measuring which tend to yield higher entropy outcomes. The main conclusions of the paper through empirical demonstrations are that (i) for matrix games, regularized nash dynamics tend to yield Nash solutions with max entropy and regret averaging methods like CFR/CFR+ yield lower entropy Nash solutions, and (ii) on extensive form game of Kuhn poker this trend does not hold.
Critical strengths
- The premise of the paper is interesting and if actually answered, could have informed practitioners on choosing solver algorithms.
Critical weaknesses
- Paper results are unconvincing: The biggest weakness is that the conclusions of the paper are very chaotic and the only conclusion that can be certainly claimed is rather unsurprising/not new. I do not understand how the fact that regularized Nash dynamics yield higher entropy Nash solutions for matrix games is new knowledge for the community. Further, for regret averaging methods like CFR/CFR+, the paper could not draw concrete conclusions in the first place, especially for the extensive form setting, where CFR/CFR+ excel/are mostly used anyway.
- Paper presentation resembles AI slop: It is very evident that large sections of the paper have been taken verbatim from an AI model and with due respect, resemble AI slop (several instances of the theme of "doing/claiming X not Y", sentences that dont add any value and are repetitions, etc.). Further, many concepts are arbitrarily stated without any explanation/definition (for example, reach-weighted formulations are mentioned as something which could help answer Conjecture 1, but never even defined).
Questions for authors
- Do the authors have any solid intuition on why the behavior of CFR/CFR+ flips (from yielding low entropy solutions in matrix games to high ones in the extensive form Kuhn poker game)? I find the discussion in the paper inadequate.
- Why did the authors only choose one extensive form game?
2 comments
Thank you for your response, I have no more questions and will finalize my score after discussion with other reviewers/AC.
Response to Reviewer j6Jt
thank you for the review. We answer your points in order.
"The only conclusion that can be certainly claimed is rather unsurprising / not new."
You are right that the principle is known, and the submitted version did not make clear what was new. The revision changes this in two ways.
"Conclusions are very chaotic."
The abstract, contributions, discussion and conclusion are rewritten around the theorem. Section 6 now lists what is open and the limitations.
"For CFR/CFR+ the paper could not draw concrete conclusions, especially for the extensive-form setting."
That is accurate, and we say so. What the paper establishes for regret averaging is limited to matrix games: CFR+ lands on lower-entropy members in 153 of the 162 ensemble games where both solvers converge (94%), the gap widens with more iterations, and the max(R,0) projection is not the cause. The theorem explains why averaging methods can differ: averages leave the set on which the argument works (Section 5). Why they land where they do is listed as open.
Q1. "Why does the behaviour of CFR/CFR+ flip between matrix games and Kuhn?"
Most of the flip was an implementation effect, which the submission reported in its appendix and which we should have put first. Our engine's CFR divides counterfactual values by the opponent's reach and averages without reach weights. In matrix games this is identical to textbook CFR; in Kuhn it is not. Textbook CFR and CFR+ land at bluff 0.203 and 0.242 in Kuhn, at or above the max-entropy member (0.201); the low values (0.005, 0.068) belong to the reach-normalized variant (Section 4.1, Table 3). We do not have a mechanism for where textbook CFR lands, and we do not claim one.
Q2. "Why only one extensive-form game?"
Kuhn was chosen because its equilibrium set is known in closed form. We added a second, minimal example (Appendix J): asym_safe written as a two-stage game with identical payoffs. R-NaD's selection moves from P(Play) = 0.654 in the matrix form to 0.494 in the two-stage form, so in extensive form the selection depends on how the game is written. Two small games do not support a general claim about sequential games, and the paper says so.
"Reach-weighted formulations are mentioned but never defined."
Corrected: Section 2 now defines it (the entropy at each information set is weighted by the probability that the player's own actions reach it).
Presentation and the ethics flag.
The submission contains the AI-use statement that ICLR requires, describing which tools were used and for what. It has been updated for the revision: it states that AI tools helped draft the new proofs and run the new experiments, and that the proofs were checked line by line by the authors. The results do not depend on trust in the text: every number is regenerated by the supplementary notebook, and two reviewers independently reproduced our update rule and its outputs to seven digits. We have rewritten the abstract, contributions, discussion and conclusion, and revised the writing of the main text: long sentences are split, the "X, not Y" constructions you pointed to are removed where they carried no content, a recap paragraph that repeated the abstract and the conclusion is deleted, and Section 5 introduces each formal statement in plain words before stating it.