- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies how to compress large strategies computed by counterfactual regret minimization (CFR) into small, interpretable decision trees for imperfect information games. The authors show that conventional imitation based distillation can overlook strategically important information sets, where even small distributional errors may lead to substantial exploitability. To address this issue, they propose exploitability guided distillation, which directly minimizes the payoff achievable by a best-responding opponent instead of solely imitating the teacher's action distributions. The proposed method combines best response based leaf optimization with a tree growing procedure that aligns split scoring with the final optimization objective. The paper also provides a theoretical bound explaining the mismatch between imitation loss and exploitability.
Critical strengths
The paper addresses an important problem in strategy distillation for imperfect information games. When a complex strategy is compressed into a small decision tree, minimizing imitation loss may overlook information sets that occur infrequently but have substantial strategic consequences. As a result, a student strategy that closely imitates a strong teacher can still be highly exploitable. I find this observation meaningful, as it highlights a limitation of conventional imitation-based distillation.
In addition, the paper presents a well structured investigation, connecting the research motivation, theoretical analysis, algorithm design, and experimental evaluation. The authors provide a concrete counterexample and a theoretical bound to explain the limitations of imitation, then develop an exploitability guided approach for both tree construction and leaf optimization. The experiments include controlled comparisons and ablation studies that help identify the contributions of different components. Overall, I find the research framework coherent and the experimental analysis relatively thorough.
Critical weaknesses
Limited evaluation of performance when both players adopt the proposed strategies.
The proposed method directly minimizes NashConv, and its experimental evaluation primarily relies on the same metric. While NashConv is a valid measure of worst case exploitability, I am concerned about how well this evaluation reflects actual gameplay, particularly when both players adopt strategies produced by the proposed method. In this setting, neither player necessarily follows a best response to the other, so the worst case exploitation measured by NashConv may rarely be realized and thus other policies would perform better when its opponent use this policy. Consequently, lower exploitability does not necessarily imply higher expected returns when both players use the distilled strategies.}
Questions for authors
The proposed method optimizes NashConv, which measures exploitability against unrestricted best responses. However, I am interested in how the resulting strategies perform when both players initially adopt the proposed distilled strategies and one player subsequently deviates. For example, Table 3 suggests that the distilled strategy follows relatively predictable folding rules, which might be exploitable through simple adjustments in bluffing frequency.
Could the authors evaluate the actual expected returns under mutual deployment and under deviations restricted to similarly small decision trees? This would help clarify whether the resulting strategies remain robust against simple, practically executable deviations, rather than only characterizing their worst-case exploitability.
We thank the reviewer for the careful reading of our manuscript.
Weakness
To evaluate performance beyond best-response opponents, we now report the exact expected payoff in chips per hand (averaged over both seats and 8 seeds) of every arm against the CFR+ teacher and against a weaker CFR+ opponent (1000 iterations), on Leduc at B=8:
Against a strong opponent that is not best-responding, the aligned tree loses the least, and the three best-found arms lose less than the three imitation-level arms. Payoffs against the teacher and against the weaker opponent agree within 0.003 chips per hand, so the ordering does not depend on the opponent's strength. The picture holds across budgets: at B=16 the aligned tree (−0.154) is within 0.001 of the best arm, and at B=32 (−0.140) it is the best of all six (Appendix A). Deviations from mutual deployment are answered under the question below.
Question
Starting from mutual deployment of the student profile on Leduc (B=8), we measure one player's exact gain (chips per hand, medians over the 16 seed-seat cells) at four deviation levels: refitting one leaf of its own tree, refitting all leaves on the fixed structure, regrowing its own B-leaf tree against the fixed opponent, and the unrestricted best response (BR):
Within the fixed tree, the aligned profile is a Nash equilibrium of the restricted game: no one-leaf or all-leaf refit gains anything. An opponent who regrows its own eight-leaf tree captures 31 percent of the unrestricted gain, the smallest share of the six arms. At B=16 the one-leaf gain is 0.023 and the regrown-tree gain 0.156 of 0.404 (39 percent), again the smallest share (Appendix A, Table 7).
We probed exactly this against the fixed tree of Table 3 (Appendix A). An opponent who opens with a bet with probability b instead of checking loses value linearly in b: the tree gains 0.30 (opponent first to act) or 0.33 (opponent second) chips per hand at b=1, because the tree re-raises with the top card and the opponent folds at the raise cap. The full-game best response never opens with a bet; it bets in the second round after a check, which is the 0.433 gain NashConv already counts. The folding rule is exploitable, NashConv prices that exploit, and bluffing more is not the way to it.
Yes, both are now reported: payoffs under mutual deployment against fixed opponents in the table under the weakness above (Section 5, Table 5, and Appendix A), and deviations restricted to trees of the same leaf budget in the table above (Table 5, columns "one leaf" and "regrown tree").
We believe the revised manuscript and the new tables address the points raised, and we will answer any further question during the discussion period.