- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a framework for two player, zero sum, imperfect information games. An MLP profiler looks at the opponents past N games to predict a strategy (one of six types). Qwen2.5 (fine tuned on data that excludes non transitive dominance cycles) takes in this strategy and predicts the next move. The authors show experiments on Kuhn Poker and Leduc Hold'em, showing how the proposed framework can better exploit unseen opponents and also generalize zero-shot to GTBench.
Critical strengths
(S-1) The observation that finetuning data mixture can influence policy performance in two player, zero sum games is interesting and I believe studying this influence is important.
(S-2) The improved zero-shot transfer to prisoner's dilemma and Liar's Dice.
Critical weaknesses
(W-1) The mixture of best response principle is not ablated in terms of game performance. Figure 3 compares only the training dynamics of GOE, GOE (only GTO) and GOE (+NTDC) not the raw game performance metrics. We see that the GOE(+NTDC) gradient norm spread is higher, but I do not see the corresponding game performances reported.
(W-2) The strategy type being fixed to one of six classes seems like a rigid assumption. How would this framework handle dynamic opponents?
(W-3) The paper used Qwen2.5 1.5-7B as a base LLM to claim that vanilla LLMs are weak opponent exploiters. I think that more testing is necessary across a range of newer / frontier models before making this claim.
Questions for authors
(Q-1) As opposed to using an MLP profiler, have you experimented with passing in the previous N actions of the opponent into the LLM so it can infer the opponent's strategy zero shot? You may be able to do this with a frontier LLM. I would be interested to see how this compares to GOE-LLM and GOE-LLM (without the profiler).
(Q-2) The Figure 4 seen in training vs. unseen in training comparison does not seem to be fair because the profiler has been trained on the unseen types (unseen therefore means only unseen to the exploiter during GRPO training). At test time, the GOE-LLM is receiving information about the opponent's strategy with 96% accuracy (this description of the strategy can improve performance). Can you train a profiler which has not seen a set of held out types and report performance?
3 comments
Dear Reviewer unsj,
We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.
Have we addressed your concerns?
Dear Reviewer unsj,
Appendix D of the revised PDF is new and marked in blue. Table 10 reports the exploitation rate on the held-out types: fully unseen, GOE-LLM w/o profiler averages 11.3% against −8.1% for GTO-LLM (Q-2). Table 12 gives the three mixtures, putting the cost of the added cycle at 18.0% against 37.9% (W-1).
Does this address your concerns, or is there an analysis you would like to see? We would be glad to clarify anything further before the discussion period closes.
Response to Reviewer unsj
Thank you for the review.
To Q-2: The profiler is trained on all six types (Lines 885–895), so "unseen" means unseen by the exploiter. The exploiter never sees the † types or their descriptions in training (Table 5). GOE-LLM w/o profiler is the limiting case of a profiler that has seen none of the held-out types: the † types are then unseen by the whole framework.
Table A. Exploitation rate (%) on the three held-out types
Fully unseen, GOE-LLM is still positive on average (11.3%), while GTO-LLM and LLM are negative. The description adds 44.0 pp (z ≥ 3.3, worst-case variance) although the exploiter never saw it in training; the untuned model does worse with it (LLM+OM vs. LLM), so this ability comes from our training. Our generalization claim concerns the exploiter; a profiler for genuinely novel types is listed in the Limitations.
To Q-1: Not tested. The closest evidence is LLM+OM, which gives the untuned model the profiler's classification of the same history, i.e. the answer that raw-history inference would have to recover. It still does worse than no information (−109.3% vs. −92.0%, Table A). For the models we evaluate, the bottleneck is turning opponent information into exploitative actions, not inferring the type.
To W-1: Table B reports game performance for all three mixtures; GOE(only GTO) is the GTO-LLM baseline:
Table B. Kuhn poker, chips per hand and exploitation rate
Adding the cycle halves the captured surplus (18.0% vs. 37.9%). Table 12 also quantifies Figure 3: GOE(+NTDC) learns slowest (critic score 0.9 at step 19 vs. 13 for GOE), has the largest gradient-norm spread (0.72 vs. 0.45), and its entropy rebounds more than GOE's at every model size. The added cycle is Aggressive ≻ Passive ≻ Value ≻ Aggressive (Appendix A.1, Lines 720–721).
To W-2:
To W-3: We have scoped the claim to the models we evaluate (Line 363). A frontier model with reasoning or code may well solve games as small as Kuhn poker, which we chose because BR* is exactly computable (Eq. 4). Computing the equilibrium is not enough, though: equilibrium play exploits nothing by definition, and GTO-LLM reaches −11.4% (Table B).
Our contribution is a training recipe that makes exploitation a learned skill rather than a per-game computation: GOE-LLM at 1.5B earns 0.130 chips per hand with one sample per move (Figure 4(b)) and transfers zero-shot to unseen games, which matters where test-time solving does not scale.
We welcome further discussion if any concerns remain.