acceptodds
Under review as a conference paper at ICLR 2027

Generalizable Opponent Exploitation in LLM Poker Agents via Mixed Best-Response Training

Abstract

Opponent exploitation, adapting to and profiting from an opponent’s weaknesses, is a crucial capability in competitive games, yet existing LLM-based poker agents are mostly prompted or fine-tuned toward equilibrium play and fail to exploit suboptimal opponents. We introduce GOE-LLM (Generalizable Opponent Exploitation with LLMs), a framework for learning opponent exploitation in two-player zerosum poker. A lightweight MLP profiler summarizes an opponent’s recent behavior into a natural-language description, and an LLM exploiter conditioned on this description is trained with group relative policy optimization on a curated mixture of best-response strategies. We propose the Mixture-of-Best-Responses Principle, which excludes non-transitive dominance cycles from the mixture, to keep training stable while preventing collapse to a single equilibrium strategy. In Kuhn poker and Leduc Hold’em, GOE-LLM outperforms prompting and equilibrium-fine-tuned baselines across three model sizes and approaches the best-response upper bound, including against opponents unseen during training. Moreover, GOE-LLM trained only on Kuhn poker carries over zero-shot to unseen imperfect-information games in GTBench, improving the win rate by 18.0 points on average almost entirely through better strategy rather than fewer illegal moves, while bringing no gain in perfect-information games. These results suggest that mixed best-response training teaches LLM agents exploitation skills that generalize beyond the specific opponents and the single game seen in training.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, Machine Learning for NLP, Language Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_unsjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a framework for two player, zero sum, imperfect information games. An MLP profiler looks at the opponents past N games to predict a strategy (one of six types). Qwen2.5 (fine tuned on data that excludes non transitive dominance cycles) takes in this strategy and predicts the next move. The authors show experiments on Kuhn Poker and Leduc Hold'em, showing how the proposed framework can better exploit unseen opponents and also generalize zero-shot to GTBench.

Critical strengths

(S-1) The observation that finetuning data mixture can influence policy performance in two player, zero sum games is interesting and I believe studying this influence is important.

(S-2) The improved zero-shot transfer to prisoner's dilemma and Liar's Dice.

Critical weaknesses

(W-1) The mixture of best response principle is not ablated in terms of game performance. Figure 3 compares only the training dynamics of GOE, GOE (only GTO) and GOE (+NTDC) not the raw game performance metrics. We see that the GOE(+NTDC) gradient norm spread is higher, but I do not see the corresponding game performances reported.

(W-2) The strategy type being fixed to one of six classes seems like a rigid assumption. How would this framework handle dynamic opponents?

(W-3) The paper used Qwen2.5 1.5-7B as a base LLM to claim that vanilla LLMs are weak opponent exploiters. I think that more testing is necessary across a range of newer / frontier models before making this claim.

Questions for authors

(Q-1) As opposed to using an MLP profiler, have you experimented with passing in the previous N actions of the opponent into the LLM so it can infer the opponent's strategy zero shot? You may be able to do this with a frontier LLM. I would be interested to see how this compares to GOE-LLM and GOE-LLM (without the profiler).

(Q-2) The Figure 4 seen in training vs. unseen in training comparison does not seem to be fair because the profiler has been trained on the unseen types (unseen therefore means only unseen to the exploiter during GRPO training). At test time, the GOE-LLM is receiving information about the opponent's strategy with 96% accuracy (this description of the strategy can improve performance). Can you train a profiler which has not seen a set of held out types and report performance?

3 comments
AuthorsOpenReview ↗

Response to Reviewer unsj

Thank you for the review.

To Q-2: The profiler is trained on all six types (Lines 885–895), so "unseen" means unseen by the exploiter. The exploiter never sees the † types or their descriptions in training (Table 5). GOE-LLM w/o profiler is the limiting case of a profiler that has seen none of the held-out types: the † types are then unseen by the whole framework.

Table A. Exploitation rate (%) on the three held-out types

MethodValue†Passive†Agg.†Mean
LLM−163.1−91.3−21.5−92.0
LLM+OM−97.8−50.0−180.1−109.3
GTO-LLM98.933.9−157.1−8.1
GOE-LLM w/o profiler43.7−37.627.711.3
GOE-LLM69.541.355.055.3

Fully unseen, GOE-LLM is still positive on average (11.3%), while GTO-LLM and LLM are negative. The description adds 44.0 pp (z ≥ 3.3, worst-case variance) although the exploiter never saw it in training; the untuned model does worse with it (LLM+OM vs. LLM), so this ability comes from our training. Our generalization claim concerns the exploiter; a profiler for genuinely novel types is listed in the Limitations.

To Q-1: Not tested. The closest evidence is LLM+OM, which gives the untuned model the profiler's classification of the same history, i.e. the answer that raw-history inference would have to recover. It still does worse than no information (−109.3% vs. −92.0%, Table A). For the models we evaluate, the bottleneck is turning opponent information into exploitative actions, not inferring the type.

To W-1: Table B reports game performance for all three mixtures; GOE(only GTO) is the GTO-LLM baseline:

Table B. Kuhn poker, chips per hand and exploitation rate

MixtureP0P1Avg. ER (%)
GOE(only GTO)−0.0010.021−11.4
GOE (cycle-free)0.0210.13337.9
GOE(+NTDC)0.0100.09018.0

Adding the cycle halves the captured surplus (18.0% vs. 37.9%). Table 12 also quantifies Figure 3: GOE(+NTDC) learns slowest (critic score 0.9 at step 19 vs. 13 for GOE), has the largest gradient-norm spread (0.72 vs. 0.45), and its entropy rebounds more than GOE's at every model size. The added cycle is Aggressive ≻ Passive ≻ Value ≻ Aggressive (Appendix A.1, Lines 720–721).

To W-2:

  1. The six classes belong to the profiler, not the exploiter. The exploiter conditions on a natural-language description from any source (Lines 158–161).
  2. Each type covers parameterized variants: 24 strategies in Kuhn poker, 14 in Leduc Hold'em (Lines 248–250).
  3. For dynamic opponents, the profiler re-classifies every k = 10 games with ≥ 96% accuracy per type (Table 6). After a switch, the first window of only post-switch games, at most 2k = 20 games later, is identified with ≥ 96% accuracy. Opponents that co-adapt to the exploiter are out of scope.

To W-3: We have scoped the claim to the models we evaluate (Line 363). A frontier model with reasoning or code may well solve games as small as Kuhn poker, which we chose because BR* is exactly computable (Eq. 4). Computing the equilibrium is not enough, though: equilibrium play exploits nothing by definition, and GTO-LLM reaches −11.4% (Table B).

Our contribution is a training recipe that makes exploitation a learned skill rather than a per-game computation: GOE-LLM at 1.5B earns 0.130 chips per hand with one sample per move (Figure 4(b)) and transfers zero-shot to unseen games, which matters where test-time solving does not scale.

We welcome further discussion if any concerns remain.

Dear Reviewer unsj,

We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.

AuthorsOpenReview ↗

Have we addressed your concerns?

Dear Reviewer unsj,

Appendix D of the revised PDF is new and marked in blue. Table 10 reports the exploitation rate on the held-out types: fully unseen, GOE-LLM w/o profiler averages 11.3% against −8.1% for GTO-LLM (Q-2). Table 12 gives the three mixtures, putting the cost of the added cycle at 18.0% against 37.9% (W-1).

Does this address your concerns, or is there an analysis you would like to see? We would be glad to clarify anything further before the discussion period closes.

Reviewer_Z95qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GOE-LLM, an approach for opponent exploitation by LLM agents in two-player zero-sum poker. GOE-LLM identifies the opponent's type with a lightweight MLP profiler and generates a strategy that exploits the opponent's weaknesses according to the predicted type. The LLM "exploiter" is fine-tuned with GRPO to reproduce actions from a mixture of matchups. The matchups are chosen so that non-transitive dominance cycles are excluded; the authors call this the Mixture-of-Best-Responses Principle. According to the results on two poker games, GOE-LLM outperforms the vanilla LLM and GTO-LLM. GOE-LLM also shows good zero-shot transfer from poker to five GTBench games.

Critical strengths
  • Identifying an opponent's weaknesses and exploiting them is an interesting motivation. The authors show that in Kuhn Poker, GOE-LLM's behavior indeed changes per opponent, thanks to the training being conditioned on opponent descriptions. This means that GOE-LLM is doing something that equilibrium imitation is not doing.
  • The separate analysis of legality and strategy, and the idea that self-consistency and GOE-LLM may be complementary across games, is interesting.
Critical weaknesses
  • I don't think the paper actually shows that filtering out "counter cycles" works; at least it does not report any candidate mixture that was actually excluded. In Kuhn Poker for example, only GTO-type and Counter-Bluff-type strategies were used for training, and GTO-type strategies can't belong to a counter cycle because they aren't exploitable; were there actually counter-cycles among the Counter-Bluff strategies then? How many Counter-Bluff-type strategies were there?

  • The proposed approach rewards the model for imitating actions of fixed strategies, not for winning games. I'm not sure it is possible to learn good Poker strategies this way, because it disincentivizes randomization. Even if the imitated player knows to bluff, let's say, 25% of the time, the model would get the highest reward for never bluffing and going with the majority decision instead. A player that is pushed towards determinization could be easier to exploit.

  • The method also relies on knowing every strategy and computing which one beats which other one. In larger games, or with human players or just unknown players, this will be hard.

  • Most of the gain comes from Prisoner's Dilemma, where defection is always the correct choice against a random opponent. If the base LLM was trained to be cooperative, it is hard to tell whether the proposed method really learns to exploit the opponent, or whether the model is simply trained to be more aggressive. According to the results, the improvement is limited (+31.3 in Prisoner's Dilemma, +4.7 in Liar's Dice, and +0.9 on average in the perfect-information games).

Questions for authors
  • What matchups, labels and sample counts make up the +NTDC mixture? What chips per hand and exploitation rates do GOE(+NTDC) and GOE(only GTO) achieve in both games? A comparison where the two mixtures have the same number of non-equilibrium strategies and differ only in whether they contain a cycle would be best.
  • Were the ground-truth actions sampled from the labeled strategies, or set to their most likely action? What sampling temperature is used at evaluation? How often do GTO-LLM and GOE-LLM bet, call and fold at the Kuhn decision points where the equilibrium is randomized?
  • In Kuhn and Leduc, can you compute how exploitable the trained policies are? How much would a best response to GTO-LLM win, and how much a best response to GOE-LLM?
  • Can you report the poker results over several training seeds, with confidence intervals?
  • The profiler was trained on all six Kuhn types, including the types called "unseen". How does GOE-LLM perform against opponents that weren't seen at all during training?
7 comments
AuthorsOpenReview ↗

Response to Reviewer Z95q [1/2]

Thank you for the detailed feedback.

To Q1 and W1: The excluded cycle is documented in Appendix A.1 (Lines 720–721): Aggressive ≻ Passive ≻ Value ≻ Aggressive. GOE(+NTDC) adds matchups whose agent strategies form this cycle. These three types are exactly the ones left out of the GOE mixture and marked † in evaluation. So the GOE mixture is cycle-free largely by construction. The principle's role is to keep the cyclic types out, which is what GOE(+NTDC) tests.

We have now evaluated all three mixtures; GOE(only GTO) is the GTO-LLM baseline:

Table A. Kuhn poker, chips per hand and exploitation rate

MixtureP0P1Avg. ER (%)
GOE(only GTO)−0.0010.021−11.4
GOE (cycle-free)0.0210.13337.9
GOE(+NTDC)0.0100.09018.0

GOE(+NTDC) captures 18.0% of the exploitable surplus, about half of GOE's 37.9%. Table 12 also quantifies Figure 3: GOE(+NTDC) learns slowest (critic score 0.9 at step 19 vs. 13 for GOE), has the largest gradient-norm spread across runs (0.72 vs. 0.45), and its entropy rebounds more than GOE's at every model size (+0.08 to +0.14 vs. +0.03 to +0.06 by step 60).

To Q2 and W2: We agree with the analysis. With R_decision ∈ {0, 1} and entropy coefficient 0, the decision reward is maximized by the modal action, and only the KL term (β = 0.001) resists determinization. The labels are realized actions sampled from each strategy's mixed policy, not its most likely action: the data are generated by playing the rule-based strategies against each other (Line 744), and they sample their actions. At evaluation, the LLM samples with temperature 0.7 and top-p 0.95.

For the objective we target, exploiting a fixed opponent, determinization costs no return: against a fixed opponent a pure best response always exists. Consistently, GOE-LLM earns the game value against GTO opponents (−0.054 as P0 and 0.058 as P1 vs. ∓1/18 ≈ ∓0.056, Table 2), so it has not drifted away from equilibrium play where no surplus exists. The cost of determinization is exploitability by an opponent that adapts to the exploiter, the standard exploitation–exploitability trade-off[1].

To Q3: We did not compute exploitability, which requires the policy at every information set. GOE-LLM, like BR*, targets exploitation without a safety constraint.


[1] Ganzfried S, Sandholm T. Safe opponent exploitation[J]. ACM Transactions on Economics and Computation (TEAC), 2015.

AuthorsOpenReview ↗

Response to Reviewer Z95q [2/2]

To Q4: We did not train multiple seeds, but the evaluation noise can be bounded without any assumption on the policy. Kuhn payoffs lie in [−2, 2] chips, so the per-hand variance is at most 4. With 3,000 hands per opponent strategy, i.e. 1,500 per seat, the standard error of the average return in Table 2 is at most 0.0090 chips, i.e. 6.7 pp of exploitation rate. Even under this worst case, GOE-LLM exceeds GTO-LLM by 49.3 pp (z ≥ 5.2) and GTO play by 37.9 pp (z ≥ 4.0).

Across training runs, Figure 4(b) gives three independently trained GOE-LLM models (1.5B, 3B, 7B); all three are positive (0.130, 0.154, 0.117 chips per hand), and GOE-LLM is the only method positive at every size.

To Q5: Correct. The profiler is trained on all six types (Table 6), so "unseen" means unseen by the exploiter. The exploiter never sees the † types or their descriptions in training (Table 5).

Table B. Exploitation rate (%) on the held-out types

MethodValue†Passive†Agg.†Mean
GTO-LLM98.933.9−157.1−8.1
GOE-LLM w/o profiler43.7−37.627.711.3
GOE-LLM69.541.355.055.3

The profiler's description, unseen in training, adds 44.0 pp (z ≥ 3.3, worst-case variance), while the untuned model given the same description does worse than without it (Table 4, LLM+OM vs. LLM).

To W3: The cycle check runs over the agent-side strategies of the candidate matchups (Algorithm 1, step 2), three matchups per game, not over the full strategy space. At test time the exploiter needs only a description (Lines 175–177); Section 4.4 replaces the profiler with one sentence. We agree that building the strategy library is harder for large games or human players, and added this to the Limitations.

To W4: GOE-LLM has not simply become more aggressive. In Kuhn poker, aggressive play exploits Passive†, but the Value style exploits Aggressive† (Lines 720–721). GOE-LLM is positive against both (41.3% and 55.0%, Table B), so its play follows the opponent description, not a fixed style.

Defection is dominant in Prisoner's Dilemma, so we use it only for the claim of Lines 469–472: the model acts on the description of a random opponent in an unseen game. Given the same sentence, Base loses 52.3/23.2/21.1% of games (1.5B/3B/7B) and GOE-LLM 12.5/9.2/0.0%.

The gains are not small for zero-shot transfer. In Liar's Dice, the win rate among completed games rises from 47.0% to 55.0% at 3B (z = 3.6) and from 73.4% to 80.0% at 7B (z = 3.5); self-consistency at 15× the cost gains about 3 pp on these two games (Lines 427–428). The +0.9 pp in perfect-information games is what Lines 410–412 predict.

We welcome further discussion if any concerns remain.

Dear Reviewer Z95q,

We are writing to ask whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining issues.

Reviewer_YGc3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GOE-LLM, a framework for training LLM-based agents to exploit opponents' strategic weaknesses in imperfect-information games. The framework combines an MLP-based opponent profiler with an LLM policy fine-tuned using GRPO on a curated mixture of best-response strategies. Its main methodological contribution is a mixture-of-best-responses principle that excludes non-transitive dominance cycles to improve training stability while preserving strategic diversity. Experiments on Kuhn Poker and Leduc Hold'em show improvements over prompting-based and equilibrium-fine-tuned LLM baselines, including against opponent types unseen during training. The authors further report zero-shot transfer to five GTBench games, with an average 18-percentage-point win-rate improvement in imperfect-information games. However, the evaluation does not establish whether the proposed LLM-based approach offers advantages over conventional game-theoretic or reinforcement-learning methods.

Critical strengths
  1. Ablation-supported training design. The paper investigates how mixing best-response strategies affects training stability and strategic diversity, with ablations comparing cycle-free mixtures, mixtures containing non-transitive dominance cycles, and equilibrium-only training. This provides useful empirical evidence for the proposed training design, beyond reporting aggregate performance gains.

  2. Evidence of cross-game transfer. The authors evaluate models trained on Kuhn Poker on five different GTBench games, reporting an average 18.0-percentage-point win-rate improvement in imperfect-information games. This suggests that the learned behavior may transfer beyond the training environment, although the generality of this result remains uncertain.

Critical weaknesses
  1. Unclear necessity and contribution of LLMs. The proposed framework relies on an MLP for opponent classification and a curated mixture of best-response strategies for policy training, both of which build on established approaches. The paper does not compare against strong conventional game-theoretic or reinforcement-learning exploiters, making it difficult to establish the necessity of using an LLM or identify its unique contribution beyond serving as a policy approximator.

  2. Insufficient evidence for generalization and scalability. The main experiments are limited to small poker games with predefined, stylized opponent types. The cross-game evaluation uses random opponents and shows gains primarily in imperfect-information games, with almost no average improvement in perfect-information games. These results do not adequately establish generalizable opponent exploitation in more complex games or against genuinely novel, adaptive opponents.

Questions for authors
  1. Generalization to unseen opponents. The paper labels Value, Passive, and Aggressive opponents as unseen during training, yet Appendix A.4 reports high profiler classification accuracy across all six opponent categories. Were these categories included in the MLP profiler's training data? Please clarify whether these opponents are unseen only to the LLM exploiter or to the entire framework, as this distinction materially affects the generalization claim.

  2. Necessity of LLMs. Can the authors compare GOE-LLM against strong non-LLM opponent-exploitation baselines, using comparable training data and computational budgets? Since the proposed framework relies on conventional opponent classification and best-response training, such comparisons are essential to establish whether the LLM provides meaningful benefits beyond a smaller policy network or established game-theoretic methods.

5 comments
AuthorsOpenReview ↗

Response to Reviewer YGc3

Thank you for the review.

To Q1: No. The profiler is trained on all six Kuhn types (Lines 157–158; Table 6, Lines 885–895), so "unseen" means unseen by the LLM exploiter during GRPO training, not by the whole framework.

The exploiter-side claim can be checked with existing results. The exploiter's training data contains only the Random, GTO and Bluff matchups (Table 5), so the † types and their descriptions never appear in its training.

Table A. Exploitation rate (%) on the three held-out types.

MethodValue†Passive†Agg.†Mean
LLM−163.1−91.3−21.5−92.0
LLM+OM−97.8−50.0−180.1−109.3
GTO-LLM98.933.9−157.1−8.1
GOE-LLM w/o profiler43.7−37.627.711.3
GOE-LLM69.541.355.055.3
  • The w/o-profiler row is unseen by the whole framework: the exploiter never trained on these types and receives no description at test time. Its mean is still positive (11.3%), with positive rates on two of the three types, whereas GTO-LLM and LLM, which also receive no description, are negative on average.
  • The profiler's description adds 44.0 pp, although the exploiter never saw these descriptions in training.
  • The untuned model cannot use the same descriptions (LLM+OM −109.3% vs. LLM −92.0%), so this ability comes from our training.

To Q2 and W1: We do not claim that an LLM exploits better than game-theoretic methods in Kuhn poker or Leduc Hold'em. Given the hidden opponent, a conventional best-response oracle (as used inside IBR, DO or PSRO) is exactly our BR* row (Lines 258–259), which we report as the upper bound; Eq. 4 measures every method against it. Our question is different (Lines 40–47): LLM agents are increasingly used in games, yet current LLM agents are prompted or fine-tuned toward equilibrium play and fail to exploit (Table 4). As the title states, our scope is LLM poker agents.

Table A also shows what the LLM contributes. A policy conditioned on an opponent-type ID, whether tabular, an MLP or a PSRO population, has no training signal for an ID it never saw. GOE-LLM instead turns an unseen natural-language description into 44.0 pp of additional exploitation. The LLM also allows zero-shot deployment: the exploiter trained only on Kuhn poker plays Liar's Dice and Prisoner's Dilemma (Lines 423–428), where a policy for Kuhn poker cannot act at all.

To W2:

  • Game scale. Kuhn poker and Leduc Hold'em are small (Lines 505–506), but they are the standard testbeds in which BR* and the exploitation rate can be computed exactly.
  • Random opponents in GTBench. GTBench ships no strategic opponents (Line 412), so cross-game results against strategic opponents remain open; we state this in the Limitations.
  • Adaptive opponents. The profiler re-classifies the opponent every k = 10 games (Algorithm 1, step 8), and Table 6 shows ≥ 96% accuracy for every type on windows of k games. After an opponent switches type, the first window made entirely of post-switch games, at most 2k = 20 games later, is therefore identified with ≥ 96% accuracy. Opponents that adapt to the exploiter are out of scope.
  • Perfect-information games. The null result is expected and stated in advance (Lines 410–412): exploitation concerns the opponent's hidden information, while board games reward accurate look-ahead (Lines 471–473). In Liar's Dice, the win rate among completed games, which excludes any effect of illegal moves, rises from 47.0% to 55.0% at 3B and from 73.4% to 80.0% at 7B. This argues against a generic "better rule follower" effect.

We welcome further discussion if any concerns remain.

Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts to clarify the scope of the generalization claim and address several of the concerns raised during the review. However, I still have two major concerns.

1. The generalization claim should be further qualified.

The clarification confirms that the MLP profiler is trained on all six predefined Kuhn opponent types, including Value, Passive, and Aggressive. Therefore, these opponent types are unseen only by the LLM exploiter during GRPO training, rather than by the overall framework.

I agree that the additional results in Table 10 provide some evidence that the exploiter can generalize to opponent descriptions not encountered during its policy training. However, the substantially stronger result (55.3% exploitation rate) relies on a profiler that has already been trained on all evaluation categories. Even without the profiler, the exploitation rate is only 11.3% on average and is negative against Passive opponents.

I therefore think the distinction between exploiter-level generalization and end-to-end framework generalization should be made more explicit throughout the paper. In particular, the current results do not establish generalization to genuinely novel opponent types at the framework level. I appreciate that the authors have acknowledged this limitation in the revised appendix, but I believe the claims in the abstract and conclusion should be consistently scoped accordingly.

2. The fundamental motivation for using LLMs in poker remains unclear.

This is actually my more fundamental concern. The authors argue that their work focuses on LLM-based poker agents, and that GOE-LLM improves over existing LLM baselines. However, I do not think this sufficiently addresses the underlying question: Why should we use an LLM to build a poker agent in the first place?

We should not take the use of LLMs for poker agents as a given and then investigate how to make them better at poker. Instead, we should first establish what advantages LLMs bring to this problem that conventional approaches cannot readily provide.

The proposed framework combines an MLP-based opponent profiler with an LLM policy trained on a curated mixture of best-response strategies. Both opponent modeling and best-response learning are well-established ideas in game theory and reinforcement learning. The current experiments demonstrate that the proposed training recipe improves the performance of the evaluated LLMs over several LLM baselines, but they do not establish that an LLM is particularly suitable or advantageous for this task.

The BR* baseline does not resolve this concern either, since it is an oracle upper bound rather than a practical non-LLM competitor. I would be much more convinced if the authors could compare against conventional approaches, such as a compact policy network trained on the same mixed best-response data and conditioned on the same opponent information, or established opponent-modeling and best-response methods under comparable settings.

The natural-language interface and zero-shot cross-game transfer are interesting aspects of the proposed approach. However, the current results do not sufficiently isolate whether these benefits are genuinely attributable to using an LLM, rather than simply being consequences of the particular training setup. In particular, the GTBench experiments against random opponents do not establish a meaningful advantage of LLMs for poker opponent exploitation.

Overall, I appreciate the additional analyses, but my main concern remains unresolved. The paper provides evidence that a particular training recipe can improve LLM-based agents, but it is still unclear why an LLM is needed for this problem and what unique advantage it offers over conventional alternatives. I consider this a fundamental question about the motivation and contribution of the work, rather than simply a missing experimental detail.

AuthorsOpenReview ↗

Response to Reviewer YGc3

Thank you for the continued discussion.

1. Scope of the generalization claim. Our generalization claim is at the exploiter level. Besides the Limitations paragraph in Appendix D, the abstract (Line 25) and the conclusion (Line 481) state "unseen by the exploiter". This makes our contribution more precise, and we appreciate the suggestion.

2. Motivation for using LLMs. We do not claim that an LLM is the best way to build an exploiter for Kuhn poker or Leduc Hold'em. Where equilibria and best responses can be computed, conventional methods do this well, which is why BR* is our upper bound. In complex real-world games, a best response is very hard to compute, and exploiting flawed opponents remains an open problem.

The strong results of LLMs across domains make them a promising route there [1][2][3], yet current LLM agents are trained toward equilibrium play and fail to exploit [4][5] (Table 4). Our paper asks what it would take for LLM-based agents to move beyond equilibrium play and exploit opponents (Lines 40–47).

Kuhn poker and Leduc Hold'em are our testbeds, as in prior LLM poker work [5], because BR* and the exploitation rate can be computed exactly there (Lines 505–506), so we can measure how much of the exploitable surplus an LLM agent captures. The goal is not only to make LLMs better at poker but to find what LLM exploiters for more complex tasks need. Our results identify two such ingredients:

  • A cycle-free mixture of best responses, which keeps fine-tuning stable without collapsing to equilibrium play (Table 12).
  • A natural-language opponent interface. The trained exploiter uses descriptions of opponent types it never saw, and the exploiter trained only on Kuhn poker plays imperfect-information games without retraining (Table 13).

Scaling these ingredients to larger games is future work (Limitations, Appendix D).

We welcome further discussion if any concerns remain.


References

[1] Zhang W, Tang K, Wu H, et al. Agent-Pro: Learning to evolve via policy-level reflection and optimization[C]. Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

[2] Duan J, Zhang R, Diffenderfer J, et al. GTBench: Uncovering the strategic reasoning capabilities of LLMs via game-theoretic evaluations[C]. Advances in Neural Information Processing Systems (NeurIPS), 2024.

[3] Liu B, Guertler L, Yu S, et al. SPIRAL: Self-play on zero-sum games incentivizes reasoning via multi-agent multi-turn reinforcement learning[C]. International Conference on Learning Representations (ICLR), 2026.

[4] Zhuang R, Gupta A, Yang R, et al. PokerBench: Training large language models to become professional poker players[C]. AAAI Conference on Artificial Intelligence (AAAI), 2025.

[5] Lin M, Dai E, Liu H, et al. How far are LLMs from professional poker players? Revisiting game-theoretic reasoning with agentic tool use[C]. International Conference on Learning Representations (ICLR), 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revised PDF

We thank the reviewers. All revisions are marked in blue. New content is in a new Appendix D, so every line reference in our responses points to an unchanged line. In-place edits are wording only:

  1. "unseen during training" → "unseen by the exploiter" (Lines 25, 248–251, 481) (YGc3, Z95q, unsj).
  2. Line 363 scopes the conclusion of "Vanilla LLMs are weak opponent exploiters" to the base models we evaluate (unsj).
  3. The Figure 8 caption (Lines 998–999) now reads "average win chips", matching its panels.

Appendix D adds:

  1. Table 10: exploitation rate on the held-out types, with and without the opponent description (YGc3, Z95q, unsj).
  2. Table 11: worst-case significance of the Kuhn poker results (Z95q).
  3. Table 12: game performance and training dynamics of GOE(only GTO), GOE, and GOE(+NTDC) (Z95q, unsj).
  4. Table 13: Liar's Dice and Prisoner's Dilemma by game, with win rate among completed games and loss rate (YGc3, Z95q).
  5. Training labels and evaluation sampling (Z95q), and a Limitations paragraph (all reviewers).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.