- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors introduce Pokerskill, a harness built from expert knowledge that extracts decision features for retrieval from a expert written skill library and filters the possible actions for each turn. They evaluate their system on five frontier LLMs against GTOWizard, claiming state of the art results for open-source poker AI systems, and examine how expert knowledge was used to create their system, claiming a generalizable pattern.
Critical strengths
None
Critical weaknesses
To be blunt, I really think this isn't a main conference level submission. While the authors establish their system is able to meaningfully increase LLM performance in HUNL there isn't nearly enough investigation into why (Table 2 is good, but the bare score alone isn't enough to show differences in behavior). The baselines are almost weak by construction. A bare prompt and a naive greedy algorithm are not sufficient to prove the strength of this approach. There are two critical baselines I consider to be missing: 1.) a version without the retrieval where the entirety of the library and everything is provided upfront to the model 2.) A GEPA style loop where the model is able to augment its own prompt and create its own harness (ADAS). As it stands, there is minimal evidence for their claim of a 'transferable design pattern', which would otherwise be the strongest claimed contribution.
Questions for authors
1.) You claim a 'transferable design pattern' but the skill library layer seems extremely bespoke for HUNL. Could you provide some examples of how this design might transfer to other domains?
2.) Could the authors provide more details about the rule-based baselines?
6 comments
The baselines are almost weak by construction. A bare prompt and a naive greedy algorithm are not sufficient to prove the strength of this approach.
We have added five non-LLM and matched-LLM controls, listed here in full (all 10,000 AIVAT hands against GTOWizard with DeepSeek-V41-Flash, the successor of the retired V4-Flash; mean ± 1 SE, mbb/hand; columns mark which parts of the prompt the configuration receives: preflop play from the P2 table, context-engine labels, retrieved P3–P5 text, the whole library in the system prompt, attack/defense budget, viable-option line).
| Configuration | Table | Labels | Retrieval | Whole lib. | Budget | Options | mbb/hand |
|---|---|---|---|---|---|---|---|
| PokerSkill | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ | −73 ± 21 |
| PokerSkill, hard enforcement of the viable set | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ (hard) | −71 ± 23 |
| PokerSkill, no viable-options line | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | −84 ± 23 |
| Default prompt | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | −153 ± 16 |
| Default prompt + preflop table | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | −139 ± 26 |
| (i) Labels only | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | −152 ± 20 |
| (ii) Labels + retrieved text | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | −101 ± 27 |
| (iii) Labels + budget + options | ✅ | ✅ | ❌ | ❌ | ✅ | ✅ | −139 ± 33 |
| Whole library, raw state | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | −165 ± 32 |
| Whole library + engine | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | −118 ± 27 |
Rule agents run the same pipeline up to the viable-option list and then use a fixed selector instead of the model:
| Selector over the viable options | mbb/hand |
|---|---|
| First option (paper, Table 1) | −132 ± 19 |
| Expert-ranked option | −120 ± 18 |
| Uniformly random option | −161 ± 21 |
What the two tables show. The structured state by itself changes nothing: the model plays as poorly with the engine's labels as with the raw cards (−152 versus −153), so the gain does not come from a better description of the state. Retrieved strategy text is the largest single ingredient (−101), but text alone leaves the model free to overcommit; the budget and viable options alone (−139) remove that freedom but leave it nothing to reason with; together they reach −73, more than the two effects add up to. Retrieval also beats exposure: the whole library in the context, with the engine kept, costs 45 mbb/hand (−118), and without the engine it is the worst configuration measured (−165), so the model cannot pick the applicable entries out of the library by itself. No fixed selector over the same options comes close to the model (−120 expert-ranked, −132 first, −161 random), and hard enforcement of the options changes fewer than 1% of the model's actions and 2 mbb/hand of the result: the model stays inside the option set of its own accord, and its choice within that set is where the strength lies. The expert-ranked and random rule agents are added to Table 1 of the revision, the LLM configurations form Table 3, and the appendix defines all three selectors.
There are two critical baselines I consider to be missing: 1.) a version without the retrieval where the entirety of the library and everything is provided upfront to the model
Done, in two versions. Putting the entire library (P1–P5, about 14k tokens) into the system prompt with only the raw game state scores −165 ± 32, the worst configuration we measured. Keeping the context engine, budget, and viable options while replacing the retrieved fragments with the whole library scores −118 ± 27, compared to −73 ± 21 with retrieval. Selecting the applicable entries is worth about 45 mbb/hand, and the whole library without the context engine is worse than no library.
2.) A GEPA style loop where the model is able to augment its own prompt and create its own harness (ADAS).
We ran this before submission. Using the same base model and evaluator, we tested (a) a reflector loop in which the LLM reads the hand logs and per-situation results and rewrites one part of the library at a time (RIVER-style one-factor mutation with an LLM judge); (b) self-consistency voting at multi-option decisions; (c) an empirical action-frequency prior mined from the agent's own showdown logs and bound as a majority action; (d) an empirical preflop table mined in the same way; (e) big-pot discipline rules mined from the largest losses; and (f) opponent-model exploitation rules mined from 400,000 hands against Slumbot. None of these improved upon the hand-written library. The two reflector rewrites were indistinguishable from the original (+11 ± 63 and +36 ± 74 mbb/hand); the four log-mined upgrades yielded +12 ± 38, +16 ± 39, +2 ± 35, and −33 ± 28; and the one rule that appeared positive against Slumbot in an arm-level comparison (+19.5 ± 5.4 at the decision level) fell to +1.8 ± 10.8 under an in-spot randomized control.
The reason is structural. Poker is an imperfect-information game with a large luck component. In our logs the raw outcome of a hand has a standard deviation of about 15 bb, so a candidate's win rate is known to ±150 mbb/hand after 10,000 hands and ±47 after 100,000; even a deliberately damaged library (with the top-pair advice inverted) was only 1.1 standard errors from the original in a 1,000-hand comparison. Separating two candidates that differ by 10 mbb/hand, the size of the smaller effects in the attribution table, takes about hands per candidate, and a search over fifty candidates about hands, with each hand costing several model calls; every "improvement" a loop accepts within an affordable budget is indistinguishable from a lucky run. The loop also inherits the limits of the model running it: rewrites are only as good as the rewriter, so weaker models, which have the most to gain, are the least able to improve themselves. This is why the library is written once by an expert and then fixed: once written down, knowledge does not have to be selected amid the noise, and the same unchanged library serves every model.
As it stands, there is minimal evidence for their claim of a 'transferable design pattern', which would otherwise be the strongest claimed contribution.
The reviewer is right that we present no second domain, and the revised paper frames this pattern as an outlook. What the paper does establish is that a general LLM, given a deterministic context engine, layered retrieval of expert skills, and a bounded action set, succeeds in a large imperfect-information game with a heavy luck component, which earlier LLM-agent work had not shown; the transfer question is addressed in Q1 below.
We thank the reviewer for the frank assessment and for naming the two missing baselines. Both have now been run, along with the attribution study requested by Reviewer vaQd, and we address each point below. The full table of new results is in the general response.
Weaknesses
Poker has been a milestone problem for imperfect-information game AI, and every strong HUNL agent so far has paid for its strength with training: solver-based systems (DeepStack, Libratus, Slumbot, GTOWizard) require millions of core-hours of CFR, while LLM-based systems such as ToolPoker rely on a solver queried at inference time, which itself must be trained. PokerSkill is the first strong HUNL agent that trains nothing and queries no solver, pairing an unmodified general-purpose LLM with an expert-written skill library. Under these constraints, it loses 44–87 mbb/hand to GTOWizard across five frontier models (Table 1). For comparison, the top human HUNL professionals lost 147 mbb/hand to Libratus (Brown & Sandholm, 2018), so PokerSkill's loss against a near-equilibrium solver is lower than that of the strongest human specialists. A training-free, solver-free agent at this level is a new result.
Two further points highlight the value of the result. First, to our knowledge, PokerSkill is both the strongest reported LLM poker agent and the strongest open, deployable HUNL agent: Slumbot, GTOWizard, Libratus, and Pluribus have not been released as runnable systems. Second, an agent at this level has uses for which near-equilibrium solvers are poorly suited. It is strong enough to be a sparring partner for human players and is a useful evaluation opponent for other poker AIs: the strong public opponents available today are solver-based and so close to equilibrium that they measure a single quantity, whereas an agent that plays well but not at equilibrium exposes a different, complementary set of weaknesses.
We agree that the score alone does not show how play changes, and we have added two types of evidence.
Mechanism. The context engine converts the raw state into the labels by which the library is indexed (hand class, board texture, action line, pot type, SPR, and position); retrieval provides the model with only the entries written for that situation; and the budget mechanism leaves it with a short list of viable actions. The LLM's role is to reason over that retrieved expert knowledge, read the opponent's line, and choose among the viable actions, including the bet size. The new attribution table isolates these components: structured labels alone do nothing (−152 vs. −153 for the default prompt), retrieved strategy text is the largest single factor (−101), the budget on its own is insufficient (−139), and only their combination reaches −73.
Behaviour. We compared the postflop action frequencies and bet sizes of the default-prompted model, PokerSkill, and the rule agents with GTOWizard's own play. Frequencies are in percent (fold/check/call/bet-or-raise); size is the median bet as a fraction of the pot. This is Table 7 of the revision.
The default-prompted model bets half pot on every street and bets the river only 18% of the time (GTOWizard: 34%). PokerSkill moves flop betting to a third of the pot and river betting to three quarters, toward the solver's size ladder, and bets the river 28% of the time. The rule agents share the same options but over-bet the flop (38%) and use a third of the pot on the turn and river, where the solver uses three quarters; the differences between the rule agents and PokerSkill, which reflect the LLM's contribution, lie in when to bet and how much.