- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates how to "translate" the mixed strategies produced by game solvers into explicit reasoning material that LLMs can understand. The authors propose MDT (Mixed-Strategy Decision Tree), which distills solver policies via a sparsely-routed tree that uses at most five features locally, so that the action distribution for each hand can be traced back to a set of inspectable branching rules and leaf mixtures. On the evaluation side, they design SCCS, which feeds a fixed LLM with the policies and routing differences of several reference hands sharing the same public context, and asks it to predict the solver strategy of a held-out target hand. On more than 250 million labeled No-Limit Texas Hold'em postflop decisions, MDT routing reduces reference interpolation error by 9.8% relative to raw input distance; providing LLMs with solver-backed references cuts error by 37–42% over direct prompting on DeepSeek and GPT; the full River policies achieve exploitability far below Deep CFR on the same tree; and the approach is further validated with a cross-game extension to Liar's Dice.
Critical strengths
- The paper is well-motivated and honestly scoped in its framing. Rather than casting solver-guided poker as yet another behavioral-cloning task that asks the model to predict the dominant action, it takes the mixed policy itself as the object to be communicated and inspected, which is a more faithful operationalization of reasoning about equilibrium. The authors explicitly position the work as a communication and interface problem, and they refrain from overclaiming that LLMs have learned to reason in the equilibrium sense.
- The representation is clean and inspectable. MDT's locally sparse routing, with each router attending to only a handful of summaries and reaching leaves through products of branch probabilities, yields a policy representation in which every piece of action mass is traceable to explicit inputs, branch computations, and leaf mixtures. The design avoids the common pitfall of post-hoc explainability: sparsity is built into the forward pass via hard masking rather than being retrofitted after training.
- The LLM evaluation is broad and disciplined. The in-context strategy-prediction study spans multiple model families and several configurations, and reports on both reference-first targets filtered by solver policy and independently fixed queries held out from training. A fixed LLM is used throughout, with failed outputs handled by a legal-uniform fallback so that failure modes are counted rather than discarded, and confidence intervals are derived through board-cluster resampling with appropriate multiple-comparison correction. The consistent error reduction across configurations and query populations is the kind of cross-configuration evidence that lends credibility to the interface claim.
Critical weaknesses
- The main text highlights the large gains from the full reference package, but the appendix reveals that the three reference-selection strategies do not differ significantly from one another, and the path-display ablation likewise fails to show that faithful routing traces enjoy a stable advantage over shuffled or repeated ones.
- The paper frames its contribution as solver-guided LLM reasoning, yet the LLMs are held fixed throughout and never updated. This makes the work, in practice, a study of a communication interface rather than of learned reasoning. The connection to the broader agenda of learning from non-human experience is gestured at but never operationalized.
- The cross-game extension is somewhat thin. The Liar's Dice experiment uses only a small batch of held-out targets and reports results averaged across many configurations, while the main text itself acknowledges substantial heterogeneity across those configurations without analyzing it further. There is also no ablation indicating which components of the NLH pipeline transfer cleanly and which had to be redesigned.
Questions for authors
Please see my comments on weaknesses.
Thank you for recognizing the value of an inspectable mixed-policy representation and the solver-communication setting.
New paired evidence for MDT computation. We added an exploratory paired experiment during rebuttal. Before inference, we fixed 40 queries on MDT-training-held-out boards; each target and all four references have at least two solver actions with probability at least 0.10 and MDT-to-solver L1 at most 0.10. This focuses the test on mixed policies that MDT represents accurately. Both conditions receive identical reference labels, numerical inputs and reference MDT predictions; target policies and forward outputs are hidden. The treatment adds local contributions, simultaneous branches and leaf action mixtures, retaining at least 95% reference mass with residual accounting.
New rebuttal experiment: revised Table 40 (Appendix G.2, p. 44)
Note. L1 is summed over actions and averaged over queries; lower is better. Intervals are paired board-bootstrap intervals for common-minus-tree error, adjusted across the two models; positive values favor the expanded tree.
All 160 requests returned valid strategies and public explanations. Tables 38-39 tested displays without leaf action distributions and had inconclusive adjusted contrasts; this follow-up also uses a different cohort and explanation-enabled prompting. Table 41 (p. 45) illustrates GPT comparing simultaneous checking and betting branches and combining reach-weighted leaf mixtures.
We use SCCS as our main evaluation protocol to test whether an LLM can apply the strategy relationships conveyed by MDT to predict an unshown hand's mixed policy. Under a fixed board, betting history and acting position, an anchor and three contrasting shadows provide known policies and MDT routing information. The LLM uses these comparisons to predict a separate target whose solver policy is hidden, allowing its use of the supplied evidence to be scored quantitatively.
We include uniform and nearest-reference selection as ablation controls to examine sensitivity to reference choice. They change which four examples are supplied while retaining the same labeled-reference and route-contrast format. Tables 24-25 show lower error with nearest references and no significant SCCS-uniform difference.
A nearby hand's known policy may already approximate the target, so copying or small adjustments can yield low error without accounting for why action frequencies change across hands. SCCS deliberately selects policy and routing contrasts to expose these differences. Its role in the main evaluation is to test how the model applies those strategic comparisons to an unshown hand.
Reasoning with a fixed LLM. LLM reasoning does not require task-specific parameter updates, as demonstrated by "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (Wei et al., 2022) and "Large Language Models are Zero-Shot Reasoners" (Kojima et al., 2022). Here, non-human experience takes the form of solver reference policies and learned MDT relationships, which the LLM uses to infer an unshown hand's mixture. With the predictor fixed, the complete reference input reduces error relative to Direct by 26.2% for DeepSeek and 20.8% for GPT on independently fixed, training-held-out queries (Table 4).
Cross-game evaluation. The Liar's Dice communication test is exploratory: 31 targets, with private dice types held out by six-fold cross-fitting. Each target has two reference examples, an anchor and a contrasting shadow; its solver policy must differ from each by at least 0.10 in total variation. Revised Table 77 (p. 64) reports each configuration separately.
On paired parse-success targets, SCCS has lower point-estimate error than Direct and +Summaries across all eight configurations. Comparisons with Greedy Route-only depend on the scoring target: for GPT-5.5 high, solver-target error falls from 0.105 to 0.104 while MDT-target error rises from 0.068 to 0.086 (31/31 targets). Gemini-3.1 Flash parses 31/31 Greedy Route-only outputs versus 15/31 SCCS outputs.
The pipeline reuses sparse probabilistic routing, leaf mixtures and same-context hidden-policy prediction. The solver, actions, summaries, fitted parameters and reference settings are game-specific; Dice uses 17 summaries without per-action continuation values or Q-gaps.