acceptodds
Under review as a conference paper at ICLR 2027

Solver-Guided Reasoning for Mixed-Equilibrium Strategies

Abstract

Reasoning in large language models (LLMs) is often grounded in human text, human demonstrations, and human-generated rationales. For equilibrium reasoning in complex games, however, relying on human data can be suboptimal. Human play is often guided by intuition and heuristics, while human explanations emphasize individual actions rather than mixed strategies. To strengthen LLM reasoning in games, we study how to articulate equilibrium strategies using solver output. We propose a Mixed-Strategy Decision Tree (MDT), which represents solver mixtures through locally sparse, inspectable decision rules. We evaluate this representation through fixed-LLM strategy prediction, using Scenario-Constrained Counterfactual Sampling (SCCS) to construct same-context hand comparisons. We instantiate the study on No-Limit Texas Hold'em using over 250 million solver-labeled decisions. With identical reference policies, MDT routing reduces interpolation error by 9.8% relative to weighting by input distance on held-out boards. In-context comparisons across eight LLM configurations show improved prediction from solver-backed references. With forward-verified inputs and all requests counted, error falls by 37.7% for DeepSeek and 41.6% for GPT on policy-filtered targets; independently fixed, training-held-out queries yield reductions of 26.2% and 20.8%. Complete River policies and Liar's Dice extend the evaluation to strategic compression and a second game. The results establish useful policy relations in MDT routing and a practical interface for studying how LLMs use solver-backed evidence.

Primary area:
reinforcement learning
Keywords:
Poker, Game Thoery, LLM Reasoning, Mixed Nash Equilibrium

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_FQufOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates how to "translate" the mixed strategies produced by game solvers into explicit reasoning material that LLMs can understand. The authors propose MDT (Mixed-Strategy Decision Tree), which distills solver policies via a sparsely-routed tree that uses at most five features locally, so that the action distribution for each hand can be traced back to a set of inspectable branching rules and leaf mixtures. On the evaluation side, they design SCCS, which feeds a fixed LLM with the policies and routing differences of several reference hands sharing the same public context, and asks it to predict the solver strategy of a held-out target hand. On more than 250 million labeled No-Limit Texas Hold'em postflop decisions, MDT routing reduces reference interpolation error by 9.8% relative to raw input distance; providing LLMs with solver-backed references cuts error by 37–42% over direct prompting on DeepSeek and GPT; the full River policies achieve exploitability far below Deep CFR on the same tree; and the approach is further validated with a cross-game extension to Liar's Dice.

Critical strengths
  1. The paper is well-motivated and honestly scoped in its framing. Rather than casting solver-guided poker as yet another behavioral-cloning task that asks the model to predict the dominant action, it takes the mixed policy itself as the object to be communicated and inspected, which is a more faithful operationalization of reasoning about equilibrium. The authors explicitly position the work as a communication and interface problem, and they refrain from overclaiming that LLMs have learned to reason in the equilibrium sense.
  2. The representation is clean and inspectable. MDT's locally sparse routing, with each router attending to only a handful of summaries and reaching leaves through products of branch probabilities, yields a policy representation in which every piece of action mass is traceable to explicit inputs, branch computations, and leaf mixtures. The design avoids the common pitfall of post-hoc explainability: sparsity is built into the forward pass via hard masking rather than being retrofitted after training.
  3. The LLM evaluation is broad and disciplined. The in-context strategy-prediction study spans multiple model families and several configurations, and reports on both reference-first targets filtered by solver policy and independently fixed queries held out from training. A fixed LLM is used throughout, with failed outputs handled by a legal-uniform fallback so that failure modes are counted rather than discarded, and confidence intervals are derived through board-cluster resampling with appropriate multiple-comparison correction. The consistent error reduction across configurations and query populations is the kind of cross-configuration evidence that lends credibility to the interface claim.
Critical weaknesses
  1. The main text highlights the large gains from the full reference package, but the appendix reveals that the three reference-selection strategies do not differ significantly from one another, and the path-display ablation likewise fails to show that faithful routing traces enjoy a stable advantage over shuffled or repeated ones.
  2. The paper frames its contribution as solver-guided LLM reasoning, yet the LLMs are held fixed throughout and never updated. This makes the work, in practice, a study of a communication interface rather than of learned reasoning. The connection to the broader agenda of learning from non-human experience is gestured at but never operationalized.
  3. The cross-game extension is somewhat thin. The Liar's Dice experiment uses only a small batch of held-out targets and reports results averaged across many configurations, while the main text itself acknowledges substantial heterogeneity across those configurations without analyzing it further. There is also no ablation indicating which components of the NLH pipeline transfer cleanly and which had to be redesigned.
Questions for authors

Please see my comments on weaknesses.

1 comment

Thank you for recognizing the value of an inspectable mixed-policy representation and the solver-communication setting.

New paired evidence for MDT computation. We added an exploratory paired experiment during rebuttal. Before inference, we fixed 40 queries on MDT-training-held-out boards; each target and all four references have at least two solver actions with probability at least 0.10 and MDT-to-solver L1 at most 0.10. This focuses the test on mixed policies that MDT represents accurately. Both conditions receive identical reference labels, numerical inputs and reference MDT predictions; target policies and forward outputs are hidden. The treatment adds local contributions, simultaneous branches and leaf action mixtures, retaining at least 95% reference mass with residual accounting.

New rebuttal experiment: revised Table 40 (Appendix G.2, p. 44)

ModelCommon evidence L1+ Expanded tree L1Error reductionAdjusted 97.5% CI
DeepSeek-Flash0.15620.123720.8%[0.0016, 0.0667]
GPT0.11900.073538.2%[0.0049, 0.0942]

Note. L1 is summed over actions and averaged over queries; lower is better. Intervals are paired board-bootstrap intervals for common-minus-tree error, adjusted across the two models; positive values favor the expanded tree.

All 160 requests returned valid strategies and public explanations. Tables 38-39 tested displays without leaf action distributions and had inconclusive adjusted contrasts; this follow-up also uses a different cohort and explanation-enabled prompting. Table 41 (p. 45) illustrates GPT comparing simultaneous checking and betting branches and combining reach-weighted leaf mixtures.

We use SCCS as our main evaluation protocol to test whether an LLM can apply the strategy relationships conveyed by MDT to predict an unshown hand's mixed policy. Under a fixed board, betting history and acting position, an anchor and three contrasting shadows provide known policies and MDT routing information. The LLM uses these comparisons to predict a separate target whose solver policy is hidden, allowing its use of the supplied evidence to be scored quantitatively.

We include uniform and nearest-reference selection as ablation controls to examine sensitivity to reference choice. They change which four examples are supplied while retaining the same labeled-reference and route-contrast format. Tables 24-25 show lower error with nearest references and no significant SCCS-uniform difference.

A nearby hand's known policy may already approximate the target, so copying or small adjustments can yield low error without accounting for why action frequencies change across hands. SCCS deliberately selects policy and routing contrasts to expose these differences. Its role in the main evaluation is to test how the model applies those strategic comparisons to an unshown hand.

Reasoning with a fixed LLM. LLM reasoning does not require task-specific parameter updates, as demonstrated by "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (Wei et al., 2022) and "Large Language Models are Zero-Shot Reasoners" (Kojima et al., 2022). Here, non-human experience takes the form of solver reference policies and learned MDT relationships, which the LLM uses to infer an unshown hand's mixture. With the predictor fixed, the complete reference input reduces error relative to Direct by 26.2% for DeepSeek and 20.8% for GPT on independently fixed, training-held-out queries (Table 4).

Cross-game evaluation. The Liar's Dice communication test is exploratory: 31 targets, with private dice types held out by six-fold cross-fitting. Each target has two reference examples, an anchor and a contrasting shadow; its solver policy must differ from each by at least 0.10 in total variation. Revised Table 77 (p. 64) reports each configuration separately.

On paired parse-success targets, SCCS has lower point-estimate error than Direct and +Summaries across all eight configurations. Comparisons with Greedy Route-only depend on the scoring target: for GPT-5.5 high, solver-target error falls from 0.105 to 0.104 while MDT-target error rises from 0.068 to 0.086 (31/31 targets). Gemini-3.1 Flash parses 31/31 Greedy Route-only outputs versus 15/31 SCCS outputs.

The pipeline reuses sparse probabilistic routing, leaf mixtures and same-context hidden-policy prediction. The solver, actions, summaries, fitted parameters and reference settings are game-specific; Dice uses 17 summaries without per-action continuation values or Q-gaps.

Reviewer_VZabOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies how to represent solver-generated mixed strategies and communicate them to a fixed LLM. Its Mixed-Strategy Decision Tree (MDT) combines leaf action distributions through probabilistic routing, restricting each final router to at most five inputs. The NLH interface uses 156 summaries, including solver-derived equity and continuation-value quantities. Scenario-Constrained Counterfactual Sampling (SCCS) supplies an anchor and three same-context reference hands with differing policies/routes; the LLM predicts a separate hand's distribution. Additional experiments evaluate complete River policies and a small Liar's Dice game. Results support useful policy information in learned routing and improved prediction from complete solver-reference packages. They do not establish improved semantic understanding, a reliable incremental benefit from displaying paths, or superiority of SCCS over query-nearest retrieval.

Critical strengths

The appendix contains extensive supplementary experiments and detail, including implementation descriptions, evaluation protocols, and additional controls.

Critical weaknesses
  1. Presentation and formatting problems. The paper has some presentation problems. For example, the formatting between table captions and their corresponding tables is incorrect. These formatting issues put the submission at a risk of desk rejection.

  2. Incomplete and largely unreadable main text. The exposition in the main text is incomplete. The motivation behind the experiments and the contributions they are intended to establish are not clearly explained. Overall, I find the paper largely unreadable in its current form. Many passages convey little substantive information. For example, much of the experimental section simply lists numerical results and interval estimates without drawing meaningful conclusions from them. Readers are left to work out why these experiments were conducted, what the reported numbers actually establish, and how they support the paper’s contributions.

  3. Unclear overall motivation. The motivation for the paper as a whole is also unclear. The title emphasizes “Solver-Guided Reasoning,” but both the solver and MDT can already directly produce strategies. Why is it necessary to ask an LLM to predict the strategies again, and what practical purpose does this additional step serve? If the intention is to use the LLM as an explanation interface, the paper should evaluate whether its explanations are correct and whether they help people understand the strategies. If the intention is to replace the solver, the paper should demonstrate that the LLM remains effective with reduced access to the solver. If the intention is to improve decision-making, the paper should demonstrate an advantage over directly using the solver or other simple prediction methods. The paper does not clearly articulate which of these objectives it is pursuing or provide a convincing justification for introducing the LLM prediction stage.

Questions for authors

See above

2 comments

Why introduce an LLM? The intended use is an offline language interface for examining solver strategies: comparing hands, discussing changes in action frequencies, and estimating an unshown mixture from the supplied relationships. Inspired by hierarchical judgments that weigh considerations favoring different actions, MDT combines leaf experts through probabilistic branching. Its local calculations expose how decision inputs set the weights and how the experts' action distributions form the final mixture. Hidden-policy prediction gives this analytical task a measurable test; the case studies show the accompanying language.

Solver access and simpler prediction methods. The solver supplies summaries, reference policies and scoring targets. A numerical policy lookup remains available when only an action distribution is needed. Our comparison to the LLM's Direct baseline tests the use of solver evidence; Table 2 separately evaluates numerical reference interpolation. In the reference-selection ablations (Tables 24-25), nearest references produce lower error, but a nearby known policy may already approximate the answer. Reusing it need not involve understanding why the mixture changes. SCCS instead presents explicit policy contrasts and uses hidden-policy error to quantify how the LLM applies them across hands.

Why explain action frequencies? At positively reached information sets in an exact equilibrium, supported actions have equal value under equilibrium beliefs and continuations (Tadelis, "Game Theory: An Introduction", 2013, Chapter 6). Those values need not determine the mixture. ReBeL's "Combining Deep Reinforcement Learning and Search for Imperfect-Information Games" (Brown et al., 2020) illustrates this with equilibrium frequencies (0.4, 0.4, 0.2) despite equal action values. This motivates examining how decision inputs relate to probability allocation.

What has been evaluated about explanations? SCCS is our quantitative behavioral evaluation of how an LLM uses MDT explanations. The solver supplies a numerical policy target; verbal plausibility alone cannot determine whether an explanation accounts for the mixing frequencies. We hold the public context fixed, provide reference hands with contrasting strategies and MDT decision evidence, and ask the LLM to infer the mixed policy of another hand whose solver strategy is withheld. Error against that hidden policy measures how accurately the model applies the supplied strategy relationships beyond the reference examples.

The case study in Appendix F.7 (Table 37) shows the accompanying explanation. For Td9c on 8s6h5d, the saved GPT-5.5 high response connects "low showdown value but meaningful straight equity" with "solver data for the nearest T9 combo favoring a mixed call/raise response". Action-averaged solver L1 falls from 0.585 to 0.025. The case places the hand properties, reference comparison, verbal rationale and predicted mixture together. Figure 1(c) illustrates this case in the main text; Appendix F.7 (Table 37) provides the detailed scores and accompanying explanation.

What the experiments establish. Local sparsity limits the actual inputs to each rule, making a complete local account possible without a long tail of unreported terms. Table 1 measures its fidelity cost and the benefit of training that constraint over post-hoc masking. Table 2 tests learned policy relationships: fixed-label interpolation improves by 9.8% with routing rather than input distance. Table 4 tests whether reference evidence helps a fixed LLM infer a hidden strategy, reducing Direct error by 26.2% for DeepSeek and 20.8% for GPT on independently fixed, training-held-out queries. River evaluates complete-policy strategic compression; Liar's Dice tests reuse beyond poker.

Does displaying the MDT computation help? We have added a new paired experiment during rebuttal, reported in Table 40 of the revised manuscript (Appendix G.2, p. 44). This exploratory experiment fixes 40 queries on MDT-training-held-out boards before inference. Each target and reference has at least two solver actions of probability at least 0.10 and MDT-to-solver L1 at most 0.10. This focuses the test on mixed policies that MDT represents accurately. Both arms receive identical reference labels, numerical inputs and reference MDT predictions, with target policies and target forward outputs hidden. Adding local contributions, simultaneous branches and leaf mixtures reduces mean error by 20.8% for DeepSeek-Flash and 38.2% for GPT, with positive two-model-adjusted 97.5% paired intervals.

Concrete main-text revision. Section 5.2 now opens with the experimental question: "The prediction test asks whether supplied relationships help an LLM estimate a mixed policy that is absent from its references." It then specifies the evidence and withheld target information. The main reference-package results precede the input ablations; the expanded-tree test reports its matched comparison and eligibility conditions separately. Appendix F.7 now includes the quoted response excerpts.

Table formatting. We checked the six main-text tables in the compiled revision for caption/body adjacency and ordering. Captions appear above their tables; the main text remains nine pages in the conference template.

Reviewer_cVVzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper tackles a fairly practical question: what happens if you take the mixed strategies a game solver produces and try to turn them into something an LLM can actually use and verify? The authors’ answer is SCCS. The idea is to pick, for any given public state, a set of reference hands that already come with solver strategies, then feed those strategies—along with some numerical features and routing information—into a fixed LLM and have it predict strategy probabilities for hands it hasn’t seen. Training data comes from Texas Hold’em, over 250 million solver-labeled decisions in total, and they also test on Liar’s Dice and the full River subgame to see how well the approach holds up elsewhere.

Critical strengths

The most intresting part of this article is the problem it tries to solve: Once a system has mastered decision-making far beyond human level, but it cannot explain its own decisions, can we turn this ability into knowledge that another agent can learn and use? And, to my surprise, one of its negative result: On a fixed set of test queries, simple nearest-neighbor reference retrieval performs much better than SCCS.It may mean that when high-quality experience is close enough to the target problem, LLMs can recover quite accurate decisions from reference examples. No complex explicit reasoning is needed. In other words, we may have focused too much on making models "explain" decisions, and too little on organizing and retrieving high-quality computational experience. This makes me think of a potential system architecture: a high-performance solver / search / simulator. I appreciate this honesty, which gives me inspiration. The experiment is solid, and full detail is revealed in the appendix. Table 1 shows the fidelity-sparsity trade-off. Imposing sparse support during training (Top-5: 0.4251) clearly outperforms post-hoc masking (0.7395). This is a practically useful finding.

Critical weaknesses

The paper does not sufficiently demonstrate that MDT paths help LLM reasoning.The core narrative centers on interpretable MDT routing and on using routing comparisons to help LLMs understand strategy differences. However, after controlling for the same reference labels and numerical inputs, the incremental value of path information is not well established. SCCS is clearly outperformed by simple nearest-neighbor reference retrieval. This is the most convincing negative result, in my view. The authors disclose and discuss this, but it still weakens the core value of SCCS as a prediction method. There is a clear fidelity–sparsity trade-off in explainability. Table 1 on page 5 shows Top-5 is indeed much better than post-hoc masking. But it still loses substantial fidelity compared with the dense baseline. In addition, when retaining 95% routing quality, Top-5 MDT needs to show 16.60 leaf nodes and 39.03 distinct features on average. Having five features per node does not mean the whole strategy can be explained by five features.So "local sparsity" holds. However, "global simplicity and ease of understanding" still needs further evidence.

Questions for authors

What is the incremental contribution of MDT routing to LLM strategy prediction?

Why should SCCS be preferred over nearest-reference selection?

What is the operational definition of solver articulation?

2 comments

Why MDT? Our choice of MDT is inspired by a familiar way of explaining strategy: organizing conditional judgments hierarchically and weighing considerations that can favor different actions. Explaining a mixed strategy requires connecting these considerations to the allocation of action probabilities. This motivates probabilistic branching and leaf experts within a hierarchy. MDT's locally sparse routers combine public context, range information and hand properties, allowing multiple branches to contribute simultaneously. Each leaf expert provides an action distribution, and the final policy combines these distributions using their path probabilities as weights. The representation makes the local calculations and their contributions to the mixture explicit, giving an LLM decision evidence to compare and discuss.

Why SCCS rather than nearest references? Our purpose is to test whether an LLM can apply the strategy relationships conveyed by MDT explanations. A query-nearest hand's known policy may already approximate the target: copying or slightly adjusting it can produce low error without explaining how hand differences change the action frequencies. Thus the lower errors of nearest selection in Tables 24-25 do not by themselves establish strategic understanding. SCCS supplies one anchor and three contrasting shadows as its four reference examples, holding the public context fixed while exposing policy and route differences. We ask the LLM to apply these comparisons to another hand and quantify its performance against that hand's hidden solver policy. Uniform and nearest selection serve as reference-selection ablations; all three conditions use the same four-reference format.

What is solver articulation? By solver articulation, we mean converting a solver's numerical mixed policies into an explicit representation of how decision inputs contribute to action probabilities. In MDT, sparse routers compute branch probabilities. Their product along each path gives the reach weight of a leaf expert, whose output is an action distribution. Summing these weighted distributions reconstructs the MDT policy, whose fidelity is measured against the solver. SCCS then evaluates whether selected reference evidence helps a fixed LLM predict a separate hand's strategy.

What does routing add? The expanded display shows how local inputs determine branch weights and how multiple leaf experts contribute to the action mixture. Adding this computation to identical reference labels, numerical inputs and reference MDT predictions improves prediction in both models below. Before inference, we fix 40 queries on training-held-out boards: each target and all four references have at least two solver actions with probability at least 0.10 and MDT-to-solver L1 at most 0.10. This focuses the test on mixed policies that MDT represents accurately. The display retains at least 95% reference mass; target policies and forward outputs remain hidden.

New rebuttal experiment: revised Table 40 (Appendix G.2, p. 44)

ModelCommon evidence L1+ Expanded tree L1Error reductionAdjusted 97.5% CI
DeepSeek-Flash0.15620.123720.8%[0.0016, 0.0667]
GPT0.11900.073538.2%[0.0049, 0.0942]

Note. L1 is summed over actions and averaged over queries. Paired board-bootstrap intervals measure common-minus-tree error and are adjusted across two models. All 160 requests return valid strategies and public explanations.

The earlier displays in Tables 38-39 retain at least 95% routing mass but omit leaf action distributions; their adjusted comparisons did not establish a gain. The new paired result concerns an expanded display on the mixed-policy cohort above, with explanations requested in both arms. For numerical reference interpolation, Table 2 separately reports a 9.8% error reduction from routing-overlap weighting with labels fixed.

Why local sparsity? Local sparsity makes it feasible to read the complete rule at each node. With many small nonzero coefficients, displaying the entire calculation produces a long rule; displaying only its largest terms leaves part of the computation unexplained. A saved diagnostic illustrates this tail: an early router's root has 129 coefficients with absolute value above 0.005, including 37 between 0.005 and 0.02. Hard Top-5 constrains the actual forward computation to at most five active inputs, so all local input terms can be shown together with the bias and branch function.

Illustrative historical root-router coefficients

Absolute coefficientNumber of inputs
abs(w) >= 0.1025
0.02 <= abs(w) < 0.1067
0.005 < abs(w) < 0.0237

Note. Counts use the coefficients printed in a saved 192-input model diagnostic, not the current Table 1 models. A term's contribution also depends on its input value; coefficient magnitude alone is not an action contribution.

The full mixture combines several such local rules. Retaining 95% routing mass uses 16.60 leaves and 39.03 distinct inputs on average, so the readable unit is the local calculation. Table 1 measures the associated fidelity cost. Its trained Top-5 and Top-50 students share topology, parameter count and fitting protocol; imposing the tighter support during training preserves substantially more fidelity than post-hoc masking.

Fidelity comparison from Table 1

ModelMean L1 (lower is better)
Trained Top-500.3549
Trained Top-50.4251
Post-hoc Top-50-to-50.7395
Global-gated ResNet0.2804

Note. ResNet is a different-architecture and different-capacity fidelity benchmark; its gap from MDT does not isolate sparsity.

Language example. An illustrative new case is Td2d on Qs5s3s, IP after OOP checks. GPT's bet probability changes from 0.35 with common evidence to 0.39 with the tree; the hidden solver value is 0.3952, and summed L1 falls from 0.0904 to 0.0104. Its explanation compares the check-heavy N46 leaf with betting contributions through N61 and N111, stating: "Multiplying reach by each leaf mixture, then including suppressed mass, produces the reported MDT mixtures." We verified the cited reference probabilities against the saved forward computations. This illustrates a multi-branch comparison rather than an explanation based on a single chosen leaf.

Reviewer_7X2FOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Summary:

This paper studies how to make game-solver outputs, specifically mixed equilibrium strategies, usable as evidence for LLM reasoning. Specifically, the authors argue that human data emphasizes single best actions rather than mixed strategies, and that LLMs struggle to recover solver action frequencies. To address this, they propose the Mixed-Strategy Decision Tree (MDT), a probabilistic tree distilled from solver policies in which each router uses at most five solver-derived summaries, and Scenario-Constrained Counterfactual Sampling (SCCS), which builds same-context reference hands with their solver policies and MDT routing contrasts for a fixed LLM to predict a hidden target hand’s mixture. Experiments on over 250 million No-Limit Hold’em solver decisions, River endgames, and Liar’s Dice show that MDT routing improves reference interpolation by 9.8% over input distance, and that SCCS reference packages reduce LLM prediction error by 21–53% relative to direct prompting.

Critical strengths
  1. The topic is intersting.
  2. The evaluation is extensive.
Critical weaknesses
  1. The motivation is unclear. The paper claims to “strengthen LLM reasoning in games,” but the LLM is frozen throughout, no gameplay is evaluated, and the method requires solver access at inference time to compute both the input summaries and the reference policies (Appendix L). If a solver is available, one can directly query the target hand’s policy, so it is unclear why an LLM should predict it. If the goal is instead to make solver strategies interpretable, this is never evaluated: the paper states that no semantic comprehension, human inspection, or natural-language rationale is measured. Thus, it is unclear what problem this paper solves and who would benefit from it.
  2. The experiments do not support the effectiveness of the proposed methods. On independently fixed queries, simple nearest-reference retrieval far outperforms SCCS (0.296 vs. 0.631 for DeepSeek and 0.072 vs. 0.404 for GPT), and SCCS is not significantly better than uniform references (Tables 24–25). Adding MDT path displays also shows no significant benefit over the same labels and features (Tables 38–39). Thus, the gains mainly come from giving the LLM solver labels of nearby hands, not from MDT or SCCS. This indicates that we do not need the proposed method.
  3. The presentation and reproducibility are poor: (1)Hard to follow. The paper is 62 pages with many fragmented sub-experiments on different populations, and the main text is difficult to follow without the appendix. (2) Missing records. Several records are missing: exact run configurations “are not fully recorded” (Appendix B.2), Table 14’s dispersion definition “could not be recovered,” and selection counts are reconstructed because event logs are unavailable (Appendix E.3).
Questions for authors

Please refer to the weaknesses.

2 comments

We disagree with the conclusion that “we do not need the proposed method.” Reference-selection accuracy, the policy information in MDT routing, and the value of displaying its computation are different questions. The cited ablations do not establish that the representation is unnecessary.

Why introduce an LLM when a solver exists? We study an offline language interface for people examining solver strategies: comparing hands, discussing changes in action frequencies, and estimating an unshown mixture from explicit decision evidence. A solver supplies numerical policies; the research question is whether an LLM can use relationships extracted from those policies. MDT makes these relationships inspectable through locally sparse routers, probabilistic branches and weighted leaf distributions. SCCS tests their application by withholding a target policy and scoring the LLM's estimate against the solver. Solver-derived summaries and reference policies are available throughout this evaluation; its endpoint is mixed-policy inference.

Reasoning with a fixed model. Reasoning does not require task-specific parameter updates. “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (Wei et al., 2022) and “Large Language Models are Zero-Shot Reasoners” (Kojima et al., 2022) study reasoning through prompting fixed models. Here, the LLM compares solver-backed examples and infers a hidden strategy. On independently fixed, training-held-out queries, the complete reference input reduces error relative to Direct by 26.2% for DeepSeek and 20.8% for GPT (Table 4).

Why SCCS for evaluating explanations? Our question is whether an LLM understands MDT's strategy relationships well enough to apply them to an unshown hand. SCCS provides an indirect, quantitative test: with the public context fixed, an anchor and three shadows expose policy and routing contrasts, and the LLM uses these comparisons to predict a separate hand's hidden mixed strategy. Nearest references can make copying or slightly adjusting a known policy sufficient for low error. Uniform sampling leaves informative strategy contrasts to chance. Their prediction scores therefore do not directly answer our question about understanding MDT. Tables 24–25 report these reference-selection ablations: nearest gives lower error, and the SCCS–uniform difference is not significant. Among these three reference constructions, SCCS is the one specifically designed to test the application of MDT explanations through deliberate strategy contrasts.

New paired evidence isolates the added MDT display. We added an exploratory paired test during rebuttal (revised Table 40, Appendix G.2, p. 44). Before inference, we fixed 40 queries on MDT-training-held-out boards. Each target and all four references have at least two solver actions with probability at least 0.10 and MDT-to-solver L1 at most 0.10, focusing on mixed policies represented accurately by MDT. Both conditions receive identical reference labels, numerical inputs and reference MDT predictions. Target policies and all target forward outputs are hidden. Only the treatment adds local contributions, simultaneous branches and leaf action mixtures, retaining at least 95% reference mass with residual accounting.

New rebuttal experiment: revised Table 40 (Appendix G.2, p. 44)

ModelCommon evidence L1+ Expanded tree L1Error reductionAdjusted 97.5% CI
DeepSeek-Flash0.15620.123720.8%[0.0016, 0.0667]
GPT0.11900.073538.2%[0.0049, 0.0942]

Note. L1 is summed over actions and averaged over queries; lower is better. Intervals are paired board-bootstrap intervals for common-minus-tree error, adjusted across the two models; positive values favor the expanded tree.

All 160 requests returned valid strategies and public explanations. Tables 38–39 tested routing displays without leaf action distributions and yielded inconclusive adjusted contrasts. The new test uses an expanded display, a qualified mixed-policy cohort and explanation-enabled prompting; its incremental estimate comes from the matched arms within that setting. In Table 41 (p. 45), GPT compares checking and betting branches and describes combining reach-weighted leaf mixtures. Its bet estimate changes from 0.35 to 0.39 against the hidden solver value 0.3952. The cited reference probabilities were checked against saved forward computations.

Table 2 independently tests MDT's learned policy relationships without an LLM: with reference labels fixed, routing-overlap weighting reduces interpolation error by 9.8% relative to standardized input distance. Table 1 measures the fidelity–sparsity trade-off under matched fitting. These representation tests, the complete-reference evaluation, and the paired display test establish different parts of the proposed interface.

Presentation and experimental structure. The main text is nine pages; the 62-page count includes references and appendices. Its experiments address representation fidelity, LLM use of strategy evidence, and complete-policy or cross-game evaluation. Section 5.2 in the revision now opens with the precise test: “whether supplied relationships help an LLM estimate a mixed policy that is absent from its references.” The main reference results precede component ablations. We will make this question-to-result structure explicit at each experimental subsection, keeping protocol detail in the appendix.

Protocols supporting the main results. The main controlled comparisons use fixed evaluation sets and explicit scoring rules. The appendix provides supporting protocols, ablations and sensitivity analyses. The qualifications in B.2, Table 14 and E.3 concern exact training-run reproduction, dispersion reporting for a supplementary architecture comparison, and reconstructed sampling counts, respectively. The paired gains in the main text and the new rebuttal experiment are calculated from retained predictions and scoring targets, with uncertainty assessed using specified board-bootstrap procedures.

We ask the reviewer to reassess the categorical conclusion that the method is unnecessary in light of these distinctions and the new paired evidence, and to specify any soundness objection that remains unresolved.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We thank the reviewers for their careful assessment. Solver policies specify action frequencies; our question is how to make the relationships behind these assignments usable for LLM inference and explanation. MDT organizes solver knowledge into inspectable computations. SCCS supplies an anchor and contrasting shadows as reference examples, then asks the model to estimate a separate target's strategy. This tests whether the evidence is useful beyond reproducing a supplied answer. Our Direct baselines show substantial errors in estimating these mixtures, and keeping the LLM fixed lets us measure how its predictions change when solver-backed evidence is supplied.

The difficulty lies in explaining the action distribution of a mixed equilibrium strategy. In "Superhuman AI for heads-up no-limit poker: Libratus beats top professionals" (Brown and Sandholm, 2018), action frequencies affect what private information an action reveals and how an opponent can respond. "Safe and Nested Subgame Solving for Imperfect-Information Games" (Brown and Sandholm, 2017) shows that strategies in unreached parts of the game can constrain a reached subgame. ReBeL, introduced in "Combining Deep Reinforcement Learning and Search for Imperfect-Information Games" (Brown et al., 2020), illustrates that even exact equilibrium action values can be insufficient for recovering the equilibrium mixture through local lookahead. Explaining why an action is reasonable therefore leaves open why it should occur at a particular frequency alongside other actions.

Our design is inspired by a familiar way of explaining strategy: organizing conditional judgments hierarchically and weighing considerations that can favor different actions. This motivates both MDT's hierarchy and its combination of probabilistic branching with leaf experts. Locally sparse routers combine public context, range information and hand properties to assign branch weights. Hard input selection makes it possible to list every active term in a local rule, avoiding an unexplained tail of small nonzero coefficients. Each leaf expert supplies an action distribution; its contribution to the final mixture is weighted by the product of branch probabilities along its path. These local calculations make the formation of the mixed policy inspectable, providing explicit decision evidence for LLM prediction and language explanation.

SCCS evaluates the use of MDT explanations through a quantitative behavioral task. A nearby hand's known policy may already approximate the answer, allowing low error through direct reuse without an account of the action frequencies. SCCS supplies same-context hands with contrasting policies and routes, exposing how hand differences accompany changes in strategy. The LLM applies these relationships to a separate hand whose solver policy is hidden, and prediction error measures the accuracy of that application. Reference-selection and path-display ablations separately examine sensitivity to example choice and displayed computation. The qualitative case study places the model's verbal account alongside its prediction.

Sparse distillation measures the fidelity cost of restricting local computations. Table 2 tests policy information in routing without an LLM: with reference policies fixed, learned routing reduces numerical interpolation error by 9.8% relative to standardized input distance. The main SCCS evaluation tests a fixed LLM's use of complete reference evidence, while the target's policy, MDT prediction and route are withheld. On independently fixed, training-held-out queries, this input reduces prediction error by 26.2% for DeepSeek and 20.8% for GPT relative to Direct (Table 4).

Additional evidence from expanded reference trees. We have added an exploratory paired experiment during rebuttal to test the value of displaying MDT's branching computation (revised manuscript, Appendix G.2, Table 40, p. 44). We fix 40 training-held-out-board queries before inference; each target and all references have at least two solver actions with probability at least 0.10 and MDT-to-solver L1 at most 0.10. This focuses the test on mixed policies that MDT represents accurately. With the same reference labels, numerical inputs and reference MDT predictions in both conditions, adding local contributions, simultaneous branches and leaf mixtures lowers mean L1 from 0.1562 to 0.1237 for DeepSeek-Flash (20.8%) and from 0.1190 to 0.0735 for GPT (38.2%). Both two-model-adjusted 97.5% paired intervals exclude zero. All 160 requests return valid strategies. This provides positive incremental evidence for the expanded display on faithfully represented mixed policies.

The qualitative case in Appendix F.7 (Table 37) makes the language interface concrete. For Td9c on 8s6h5d, the model initially favors folding because of poor out-of-position realization. With reference evidence, its rationale describes low showdown value and straight potential, then invokes a nearby T9 policy to motivate a mixed call/raise response. The example shows the hand description, reference comparison and resulting prediction together.

Updated manuscript. We have uploaded a revised PDF containing the new paired experiment and a multi-branch explanation example (Appendix G.2, Table 40 on p. 44 and Table 41 on p. 45).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.