acceptodds
Under review as a conference paper at ICLR 2027

Interface-Controlled Evaluation and Size Transfer of Language-Model Equilibrium Solvers on Procedural Zero-Sum Matrix Games

Abstract

We study language models as tool-free equilibrium solvers that read a serialized payoff matrix and write a strategy pair, and ask whether they can learn to solve unseen zero-sum games and whether named-game evaluations of such models measure the game or the prompt. Every answer is scored by an exploitability certificate that needs no equilibrium label. First, in an interface-controlled audit of Qwen3.5-9B on ten named base games with 12,600 released observations, the correct-label advantage is 24.6 points under raw prompting but 1.6 points under a matched chat template, and the interaction stays positive when base-game identities are resampled. Second, training on – games transfers to unseen sizes under a fixed short-answer protocol: at , the best-of-four success rate is 61% for LP-label SFT, 37% for a residual-reward GRPO recipe that uses no equilibrium labels, 16% for a chat-template base that answers validly in 93% of samples, and 15% for a format-only SFT control; SFT reaches 33% at pass@1. Third, the dominated-action padding control used to probe size effects needs calibration: a payoff-blind policy passes 45 of 50 padded games under range-normalized scoring but 6 of 50 at the embedded game's precision. We release the records, the calibrated control, and the scoring code.

Primary area:
datasets and benchmarks
Keywords:
language model evaluation, matrix games, equilibrium prediction, interface sensitivity, procedural benchmarks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_poj8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper evaluates language models that read serialized zero-sum payoff matrices and directly output mixed-strategy pairs.

It scores responses using normalized exploitability and audits how named-game label effects change between raw and matched chat interfaces. It trains Qwen3.5-9B on freshly generated 2×22\times2 and 3×33\times3 games using LP-label supervised fine-tuning or the label-free VERGE recipe, then evaluates transfer through 7×77\times7.

The experiments include format-only training, a format-competent chat baseline, payoff-reading heuristics, and both pass@1 and verifier-selected best-of-four results. The paper also shows that payoff-range normalization can make dominated-action padding misleading and introduces a calibrated control with randomized core positions.

Critical strengths

1, Checkable scoring and bounded theoretical claims.

The exploitability certificate evaluates strategy pairs without choosing an equilibrium label, and Theorem 1 bounds the effect of payoff perturbations on the unnormalized residual. This gives a concrete basis for examining numerical rendering errors, although it does not establish an advantage of VERGE over SFT.

2, Controls address identifiable evaluation confounds

The chat-template baseline, format-only SFT control, identity-level uncertainty estimates, and padding calibration distinguish several sources of apparent improvement. These controls make the reported gains more interpretable than a comparison against raw prompting alone.

Critical weaknesses

1, The learning contribution lacks controlled attribution.

VERGE changes the likelihood ratio, clipping bounds, reward grouping, anchor term, and rollout/update modes together. The anchor also increases the likelihood of every sampled response, while the training budgets differ from SFT in optimizer updates and sampled responses. Although these differences are disclosed, the experiments do not isolate whether the residual reward, the update rule, or format adaptation drives its behavior. The results therefore support a particular recipe more directly than a general conclusion about residual-based equilibrium learning.

2, Size transfer remains fragile and insufficiently characterized.

In Table 2, SFT pass@1 falls from 33% at 5×55\times5 to 12% at 7×77\times7, while VERGE falls from 21% to 7% and exhibits substantial variation across seeds. Under the calibrated padding test, moving the embedded core from fixed positions to randomized positions also reduces SFT performance considerably. Since training is evaluated on one backbone and principally one payoff generator, the evidence leaves unresolved how much transfer reflects a stable equilibrium computation rather than distribution-specific or positional behavior.

Questions for authors

1, See Critical Weakness 1

2, See Critical Weakness 2

3, The paper does not include Generative Adversarial Equilibrium Solvers [1], which also learns a mapping from game instances to strategy profiles without equilibrium labels. Could you include its formulation as an additional numerical reference under the same generator and exploitability score? Its numerical interface differs from the language-model interface, so the comparison should assess equilibrium approximation rather than prompt sensitivity.

[1] Denizalp Goktas, David C. Parkes, Ian Gemp, Luke Marris, Georgios Piliouras, Romuald Elie, Guy Lever, and Andrea Tacchetti. “Generative Adversarial Equilibrium Solvers.” ICLR, 2024.

4, How much of the training gain remains when all baselines receive an explicit minimax or Nash-equilibrium objective, with decoding and output constraints held fixed?

3 comments
AuthorsOpenReview ↗

Response to Reviewer poj8 (1/2): numerical reference (Q3) and size transfer (W2/Q2)

Thank you for the detailed and constructive review. This first comment covers Q3 and W2 / Q2.

Q3. "Could you include [GAES] as an additional numerical reference under the same generator and exploitability score?"

Yes. We implemented a GAES reference following its generator–discriminator formulation (cumulative-regret objective, stochastic exploitability descent) and its reported normal-form architecture and optimization settings: generator MLP with four hidden layers of 1,024 units and a player-wise softmax, Adam at 1e-3, batch 50, 10,000 iterations, payoffs normalized to [0, 1], five seeds, four discriminators. Two settings are adaptations: the fixed split has 1,700 training and 200 validation games per size (that work describes 2,000 games per class with 200 for validation), and the learned-discriminator schedule is taken from other sections of its appendix because the normal-form section does not specify one. The reference is trained separately at each size on our generator, on games disjoint from all test games, and scored with our normalized exploitability and the 0.10 threshold. We report the true best-response discriminator at the final iterate, with no tuning and no selection over seeds. Mean success over five seeds in this configuration (fixed training set):

SizeGAES: 2,000 unseen games / Table-2 games (of 50)LP-label SFT: best-of-4 / first sampleVERGE: best-of-4 / first sample
2×2100.0% / 50.0100% / 100%85% / 77%
3×399.7% / 50.097% / 84%66% / 50%
4×494.0% / 47.278% / 49%50% / 31%
5×568.0% / 33.061% / 33%37% / 21%
6×643.9% / 25.650% / 23%29% / 11%
7×732.3% / 14.632% / 12%18% / 7%

(2×2 uses 1,000 unseen games.) Two readings follow. First, on a fixed split of 1,700 games under 10,000 outer updates with batch size 50, this numerical reference also declines with size. Second, on the 50 Table-2 games, SFT's verifier-selected best-of-four rates (61%, 50%, 32% at 5×5–7×7) are numerically close to this reference (33.0, 25.6 and 14.6 of 50 games: 66.0%, 51.2%, 29.2%). The protocols differ: GAES reads exact real-valued payoffs and is trained at each test size, whereas the language models read a one-decimal rendering and are trained only at 2×2–3×3. The revised PDF reports this reference in Section 4.4 and all 240 training runs (six sizes × four discriminators × two data regimes × five seeds), also evaluated at the 0.05 threshold, in Appendix A.9.

W2 / Q2. "Size transfer remains fragile and insufficiently characterized."

The decline with size is part of the reported result: SFT pass@1 goes from 33% at 5×5 to 12% at 7×7. The numerical reference above declines with size on the same generator. With scoring at the embedded game's precision, randomized-position success remains above the core-uniform control: 32.7% across the three adapters' evaluations (49/150 12×12 games), versus 12% (6/50), with a game-level difference of 20.7 points, 95% CI [6.7, 34.7].

AuthorsOpenReview ↗

Response to Reviewer poj8 (2/2): controlled attribution (W1/Q1) and explicit objective (Q4)

W1 / Q1. "The learning contribution lacks controlled attribution … whether the residual reward, the update rule, or format adaptation drives its behavior."

We ran a controlled comparison in which every trained arm takes 2,000 optimizer updates from the same base checkpoint with the same LoRA configuration, schedule and seeds (42, 123, 7), which removes the difference in optimizer updates between SFT and VERGE. The two VERGE arms run the released code with 16,000 sampled responses each and differ in one line: the reward is the exploitability residual in one and a payoff-blind validity reward in the other (1 for a valid answer, −0.25 otherwise). The arms, the evaluation games and the comparison were chosen before any run. First-sample success on 300 further games (sizes 2–7), mean over three seeds:

ArmFirst-sample success
Base model7.0%
Format-only SFT5.4%
LP-label SFT52.0%
VERGE, residual reward12.4%
VERGE, payoff-blind validity reward5.3%

Holding the update rule, sampling budget and number of optimizer updates fixed, replacing the residual reward by the payoff-blind validity reward gives 100.0% valid outputs and lowers first-sample success from 12.4% to 5.3% (difference 7.1 points, 95% CI [4.1, 10.1]; game-level bootstrap, 20,000 replicates). The gain of this recipe over format adaptation is therefore attributable to the payoff-dependent reward. The validity-reward arm lands at the level of format-only SFT (difference −0.1 points, 95% CI [−3.4, 3.1]): a reward that never reads the payoffs produces format adaptation and nothing more.

Q4. "How much of the training gain remains when all baselines receive an explicit minimax or Nash-equilibrium objective, with decoding and output constraints held fixed?"

The full LP-label SFT margin remains. Each arm in this comparison received the sentence "Your objective is to output a Nash equilibrium of this zero-sum game: minimax-optimal mixed strategies for both players, such that neither player can improve their expected payoff by unilaterally changing strategy", under the chat template, four samples at T=0.7 and top-p 0.95, a 128-token cap and one parser. On the 300 unseen games, with the sentence the three SFT seeds solve 157 / 164 / 154 games on the first sample, format-only SFT 17 / 16 / 15, and the base model 21; without it, 154 / 157 / 157, 17 / 16 / 16 and 21. The margin of LP-label SFT over the base model is 45.0 points without the sentence and 45.8 with it (95% CI [41.6, 50.0]); over format-only SFT it is 46.6 and 47.4 (95% CI [43.1, 51.9]). Both intervals exclude zero, and the retained fractions of the two margins are 1.02 and 1.02. Stating the objective does not produce the capability: the base model solves 21 games either way. The same holds on the paper's 300 evaluation games (first-sample successes, original → explicit: base 25 → 20; LP-label SFT 167 / 160 / 157 → 166 / 159 / 155). With the sentence, the VERGE seed-7 adapter of Table 2 still holds a 22.7-point advantage over the base model on the unseen games (89 versus 21 of 300 games; 113 without the sentence; Appendix A.8). Output lengths are matched: 53–56 generated tokens per response for the base model and 60 for the SFT adapters, under the same cap.

Addition to W2 / Q2 (second payoff generator). On 300 games from a different generator defined in the released code (rounded Gaussian payoffs instead of uniform integers), the three SFT adapters solve 190, 180 and 184 games on the first sample against 26 for the base model (difference 52.9 points, 95% CI [48.2, 57.4]).

The revised PDF (new text in blue) reports these results in Section 4.4, Appendix A.7 (Tables 11–12) and Appendix A.8 (Table 13).

Thanks for the detailed responses and additional experiments, which have addressed my main questions and concerns. I plan to raise my score from 2 to 3.

Reviewer_1n82OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Paper studies whether LLM can produce reasonably close to nash equilibrium from two-player zero-sum payoff matrices, and also whether evaluations could distinguish this capability.

Critical strengths
  1. Paper findings are interesting regarding
  1. using the correct name of the game helps performance
  2. generalization to untrained sizes
Critical weaknesses
  1. The scope seems limited by studying LLM on a classical/tractable problem. The scope seems only be studying / challenging LLMs on a very narrow problem set.

  2. The generalization to untrained larger sizes would be more surprising from traditional machine learning perspective, but LLM seems to usually generalize and extrapolate well even in language spaces. Therefore merely generalizing to an unseen size seems expected.

Questions for authors

n/a

1 comment
AuthorsOpenReview ↗

Response to Reviewer 1n82

Thank you for the review. We answer both concerns directly.

"The scope seems limited by studying LLM on a classical/tractable problem."

This setting provides a directly computable exploitability score without equilibrium labels or a judge model. Here, "success" means normalized exploitability below 0.10, not an exact Nash equilibrium. For a predicted strategy pair, exploitability is computed directly from the evaluation payoff matrix, without an equilibrium label or a sampled opponent. Together with the accompanying controls, this lets us examine three things that named-game evaluations of LLM strategic reasoning mix together: whether the model produced a valid answer, how its success compares with format-competent and payoff-blind controls, and whether the scoring rule can tell the difference. The conclusions concern that evaluation practice, not a replacement for LP solvers. The interface comparison now covers four checkpoints from two families (8B–31B). Under a matched chat interface, the point estimates of the advantage of naming the game are within 3 points on all four (Qwen3.5-9B 1.6, Qwen3-8B −2.7, Qwen3.8-27B 2.0, Gemma-4-31B-it 0.9; all 95% intervals include zero), against 24.6 points under raw prompting on Qwen3.5-9B.

"…merely generalizing to an unseen size seems expected."

For this backbone and protocol, LP-label SFT substantially increases unseen-size success beyond the tested base and format-only controls, and the size of that increase is the result. On unseen 5×5 games, with decoding held fixed and prompt variants as indicated (Table 2):

System (trained sizes)First sampleBest of four
Base model, chat template (none)6%16%
Format-only SFT (2×2–3×3)5%15%
Three-example few-shot prompt (none)0%0%
LP-label SFT (2×2–3×3)33%61%

The base model with its chat template answers validly in 93% of samples, so its 6% is not a formatting failure. At 7×7 in Table 2, LP-label SFT still solves 12% on the first sample and 32% with best-of-four, where every base row and uniform play are at or below 4%. Stating the Nash-equilibrium objective in the prompt does not close the gap. In a separate chat-template evaluation of the same games (Tables 11–12 of the revision), on the 150 games at 5×5–7×7 the base model achieves six first-sample successes with either prompt, and the three SFT seeds achieve 36–43 with the explicit objective, compared with 37–42 without it. On 300 further unseen games (Section 4.4; Appendix A.8), the three SFT seeds solve 157, 164 and 154 on the first sample with the objective stated, against 21 for the base model and 17, 16 and 15 for format-only SFT.

We report the transfer as a function of size (33% at 5×5, 12% at 7×7) instead of a single headline number. The finding is therefore quantitative: how much of what is learned on 2×2–3×3 games carries to 5×5–7×7 relative to controls for output format and prompt wording.

The revised PDF (new text in blue) adds the two larger checkpoints and the explicit-objective control as Section 4.4.

Reviewer_1qCTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies tool-free language models that read serialized zero-sum payoff matrices and output both players' mixed strategies. It combines an interface audit of named-game evaluations, procedural training on 2×2–3×3 games, and a calibration of dominated-action padding. Answers are evaluated through NashConv divided by twice the payoff span; success below 0.10 therefore permits total unilateral regret below 20% of that span, rather than requiring an exact equilibrium. The audit finds that Qwen3.5-9B's correct-label advantage falls from 24.6 percentage points under raw prompting to 1.6 under a bundled chat interface. LP-label SFT and the residual-reward VERGE recipe are evaluated on larger, newly generated matrices. At 5×5, their reported best-of-four success rates are 61% and 37%, versus 16% for the chat-template base; SFT's pass@1 is 33%. A payoff-blind policy passes 45/50 dominated 12×12 games under full-span scoring but only 6/50 under core-span scoring. The evidence supports useful evaluation corrections and prompt-specific approximate size transfer, most convincingly for SFT.

Critical strengths
  1. Experimental transparency and substantial support for reproducibility. The paper provides detailed protocols, prompts, numerical conventions, training budgets, code, and records. It distinguishes results re-scored from full responses from those available only as stored summaries or per-sample rewards.
  2. Meaningful task formulation and research questions. Using zero-sum matrix-game solving as the target task, the paper asks relevant questions about how game-name cues and prompting interfaces affect measured performance, and whether learning on small matrices transfers to larger games. These questions are meaningful for understanding what language-model evaluations actually measure and how well the learned behavior generalizes beyond the training sizes.
Critical weaknesses

First, the experiments are restricted to Qwen3.5-9B and Qwen3-8B, with the training study conducted only on Qwen3.5-9B. These are relatively small models from the same family. Their behavior does not adequately establish how substantially larger and more capable models behave, particularly when stronger generalization could change the balance between reliance on familiar game knowledge and computation from the displayed payoffs. The interface interaction also differs considerably between the two tested checkpoints, both in magnitude and in its decomposition. This limits the extent to which the findings support broader conclusions about language-model strategic reasoning.

Second, the padding analysis primarily identifies a problem in the paper’s constructed evaluation streams. Although the calibration is useful, the paper does not demonstrate that correcting this problem materially changes a specific established conclusion. Its broader significance therefore remains less clear than the numerical results themselves. Similarly, the training component provides evidence of size transfer but does not introduce a particularly strong new algorithm or explain the mechanism responsible for that transfer.

My main concern is that, as an analysis paper, the manuscript does not sufficiently develop its measurements into clear and consequential scientific takeaways. The experimental section relies heavily on successive numerical comparisons, confidence intervals, and control results, making it difficult to identify the central insight connecting the experiments. The discussion offers useful evaluation recommendations, but provides limited explanation of what the results reveal about memorization, computation, and generalization, or how they should inform future model training and development. This weakens both the contribution and the reading experience.

Questions for authors

Which central conclusions are intended to apply beyond Qwen3.5-9B and Qwen3-8B? What evidence in the current study supports those conclusions for substantially larger or more capable models, given the difference in both interaction magnitude and decomposition between the two checkpoints? Which specific existing evaluation or published conclusion is materially changed by the interface or padding correction? Alternatively, is the intended contribution primarily the construction and calibration of a new local test protocol? What is the principal scientific insight connecting the interface audit, size-transfer results, and padding calibration, beyond the individual numerical findings? In particular, which observed result supports a concrete implication for improving model training or strategic generalization, rather than only improving evaluation procedure?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 1qCT

Thank you for the careful reading. We answer your three questions below.

"Which central conclusions are intended to apply beyond Qwen3.5-9B and Qwen3-8B? What evidence … supports those conclusions for substantially larger or more capable models?"

We ran the same interface comparison, unchanged (three grids, ten base games, 900 paired cases per interface, 1,024 tokens, greedy decoding), on Qwen3.8-27B and Gemma-4-31B-it: 10,800 new observations. Correct-label minus anonymous pass@1 in points, with 95% intervals over base-game identities:

CheckpointRawMatched chatRaw − chat
Qwen3.5-9B24.6 [9.1, 41.2]1.6 [−1.8, 4.9]23.0 [5.8, 42.1]
Qwen3-8B2.1 [−0.3, 6.0]−2.7 [−7.0, 0.7]4.8 [0.9, 9.3]
Qwen3.8-27B−3.0 [−6.9, 0.4]2.0 [−2.8, 7.9]−5.0 [−11.7, 1.0]
Gemma-4-31B-it1.8 [−0.1, 4.0]0.9 [−0.1, 2.6]0.9 [−1.8, 3.7]

Under matched chat the label effect lies between −2.7 and +2.0 points on all four checkpoints, and every identity-level 95% interval includes zero. On anonymous payoff-matrix inputs under chat, the larger checkpoints reach 89.4% (Gemma-4-31B-it) and 62.9% (Qwen3.8-27B) pass@1. The large raw-interface advantage is observed on Qwen3.5-9B only, where the reported decomposition assigns 24.0 of the 24.6 points to validity. The size of a raw-interface gap is thus checkpoint-dependent.

"Which specific existing evaluation or published conclusion is materially changed by the interface or padding correction?"

The contribution is a calibrated test protocol that identifies two ways such evaluations can be misread, shown on two results. First, on identical variants of named games, our estimate of the named-versus-anonymous gap on Qwen3.5-9B is 24.6 points under raw prompting and 1.6 under matched chat, so a gap measured under one interface need not reflect game knowledge. Second, dominated-action padding with range-normalized scoring is passed by a policy that never reads a payoff on 45 of 50 12×12 games, and on 6 of 50 at the embedded game's precision, so a pass under that control does not distinguish solving the embedded game from ignoring the padding. We release both the calibrated control and the evaluation protocol.

"What is the principal scientific insight … which observed result supports a concrete implication for improving model training or strategic generalization?"

The common lesson is that nominal success rates require controls for response validity, task-directed training, and scoring tolerance. On Qwen3.5-9B, the reported decomposition assigns 24.0 of the 24.6 raw label-effect points to validity; SFT gains remain against the tested format controls; and padding success changes sharply with scoring precision.

For training, we state comparisons with controls. In a separate chat-template run on the same games, with an explicit Nash/minimax objective in the prompt, the base model solves 20 of 300 games (aggregated over sizes 2×2–7×7) on the first sample (25 without it), while three SFT seeds solve 166, 159 and 155 (167, 160 and 157 without it). On the 150 of these games at 5×5–7×7, the base model solves 6 with either prompt and the SFT seeds 36–43 with the objective against 37–42 without it. In Table 2, format-only SFT answers validly in every sample and solves 5% of 5×5 games on the first sample, against 33% for LP-label SFT. On 300 further games, the margin of LP-label SFT over the base model is 45.0 points without the objective sentence and 45.8 with it (95% CI [41.6, 50.0]). In a controlled pair that differs only in the reward, replacing VERGE's exploitability residual by a payoff-blind validity reward lowers first-sample success from 12.4% to 5.3% (7.1 points, 95% CI [4.1, 10.1]). The implication for training is concrete: the gain comes from a payoff-dependent signal, and neither stating the objective nor format adaptation supplies it.

The revised PDF (new text in blue) reports these results in Section 4.4, Table 5, Appendix A.6 and Tables 11–13.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.