- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper evaluates language models that read serialized zero-sum payoff matrices and directly output mixed-strategy pairs.
It scores responses using normalized exploitability and audits how named-game label effects change between raw and matched chat interfaces. It trains Qwen3.5-9B on freshly generated and games using LP-label supervised fine-tuning or the label-free VERGE recipe, then evaluates transfer through .
The experiments include format-only training, a format-competent chat baseline, payoff-reading heuristics, and both pass@1 and verifier-selected best-of-four results. The paper also shows that payoff-range normalization can make dominated-action padding misleading and introduces a calibrated control with randomized core positions.
Critical strengths
1, Checkable scoring and bounded theoretical claims.
The exploitability certificate evaluates strategy pairs without choosing an equilibrium label, and Theorem 1 bounds the effect of payoff perturbations on the unnormalized residual. This gives a concrete basis for examining numerical rendering errors, although it does not establish an advantage of VERGE over SFT.
2, Controls address identifiable evaluation confounds
The chat-template baseline, format-only SFT control, identity-level uncertainty estimates, and padding calibration distinguish several sources of apparent improvement. These controls make the reported gains more interpretable than a comparison against raw prompting alone.
Critical weaknesses
1, The learning contribution lacks controlled attribution.
VERGE changes the likelihood ratio, clipping bounds, reward grouping, anchor term, and rollout/update modes together. The anchor also increases the likelihood of every sampled response, while the training budgets differ from SFT in optimizer updates and sampled responses. Although these differences are disclosed, the experiments do not isolate whether the residual reward, the update rule, or format adaptation drives its behavior. The results therefore support a particular recipe more directly than a general conclusion about residual-based equilibrium learning.
2, Size transfer remains fragile and insufficiently characterized.
In Table 2, SFT pass@1 falls from 33% at to 12% at , while VERGE falls from 21% to 7% and exhibits substantial variation across seeds. Under the calibrated padding test, moving the embedded core from fixed positions to randomized positions also reduces SFT performance considerably. Since training is evaluated on one backbone and principally one payoff generator, the evidence leaves unresolved how much transfer reflects a stable equilibrium computation rather than distribution-specific or positional behavior.
Questions for authors
1, See Critical Weakness 1
2, See Critical Weakness 2
3, The paper does not include Generative Adversarial Equilibrium Solvers [1], which also learns a mapping from game instances to strategy profiles without equilibrium labels. Could you include its formulation as an additional numerical reference under the same generator and exploitability score? Its numerical interface differs from the language-model interface, so the comparison should assess equilibrium approximation rather than prompt sensitivity.
[1] Denizalp Goktas, David C. Parkes, Ian Gemp, Luke Marris, Georgios Piliouras, Romuald Elie, Guy Lever, and Andrea Tacchetti. “Generative Adversarial Equilibrium Solvers.” ICLR, 2024.
4, How much of the training gain remains when all baselines receive an explicit minimax or Nash-equilibrium objective, with decoding and output constraints held fixed?
3 comments
Response to Reviewer poj8 (2/2): controlled attribution (W1/Q1) and explicit objective (Q4)
W1 / Q1. "The learning contribution lacks controlled attribution … whether the residual reward, the update rule, or format adaptation drives its behavior."
We ran a controlled comparison in which every trained arm takes 2,000 optimizer updates from the same base checkpoint with the same LoRA configuration, schedule and seeds (42, 123, 7), which removes the difference in optimizer updates between SFT and VERGE. The two VERGE arms run the released code with 16,000 sampled responses each and differ in one line: the reward is the exploitability residual in one and a payoff-blind validity reward in the other (1 for a valid answer, −0.25 otherwise). The arms, the evaluation games and the comparison were chosen before any run. First-sample success on 300 further games (sizes 2–7), mean over three seeds:
| Arm | First-sample success |
|---|---|
| Base model | 7.0% |
| Format-only SFT | 5.4% |
| LP-label SFT | 52.0% |
| VERGE, residual reward | 12.4% |
| VERGE, payoff-blind validity reward | 5.3% |
Holding the update rule, sampling budget and number of optimizer updates fixed, replacing the residual reward by the payoff-blind validity reward gives 100.0% valid outputs and lowers first-sample success from 12.4% to 5.3% (difference 7.1 points, 95% CI [4.1, 10.1]; game-level bootstrap, 20,000 replicates). The gain of this recipe over format adaptation is therefore attributable to the payoff-dependent reward. The validity-reward arm lands at the level of format-only SFT (difference −0.1 points, 95% CI [−3.4, 3.1]): a reward that never reads the payoffs produces format adaptation and nothing more.
Q4. "How much of the training gain remains when all baselines receive an explicit minimax or Nash-equilibrium objective, with decoding and output constraints held fixed?"
The full LP-label SFT margin remains. Each arm in this comparison received the sentence "Your objective is to output a Nash equilibrium of this zero-sum game: minimax-optimal mixed strategies for both players, such that neither player can improve their expected payoff by unilaterally changing strategy", under the chat template, four samples at T=0.7 and top-p 0.95, a 128-token cap and one parser. On the 300 unseen games, with the sentence the three SFT seeds solve 157 / 164 / 154 games on the first sample, format-only SFT 17 / 16 / 15, and the base model 21; without it, 154 / 157 / 157, 17 / 16 / 16 and 21. The margin of LP-label SFT over the base model is 45.0 points without the sentence and 45.8 with it (95% CI [41.6, 50.0]); over format-only SFT it is 46.6 and 47.4 (95% CI [43.1, 51.9]). Both intervals exclude zero, and the retained fractions of the two margins are 1.02 and 1.02. Stating the objective does not produce the capability: the base model solves 21 games either way. The same holds on the paper's 300 evaluation games (first-sample successes, original → explicit: base 25 → 20; LP-label SFT 167 / 160 / 157 → 166 / 159 / 155). With the sentence, the VERGE seed-7 adapter of Table 2 still holds a 22.7-point advantage over the base model on the unseen games (89 versus 21 of 300 games; 113 without the sentence; Appendix A.8). Output lengths are matched: 53–56 generated tokens per response for the base model and 60 for the SFT adapters, under the same cap.
Addition to W2 / Q2 (second payoff generator). On 300 games from a different generator defined in the released code (rounded Gaussian payoffs instead of uniform integers), the three SFT adapters solve 190, 180 and 184 games on the first sample against 26 for the base model (difference 52.9 points, 95% CI [48.2, 57.4]).
The revised PDF (new text in blue) reports these results in Section 4.4, Appendix A.7 (Tables 11–12) and Appendix A.8 (Table 13).
Thanks for the detailed responses and additional experiments, which have addressed my main questions and concerns. I plan to raise my score from 2 to 3.
Response to Reviewer poj8 (1/2): numerical reference (Q3) and size transfer (W2/Q2)
Thank you for the detailed and constructive review. This first comment covers Q3 and W2 / Q2.
Q3. "Could you include [GAES] as an additional numerical reference under the same generator and exploitability score?"
Yes. We implemented a GAES reference following its generator–discriminator formulation (cumulative-regret objective, stochastic exploitability descent) and its reported normal-form architecture and optimization settings: generator MLP with four hidden layers of 1,024 units and a player-wise softmax, Adam at 1e-3, batch 50, 10,000 iterations, payoffs normalized to [0, 1], five seeds, four discriminators. Two settings are adaptations: the fixed split has 1,700 training and 200 validation games per size (that work describes 2,000 games per class with 200 for validation), and the learned-discriminator schedule is taken from other sections of its appendix because the normal-form section does not specify one. The reference is trained separately at each size on our generator, on games disjoint from all test games, and scored with our normalized exploitability and the 0.10 threshold. We report the true best-response discriminator at the final iterate, with no tuning and no selection over seeds. Mean success over five seeds in this configuration (fixed training set):
(2×2 uses 1,000 unseen games.) Two readings follow. First, on a fixed split of 1,700 games under 10,000 outer updates with batch size 50, this numerical reference also declines with size. Second, on the 50 Table-2 games, SFT's verifier-selected best-of-four rates (61%, 50%, 32% at 5×5–7×7) are numerically close to this reference (33.0, 25.6 and 14.6 of 50 games: 66.0%, 51.2%, 29.2%). The protocols differ: GAES reads exact real-valued payoffs and is trained at each test size, whereas the language models read a one-decimal rendering and are trained only at 2×2–3×3. The revised PDF reports this reference in Section 4.4 and all 240 training runs (six sizes × four discriminators × two data regimes × five seeds), also evaluated at the 0.05 threshold, in Appendix A.9.
W2 / Q2. "Size transfer remains fragile and insufficiently characterized."
The decline with size is part of the reported result: SFT pass@1 goes from 33% at 5×5 to 12% at 7×7. The numerical reference above declines with size on the same generator. With scoring at the embedded game's precision, randomized-position success remains above the core-uniform control: 32.7% across the three adapters' evaluations (49/150 12×12 games), versus 12% (6/50), with a game-level difference of 20.7 points, 95% CI [6.7, 34.7].