- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies opponent modeling in a repeated two-action hide-and-seek game. Using algorithmic opponents with known internal states, it evaluates game performance, action prediction, and belief tracking separately. The paper tests prompting, supervised training, and access to opponent information, finding that better play does not necessarily imply accurate belief tracking.
Critical strengths
The use of opponents with known internal states enables direct measurement of prediction and belief errors. The correct vs. shuffled belief intervention is a useful experimental control.
Critical weaknesses
The scientific contribution is too limited. The main conclusions are largely expected: successful behavior does not establish accurate belief tracking, useful information improves decisions, and matching training conditions to testing conditions improves prediction. Demonstrating these observations in one setting does not, by itself, provide a substantial research contribution. The paper does not identify a new mechanism, explain the conditions under which opponent modeling succeeds or fails, or establish a broadly useful improvement. The number of experiments does not compensate for their limited insight.
The diagnostic framework does not resolve the central question motivating the paper. In the chosen game, different beliefs and simple reactive strategies can produce the same actions. The paper acknowledges this limitation, but leaves the design of tasks that distinguish these explanations to future work. That distinction is central to establishing the framework’s scientific value. The paper also does not demonstrate that its diagnostics predict performance against unfamiliar opponents or that addressing the measured errors improves live interaction.
These are gaps in the contribution itself, not merely missing implementation details or additional statistical checks. Even assuming the reported results are correct and reproducible, the paper does not establish enough new knowledge to justify acceptance at a main ML conference as ICLR.
Questions for authors
What substantive new conclusion can researchers draw from this paper beyond the observations listed above? Which reported experiment establishes that conclusion? What evidence demonstrates that the proposed diagnostics provide useful information about performance in new interactions, rather than simply measuring additional errors within the same game?
12 comments
(2/6) Response for Weaknesses
Demonstrating these observations in one setting does not, by itself, provide a substantial research contribution. The paper does not identify a new mechanism, explain the conditions under which opponent modeling succeeds or fails, or establish a broadly useful improvement.
-
The benefits of mixed exposure did not transfer to the tested new opponent (re: “matching training conditions to testing conditions improves prediction”). Both against familiar ToM0/1/2 opponents (G1) and against our unseen GPT-5.4 opponent (G2), test-time inputs are history-only. Within the familiar family, mixed history-only exposure reduced live expected one-step regret from 0.275 to 0.144 (with lower forecast MSE for ToM1/2 but higher MSE for ToM0 in each role; Appendix H.1, Table 2; see Fig. 3f for fixed histories). Against GPT-5.4, however, the same mixed checkpoint had higher Brier error than specification-only training on identical replay prefixes (0.26037 vs. 0.22643, Δ = +0.03394 [0.02485, 0.04291]) and earned less in live play (0.00 vs. +1.97; §4.2, p. 8; Table 2). Matching test-time input format helped on familiar histories but failed to transfer when the opponent policy changed.
-
Conditions of success vs. failure across opponent types and extension to a three-action game (re: “in one setting” and failure conditions). We agree that the binary game alone left generality open, so we added live deployments (G1, G2; Table 2) and three-action RPS (G3, G4; §4.3, p. 9, lines 460–470; Appendix I, Tables 3–6). Across 50-round hide-and-seek (Fig. S1, GRPO Fig. S4) and 50-round RPS (Table 4), models often exploit simpler non-recursive regularities—such as WSLS (+0.8 to +48.0 across flagship model-by-role cells in Fig. S1; +38.3/+33.4 for seeker/hider at the 600-update GRPO checkpoint in Fig. S4) and biased random / WSLS-cycle in RPS (+12.00 and +45.80 for GPT-5.4, Table 4)—whereas against adaptive ToM2, payoffs are negative across flagship hide-and-seek runs (−2.4 to −18.6, Fig. S1) and the GPT-5.4 action-only RPS benchmark (−2.65, Table 4). In the separate discrete prompting test, GPT-5.4 averaged −1.65 action-only and +0.15 structured (difference +1.80, 95% CI [−1.55, 5.15]; Appendix I.1, Table 3). In continuous-report RPS, GPT-5.4’s bₜ and qₜ reports trail uniform and frequency baselines across all ToM0/1/2 cells (Table 5). Supplying true bₜ in ten-round live RPS again lowered immediate regret from 0.072 to 0 (Table 6), extending the use-versus-inference distinction to three actions without claiming to extend SFT or RL results to RPS.
(3/6) Response for Weaknesses
The diagnostic framework does not resolve the central question motivating the paper. In the chosen game, different beliefs and simple reactive strategies can produce the same actions. The paper acknowledges this limitation, but leaves the design of tasks that distinguish these explanations to future work. That distinction is central to establishing the framework’s scientific value.
Can simple action rules explain the choices? In our 16 paired RPS histories, the last three joint moves and full-history joint move counts are identical, yet the two branches require different best responses (G4; §4.3, lines 467–470; Appendix I.3, p. 33, lines 1751–1764). Rules based solely on those matched cues cannot distinguish the branches; GPT-5.4’s forecasts yielded both optimal responses in 16/16 pairs with true beliefs, versus 1/16 from history alone and 0/16 with swapped beliefs—ruling out the matched cues as a sufficient explanation of the true-belief choices, though not all possible heuristics. Separately, in a 15-family, 94-candidate simulation pilot on ordinary probing histories (Appendix I.4, pp. 33–34), BIC recovered recursive ToM1/2 in 0/10 games, while a focal-action frequency proxy approximated ToM2’s bₜ with 0.00521 MSE (vs. 0.02095 uniform) on the five ToM2 games, illustrating how non-recursive proxies can closely track targets on ordinary play without recovering the generating mechanism.
(1/6) Response for Weaknesses
Thank you for asking what we learned beyond general intuition. Expecting in principle that good play can diverge from accurate belief tracking does not show when or how far they diverge once an agent’s own actions change what an adapting opponent does next. Benchmarks such as FANToM and ToMBench evaluate responses to supplied dialogues or scenarios (§1, lines 039–049; §2.1, lines 125–133), while repeated-game LLM studies (§2.2, lines 137–149) rely primarily on choices and payoffs—where very different forecasts can favor the same move. Algorithmic opponents with known belief updates fill that gap by letting us score payoff, action forecasts, and belief reports against simulator targets (as 69rs notes) and separate using opponent information from inferring it from history (as zux9 notes). Below we address each part of your critique:
Payoff gains and internally consistent action chains coexist with inaccurate forecasts and belief reports (re: “successful behavior does not establish accurate belief tracking”). Across our evaluated sizes and reasoning settings, larger models and higher effort do not monotonically improve live ToM2 payoff (Fig. 2a,b; Fig. S1). Moreover, coherent intermediate outputs do not imply accurate opponent predictions: in three-action RPS against ToM2 (Appendix I.1, Table 3), GPT-5.4’s prediction-to-best and best-to-final fields each agreed on 1,000/1,000 turns while its next-move accuracy was 33.9% (near the one-third chance reference); in a 120-update probability-RL run (§5, lines 495–499; Appendix G.4), output-field consistency exceeded 99% while target-q MSE (0.11749) exceeded the constant baseline (0.07783). Similarly, behavioral SFT improved live Qwen2.5-7B payoff on several opponents (Fig. 2d, Fig. S3), yet on fixed-history belief probes both behavioral and five-field SFT checkpoints remained worse than a constant baseline (Fig. 3b,d).
Separating rule execution and belief use from autonomous inference (re: “useful information improves decisions”). Rather than merely showing that extra hints help, §4.3 (Fig. 4) separates what GPT-5.4 can do with opponent information from what it recovers from play. With supplied recursive rules and self-generated prior states, 52/54 (96.3%) of scheduled noninitial action probabilities are within 0.001 of the simulator (46/54 for full state coordinates; Fig. 4a, Appendix F.1). Without update rules, replacing self-reported b with simulator bₜ on fixed histories reduces forecast MSE from 0.04319 to 0.01484 (Fig. 4d) and raises reference-best-action agreement from 66.7% to 96.3% (Appendix F.4), while supplying true rather than shuffled bₜ lowers expected regret by 0.29640 (95% CI [0.19726, 0.39106]; Fig. 4e, §5, lines 500–502). Yet from history alone, GPT-5.4’s belief MSE is worse than constant (0.02717 vs. 0.01258; Fig. 4c), and deeper nested reports or self-memory (L1–L3, Fig. 4f) do not reliably improve forecasts—locating an inference bottleneck under the tested conditions without claiming to identify the model’s internal mechanism.