acceptodds
Under review as a conference paper at ICLR 2027

Beyond Successful Play: Diagnosing Opponent Modeling in Large Language Models

12 views

Abstract

Language-model agents are expected to adapt to others whose behavior changes through interaction. Yet adaptation may rely on learned behavioral patterns rather than theory of mind (ToM): the ability to infer others' mental states, including their beliefs and intentions. We introduce a benchmark and diagnostic framework for repeated hide-and-seek against algorithmic opponents with explicitly defined recursive belief updates. Access to their ground-truth beliefs and action probabilities allows us to distinguish adaptive play, action prediction, and belief tracking through live interaction, fixed-history evaluation, and controlled information interventions. With structured elicitation and behavioral supervised fine-tuning (SFT), we find improved play in several tested settings, but our diagnostic evaluations reveal remaining limitations in opponent prediction and belief tracking. We therefore test whether directly supervising opponent action probabilities can address this predictive deficit in Qwen2.5-7B-Instruct. Predictions improve when the model is told the opponent's type and parameter settings, but become less accurate when it must infer the opponent from interaction history without this information. Training on examples that also omit these details reduces the gap, although improvements vary across opponents. Is this limitation specific to smaller open-weight models, or does it also appear in a frontier model? GPT-5.4 also benefits from correct opponent beliefs: supplying them improves its predictions and choices, whereas deeper explicit belief reports and self-generated memory do not consistently improve forecasting from interaction history. This contrast suggests that using useful belief information is easier than inferring it autonomously, even for a frontier model. Together, our findings distinguish improved play and action prediction from reliable belief tracking, providing a framework for evaluating mentalizing and testing targeted improvements.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Theory of Mind, Opponent Modeling, Belief Tracking, Large Language Models, Cognitive Modeling, Interactive Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jXsLOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies opponent modeling in a repeated two-action hide-and-seek game. Using algorithmic opponents with known internal states, it evaluates game performance, action prediction, and belief tracking separately. The paper tests prompting, supervised training, and access to opponent information, finding that better play does not necessarily imply accurate belief tracking.

Critical strengths

The use of opponents with known internal states enables direct measurement of prediction and belief errors. The correct vs. shuffled belief intervention is a useful experimental control.

Critical weaknesses

The scientific contribution is too limited. The main conclusions are largely expected: successful behavior does not establish accurate belief tracking, useful information improves decisions, and matching training conditions to testing conditions improves prediction. Demonstrating these observations in one setting does not, by itself, provide a substantial research contribution. The paper does not identify a new mechanism, explain the conditions under which opponent modeling succeeds or fails, or establish a broadly useful improvement. The number of experiments does not compensate for their limited insight.

The diagnostic framework does not resolve the central question motivating the paper. In the chosen game, different beliefs and simple reactive strategies can produce the same actions. The paper acknowledges this limitation, but leaves the design of tasks that distinguish these explanations to future work. That distinction is central to establishing the framework’s scientific value. The paper also does not demonstrate that its diagnostics predict performance against unfamiliar opponents or that addressing the measured errors improves live interaction.

These are gaps in the contribution itself, not merely missing implementation details or additional statistical checks. Even assuming the reported results are correct and reproducible, the paper does not establish enough new knowledge to justify acceptance at a main ML conference as ICLR.

Questions for authors

What substantive new conclusion can researchers draw from this paper beyond the observations listed above? Which reported experiment establishes that conclusion? What evidence demonstrates that the proposed diagnostics provide useful information about performance in new interactions, rather than simply measuring additional errors within the same game?

12 comments
AuthorsOpenReview ↗

(1/6) Response for Weaknesses

The scientific contribution is too limited. The main conclusions are largely expected: successful behavior does not establish accurate belief tracking, useful information improves decisions, and matching training conditions to testing conditions improves prediction.

Thank you for asking what we learned beyond general intuition. Expecting in principle that good play can diverge from accurate belief tracking does not show when or how far they diverge once an agent’s own actions change what an adapting opponent does next. Benchmarks such as FANToM and ToMBench evaluate responses to supplied dialogues or scenarios (§1, lines 039–049; §2.1, lines 125–133), while repeated-game LLM studies (§2.2, lines 137–149) rely primarily on choices and payoffs—where very different forecasts can favor the same move. Algorithmic opponents with known belief updates fill that gap by letting us score payoff, action forecasts, and belief reports against simulator targets (as 69rs notes) and separate using opponent information from inferring it from history (as zux9 notes). Below we address each part of your critique:

  1. Payoff gains and internally consistent action chains coexist with inaccurate forecasts and belief reports (re: “successful behavior does not establish accurate belief tracking”). Across our evaluated sizes and reasoning settings, larger models and higher effort do not monotonically improve live ToM2 payoff (Fig. 2a,b; Fig. S1). Moreover, coherent intermediate outputs do not imply accurate opponent predictions: in three-action RPS against ToM2 (Appendix I.1, Table 3), GPT-5.4’s prediction-to-best and best-to-final fields each agreed on 1,000/1,000 turns while its next-move accuracy was 33.9% (near the one-third chance reference); in a 120-update probability-RL run (§5, lines 495–499; Appendix G.4), output-field consistency exceeded 99% while target-q MSE (0.11749) exceeded the constant baseline (0.07783). Similarly, behavioral SFT improved live Qwen2.5-7B payoff on several opponents (Fig. 2d, Fig. S3), yet on fixed-history belief probes both behavioral and five-field SFT checkpoints remained worse than a constant baseline (Fig. 3b,d).

  2. Separating rule execution and belief use from autonomous inference (re: “useful information improves decisions”). Rather than merely showing that extra hints help, §4.3 (Fig. 4) separates what GPT-5.4 can do with opponent information from what it recovers from play. With supplied recursive rules and self-generated prior states, 52/54 (96.3%) of scheduled noninitial action probabilities are within 0.001 of the simulator (46/54 for full state coordinates; Fig. 4a, Appendix F.1). Without update rules, replacing self-reported b with simulator bₜ on fixed histories reduces forecast MSE from 0.04319 to 0.01484 (Fig. 4d) and raises reference-best-action agreement from 66.7% to 96.3% (Appendix F.4), while supplying true rather than shuffled bₜ lowers expected regret by 0.29640 (95% CI [0.19726, 0.39106]; Fig. 4e, §5, lines 500–502). Yet from history alone, GPT-5.4’s belief MSE is worse than constant (0.02717 vs. 0.01258; Fig. 4c), and deeper nested reports or self-memory (L1–L3, Fig. 4f) do not reliably improve forecasts—locating an inference bottleneck under the tested conditions without claiming to identify the model’s internal mechanism.

AuthorsOpenReview ↗

(2/6) Response for Weaknesses

Demonstrating these observations in one setting does not, by itself, provide a substantial research contribution. The paper does not identify a new mechanism, explain the conditions under which opponent modeling succeeds or fails, or establish a broadly useful improvement.

  1. The benefits of mixed exposure did not transfer to the tested new opponent (re: “matching training conditions to testing conditions improves prediction”). Both against familiar ToM0/1/2 opponents (G1) and against our unseen GPT-5.4 opponent (G2), test-time inputs are history-only. Within the familiar family, mixed history-only exposure reduced live expected one-step regret from 0.275 to 0.144 (with lower forecast MSE for ToM1/2 but higher MSE for ToM0 in each role; Appendix H.1, Table 2; see Fig. 3f for fixed histories). Against GPT-5.4, however, the same mixed checkpoint had higher Brier error than specification-only training on identical replay prefixes (0.26037 vs. 0.22643, Δ = +0.03394 [0.02485, 0.04291]) and earned less in live play (0.00 vs. +1.97; §4.2, p. 8; Table 2). Matching test-time input format helped on familiar histories but failed to transfer when the opponent policy changed.

  2. Conditions of success vs. failure across opponent types and extension to a three-action game (re: “in one setting” and failure conditions). We agree that the binary game alone left generality open, so we added live deployments (G1, G2; Table 2) and three-action RPS (G3, G4; §4.3, p. 9, lines 460–470; Appendix I, Tables 3–6). Across 50-round hide-and-seek (Fig. S1, GRPO Fig. S4) and 50-round RPS (Table 4), models often exploit simpler non-recursive regularities—such as WSLS (+0.8 to +48.0 across flagship model-by-role cells in Fig. S1; +38.3/+33.4 for seeker/hider at the 600-update GRPO checkpoint in Fig. S4) and biased random / WSLS-cycle in RPS (+12.00 and +45.80 for GPT-5.4, Table 4)—whereas against adaptive ToM2, payoffs are negative across flagship hide-and-seek runs (−2.4 to −18.6, Fig. S1) and the GPT-5.4 action-only RPS benchmark (−2.65, Table 4). In the separate discrete prompting test, GPT-5.4 averaged −1.65 action-only and +0.15 structured (difference +1.80, 95% CI [−1.55, 5.15]; Appendix I.1, Table 3). In continuous-report RPS, GPT-5.4’s bₜ and qₜ reports trail uniform and frequency baselines across all ToM0/1/2 cells (Table 5). Supplying true bₜ in ten-round live RPS again lowered immediate regret from 0.072 to 0 (Table 6), extending the use-versus-inference distinction to three actions without claiming to extend SFT or RL results to RPS.

AuthorsOpenReview ↗

(3/6) Response for Weaknesses

The diagnostic framework does not resolve the central question motivating the paper. In the chosen game, different beliefs and simple reactive strategies can produce the same actions. The paper acknowledges this limitation, but leaves the design of tasks that distinguish these explanations to future work. That distinction is central to establishing the framework’s scientific value.

Can simple action rules explain the choices? In our 16 paired RPS histories, the last three joint moves and full-history joint move counts are identical, yet the two branches require different best responses (G4; §4.3, lines 467–470; Appendix I.3, p. 33, lines 1751–1764). Rules based solely on those matched cues cannot distinguish the branches; GPT-5.4’s forecasts yielded both optimal responses in 16/16 pairs with true beliefs, versus 1/16 from history alone and 0/16 with swapped beliefs—ruling out the matched cues as a sufficient explanation of the true-belief choices, though not all possible heuristics. Separately, in a 15-family, 94-candidate simulation pilot on ordinary probing histories (Appendix I.4, pp. 33–34), BIC recovered recursive ToM1/2 in 0/10 games, while a focal-action frequency proxy approximated ToM2’s bₜ with 0.00521 MSE (vs. 0.02095 uniform) on the five ToM2 games, illustrating how non-recursive proxies can closely track targets on ordinary play without recovering the generating mechanism.

Reviewer_zux9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces a repeated hide-and-seek benchmark where algorithmic opponents' internal beliefs and action probabilities are known. Then, the authors use the benchmark to evaluate payoff, opponent-action prediction, and belief tracking, respectively. By doing so, the authors can study whether the task performance in repeated interaction results from opponent modeling or simpler behavioral heuristics. The main result of this work is that LLMs can often use supplied opponent information effectively to better achieve the task. However, it is much harder and less reliable for LLMs to infer the same information only from the interaction history.

Critical strengths
  1. The diagnostic setup is quite clean. Because the algorithmic opponents have known internal states, the authors can directly compare the ground truth of beliefs and action probabilities with the beliefs and probabilities inferred by LLMs.

  2. This work makes a useful distinction between "using" and "inferring" opponent information. The information-intervention experiments indicate that using the correct opponent beliefs can improve decisions, while inferring those beliefs from interaction history is much harder.

Critical weaknesses
  1. The opponent setting is still largely closed-set. The LLM models are trained on a fixed family of opponents such as RB, QL, WSLS, and ToM0~2, and some training examples contain these opponent types or state schema. Therefore, in the setting where LLM models are given only interaction history, it may be just identifying a known opponent family (because LLMs have access to the opponent type information during the training) and estimating its state rather than truly open-ended opponent modeling.

  2. The benchmark is very simple and the transfer evidence is limited. The main task only has two actions, where different beliefs or heuristics can easily lead to the same behavior. The GPT-5.4 experiments are useful as a more open transfer test, but performance is not consistently strong, and ground-truth beliefs are no longer available in that setting.

Questions for authors
  1. In the history-only setting, the model has already been trained on a fixed set of opponent types and, in some conditions, has seen their type/state information during training. How much of the reported ability is closed-set opponent identification and state estimation, rather than open-ended opponent modeling?

  2. How well do the conclusions transfer to richer environments with larger action spaces, where belief-based reasoning and simple reactive heuristics are more likely to produce different behaviors?

  3. The GPT-5.4 live opponent is a useful step toward a more realistic adaptive opponent, but there is no ground-truth belief state in that case. Is there a way to evaluate whether the model is really doing opponent modeling?

6 comments
AuthorsOpenReview ↗

Response for Weakness 1

The opponent setting is still largely closed-set. The LLM models are trained on a fixed family of opponents such as RB, QL, WSLS, and ToM0~2, and some training examples contain these opponent types or state schema. Therefore, in the setting where LLM models are given only interaction history, it may be just identifying a known opponent family (because LLMs have access to the opponent type information during the training) and estimating its state rather than truly open-ended opponent modeling.

Thank you for raising this concern. Recognizing a familiar opponent family and estimating its state is a plausible explanation. To test whether the forecasting gain extends beyond those families, we evaluated both probability-SFT checkpoints against GPT-5.4, an opponent neither had trained against. Both received only the interaction history, and we compared them on identical replay histories as well as in new live games. The mixed-exposure checkpoint had higher next-action prediction error in both tests: its advantage against familiar opponents did not carry over to GPT-5.4 (G2; Appendix H.2, Table 2).

AuthorsOpenReview ↗

Response for Weakness 2

The benchmark is very simple and the transfer evidence is limited. The main task only has two actions, where different beliefs or heuristics can easily lead to the same behavior. The GPT-5.4 experiments are useful as a more open transfer test, but performance is not consistently strong, and ground-truth beliefs are no longer available in that setting.

We addressed these two limitations with complementary tests: (1) in three-action RPS, we scored continuous 50-round belief and action forecasts (Table 5) and tested 16 paired histories that match last-three moves and full-history counts while optimal responses differ (1/16 solved from history alone vs. 16/16 with true beliefs and 0/16 swapped; G4, Appendix I.3); and (2) against GPT-5.4, pairing shared-prefix replay with live play shows that the forecasting disadvantage was already present on identical histories rather than arising solely from live trajectory shift (G2, Appendix H.2, Table 2).

AuthorsOpenReview ↗

Response for Question 1

In the history-only setting, the model has already been trained on a fixed set of opponent types and, in some conditions, has seen their type/state information during training. How much of the reported ability is closed-set opponent identification and state estimation, rather than open-ended opponent modeling?

In-family replay cannot partition the original error reduction between recognizing a known family and estimating within-family state. Evaluating the same checkpoints under history-only inputs against GPT-5.4 tests directly whether the gain transfers outside the training family (G2; §4.2, p. 8, lines 427–431; Appendix H.2, Table 2). On identical GPT-5.4 replay prefixes (where constant 0.5 scores 0.25 and past-frequency scores 0.28189), Specification-only SFT scored 0.22643, whereas Mixed exposure worsened Brier error to 0.26037 before either policy’s moves could alter the trajectory, and earned less in live play (0.00 vs. +1.97). Thus, the history-only training gain did not transfer to this unseen opponent, and common replay rules out diverging live histories as the sole explanation.

Detailed results for G2. Transfer to GPT-5.4 (60 ten-round games per checkpoint and setting). Intervals use paired games for replay and independent arm resampling for live play (revised Table 2).

Evaluation / metricSpecification-onlyMixedΔ [95% CI]
Common replay: action Brier0.226430.26037+0.03394 [0.02485, 0.04291]
Live: action Brier0.232440.25736+0.02493 [0.01211, 0.03725]
Live: payoff per ten-round game+1.970.00−1.97 [−3.07, −0.87]
Reviewer_69rsOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines whether successful play in repeated games indicates genuine opponent modeling in LLMs. The authors introduce a hide-and-seek benchmark with algorithmic opponents whose beliefs and action probabilities are known, allowing them to evaluate game payoff, action prediction, and belief tracking separately.

Experiments show that larger models and more explicit reasoning do not consistently improve performance. Behavioral SFT improves game payoff but does not necessarily lead to accurate belief estimates. Direct probability supervision improves forecasting, particularly when opponent information is provided.

Critical strengths

The paper evaluates game payoff, action prediction, and belief tracking separately, rather than treating successful play as evidence of theory of mind. Using algorithmic opponents with known belief updates allows these abilities to be evaluated against simulator derived targets. This provides a more detailed assessment than game outcomes alone.

The information-intervention experiments are particularly informative. For GPT-5.4, supplying the true opponent belief substantially improves action prediction and best-action agreement. In a separate matched probability-SFT experiment, training on histories that sometimes omit opponent specifications improves history-only forecasting. Together, these results support the paper’s central distinction between using opponent information and inferring it.

Critical weaknesses

The benchmark is limited to a simple binary game. The experiments focus on a two-action hide-and-seek game, where different beliefs or simple behavioral heuristics can lead to the same actions. This makes it difficult to distinguish genuine belief tracking from learning action patterns. Evaluation on more complex games maybe useful to the conclusions.

Improvements in opponent prediction are not fully validated in live interaction. The probability-SFT experiments show improvements in forecasting on fixed histories. However, these checkpoints are not evaluated in matched live-game experiments.

Questions for authors

same as weakness. Evaluation on more complex games maybe useful to the conclusions. It is unclear the improved predictions translate into better decisions and game performance during actual interaction.

4 comments
AuthorsOpenReview ↗

Response for Weakness 1

The benchmark is limited to a simple binary game. The experiments focus on a two-action hide-and-seek game, where different beliefs or simple behavioral heuristics can lead to the same actions. This makes it difficult to distinguish genuine belief tracking from learning action patterns. Evaluation on more complex games maybe useful to the conclusions.

Thank you for highlighting this limitation. Adding a third action does not by itself remove behavioral ambiguity; the sharper test is holding specified behavioral cues fixed while the optimal response changes. In three-action Rock-Paper-Scissors (RPS), we built 16 history pairs with the same last three joint moves and the same full-history joint move counts, but different unique best responses (G4; §4.3, p. 9, lines 467–470; Appendix I.3, p. 33, lines 1751–1764). Rules using only those matched cues cannot distinguish the two branches (a separately scored first-order Markov comparator solved 1/16). With history and readout fixed, GPT-5.4’s forecasts yielded both optimal responses in 1/16 pairs from history alone, 0/16 with swapped beliefs, and 16/16 with true beliefs—ruling out the matched cues alone as an explanation of the true-belief choices while leaving broader history-dependent rules open. Separately, in 50-round live RPS against ToM0/1/2, GPT-5.4’s autonomous bₜ and qₜ reports remained less accurate than uniform and frequency baselines in all cells (G3; Appendix I.2, Table 5).

AuthorsOpenReview ↗

Response for Weakness 2

Improvements in opponent prediction are not fully validated in live interaction. The probability-SFT experiments show improvements in forecasting on fixed histories. However, these checkpoints are not evaluated in matched live-game experiments.

We agree that fixed-history improvements must be tested when predictions drive actual play. We deployed both probability-SFT checkpoints from Fig. 3e,f in matched closed-loop hide-and-seek (60 ten-round games per checkpoint): at each round, the model’s forecast q̂ₜ selected the action via the same one-step best-response rule, and that executed move updated the game history (G1; §4.2, p. 7, lines 374–377 and p. 8, lines 423–426; Appendix H.1, Table 2). We also ran a separate closed-loop belief intervention with GPT-5.4 in ten-round RPS under a shared readout rule (G3; §4.3, p. 9, lines 463–466; Appendix I.3, Table 6).

Detailed results for G1. Live hide-and-seek, using the Fig. 3e,f checkpoints. Differences are mixed minus specification-only; intervals use paired whole-game resampling (revised Table 2).

MetricSpecification-onlyMixedΔ [95% CI]
Forecast MSE vs. simulator probability0.042910.03247−0.01044 [−0.01765, −0.00205]
Expected one-step regret0.274840.14438−0.13047 [−0.17588, −0.08212]
Total payoff per ten-round game+0.90+1.30+0.40 [−0.53, 1.33]

Parallel live test in RPS — GPT-5.4 belief intervention (24 ten-round games/arm against ToM1/2, known readout; revised Appendix I.3, Table 6): Supplying true simulator belief bₜ versus history-only reduced all-output qₜ error from 0.03988 to < 10⁻⁸ (Δ = −0.03988 [−0.06304, −0.02393]; 0.03065 → < 10⁻⁸ with fallback scoring), lowered immediate expected regret from 0.07219 to 0 (Δ = −0.07219 [−0.11135, −0.04564]), and matched all 24/24 trajectories of the exact-qₜ immediate oracle, while ten-round cumulative payoff was +0.833 vs. +0.625 (Δ = −0.208 [−1.708, 1.500]). Note on immediate regret vs. ten-round return: in both live studies, the controller optimizes immediate one-step expected reward rather than long-horizon return (Appendix H.1, lines 1549–1551; I.3, lines 1765–1774); matching 24/24 immediate-oracle trajectories in RPS shows that the controller followed the same paths as the immediate oracle in these games, even though neither ten-round experiment established a cumulative payoff gain.

AuthorsOpenReview ↗

Response for Questions

same as weakness. Evaluation on more complex games maybe useful to the conclusions. It is unclear the improved predictions translate into better decisions and game performance during actual interaction.

In short, the three-action RPS experiments confirm that using supplied beliefs remains far more reliable than inferring them from history (16/16 vs. 1/16 on cue-matched pairs; Appendix I.3). In live play, both the matched probability-SFT deployment (Table 2) and the RPS belief intervention (Table 6) establish lower one-step regret on the agent’s executed moves without establishing a ten-round total-payoff gain.

AuthorsOpenReview ↗

Follow up on Rebuttal

Dear Reviewer 69rs,

As the deadline for the discussion period approaches, we are wondering if our rebuttal addresses your concerns. If there are remaining questions and concerns, we are happy to discuss further. Otherwise, we would very much appreciate it if you could consider updating your ratings.

Thanks! Submission 52990 Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response for All Reviewers

Thank you to all reviewers for your constructive feedback. Our work examines a core evaluation question for interactive LLM agents: when a model plays well in repeated games, when does that reflect tracking an adapting opponent’s state rather than exploiting behavioral regularities, and where does opponent modeling break down? The revised manuscript clarifies our contributions and scope in the Introduction (§1, p. 3, lines 108–119) and Discussion (§5, pp. 9–10, lines 474–527), adds audited hider reasoning cells and scaling structured-output cells across six opponents in Appendix D (Figs. S1–S2), and reports four new sets of closed-loop and three-action studies:

G1. Live play against familiar opponents. We deployed the matched Qwen2.5-7B probability-SFT checkpoints from Fig. 3e,f in 60 live ten-round hide-and-seek games per arm without opponent specifications, using the same one-step best-response rule. Mixed exposure reduced live qₜ forecast MSE from 0.04291 to 0.03247 (Δ = −0.01044, 95% CI [−0.01765, −0.00205]) and lowered expected one-step regret from 0.275 to 0.144 (Δ = −0.130, 95% CI [−0.176, −0.082]), while the ten-round cumulative payoff difference (+0.90 vs. +1.30) remained inconclusive (revised §4.2, pp. 7–8; Appendix H.1, Table 2).

G2. Transfer to a new opponent. Evaluating the same two checkpoints under history-only inputs against an unseen GPT-5.4 opponent (60 ten-round games per arm; scored on realized actions, not unobserved beliefs) showed that the gains against familiar opponents did not carry over: mixed exposure increased next-action Brier error on shared replay prefixes (0.22643 to 0.26037) and in new live games (0.23244 to 0.25736), while mean ten-round live payoff fell from +1.97 to 0.00 (revised §4.2, p. 8; Appendix H.2, Table 2). Because the forecast disadvantage was already present on shared histories, different live trajectories cannot fully explain this forecasting disadvantage.

G3. Three-action rock-paper-scissors (RPS). The distinction between using supplied beliefs and inferring them from play reappears in three-action RPS across separate studies (§4.3, p. 9, lines 460–466; Appendix I.1–I.3, Tables 3–6). In 50-round continuous-report play, GPT-5.4’s autonomous bₜ and qₜ estimates trailed uniform and frequency baselines in all ToM0/1/2 cells (Table 5), whereas in separate action-only play it earned positive returns on biased random (+12.00) and WSLS-cycle (+45.80) versus +0.35, +0.60, and −2.65 on ToM0/1/2 (Table 4). In a ten-round live intervention (24 games/arm; Appendix I.3, Table 6), supplying true bₜ under the known readout reduced qₜ error from 0.03988 to < 10⁻⁸ and one-step regret from 0.072 to 0. The forecast-driven controller matched all 24 paths of the exact-qₜ immediate oracle, without an established ten-round payoff gain. In a separate five-model discrete structured-prompting test against ToM2 (Appendix I.1, Table 3), only Llama 3.3-70B had a paired payoff CI excluding zero, while GPT-5.4 showed 1,000/1,000 consistency across reported chain links alongside 33.9% next-move accuracy (near the one-third chance reference). We did not repeat SFT or RL in RPS.

G4. Tests against specific heuristics. We constructed 16 RPS history pairs with identical last-three joint moves and full-history joint move counts, but different unique optimal responses (§4.3, lines 467–470; Appendix I.3). With prompt format and readout fixed, GPT-5.4’s forecasts yielded both one-step optimal responses in 1/16 pairs from history alone, 0/16 with swapped beliefs, and 16/16 with true beliefs—ruling out rules based solely on those matched recency and frequency cues as a sufficient explanation of the true-belief choices. Separately, in a 15-family, 94-candidate simulation pilot on ordinary probing histories (Appendix I.4), BIC selected recursive ToM1/2 in 0/10 games, while a focal-action frequency proxy approximated ToM2’s bₜ to 0.00521 MSE (vs. 0.02095 uniform) on the five ToM2 games.

Across both games, GPT-5.4 made better immediate decisions when given accurate beliefs even though its history-only belief reports trailed simple baselines, and against familiar ToM opponents mixed exposure lowered live one-step regret without transferring to an unseen GPT-5.4 opponent.

The new experiments help locate where opponent modeling breaks down. In both games, GPT-5.4 made better immediate choices when given the correct opponent belief, even though its own belief estimates from play trailed simple baselines. The live tests revealed a separate limit: mixed exposure reduced one-step regret against familiar opponents, but its forecasting advantage reversed against the unseen GPT-5.4 opponent. A payoff score alone cannot tell us whether a model can use an accurate belief, infer it from play, or carry a forecasting gain over to a new opponent. Our framework lets us test those questions separately.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.