- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents OppoMem: an Opponent-aware Memory framework for self-evolving LLM agents in strategic games. Oppomem utilizes an opponent-conditioned retrieval mechanism that dynamically selects memories based on online opponent modeling, together with a more effective memory evaluation mechanism that estimates the utility of individual memories in long-horizon decision-making scenarios. The paper presents extensive experiments across diverse game settings demonstrate the effectiveness of OppoMem and establish opponent modeling and memory utility evaluation as key components for building effective self-evolving agents.
Critical strengths
The paper presents a novel strategy where the memory is indexed by opponent type and retrieved by the agent first identifying the opponent type from observation.
Critical weaknesses
The paper did not present qualitative analysis and studies on variants of the setting, which could potentially lead to more interesting discoveries and applications (see questions).
Questions for authors
- Since the label is derived from the full trajectory while the agent determination is derived from oartial observations, does the author observe impact or failure mode at the earlier stage of the game when the data is less sufficient
- Are the range of opponents types modeled or studied? Can different LLMs have blund spot on particular opponents model? Can the model be given a prompt to introduce it to more exotic opponents models.
- How can the current setup extend to the consideration of the opponent agent using opponent models on the agent itself? What about nore levels of awareness.
- Can the framework be appointed to analyze the recent openai incident on huggingface? Eg a prisoner dillema between agent choosing to report to human the group jail break activity versus following the group.
7 comments
Response to Reviewer MHQ6 (2/4)
Scope of Opponent Types
Q2.1: Are the range of opponents types modeled or studied?
TL;DR: Opponent types are inferred from trajectories and clustered for memory retrieval, without a predefined taxonomy.
Action: We will include an analysis of this issue in the Appendix section.
Thank you again for your thorough observation and interesting question. First, please allow us to repeat our definition of opponent type. As you mentioned in question 1, opponent type is obtained from the complete trajectories collected, rather than being predefined.
Since different opponents may have the same behavior pattern, the same opponent may exhibit different behavior patterns at different stages, and the ego agent also explicitly models the opponent's behavior pattern semantically from a partial observation history, opponent types are very numerous and complex. It is precisely for these reasons that we cluster these opponent types and store them in embedding form for subsequent use in memory retrieval.
We do not assume a predefined, closed taxonomy of opponent types. On the one hand, the agents we test are LLM-based and usually show different behavior patterns in different games, and even show different behavior patterns in the same game, so opponent type has no ground truth. On the other hand, as in our response to your question 1, our method is more focused on how to help the ego agent make decisions through better memory retrieval, rather than on better obtaining opponent types. We present 8 representative opponent types summarized from RPS as a case study:
| Representative pattern | Fixed action | Biased mixture | Periodic cycle | Non-repetition | Lag-one imitation | Lag-one countering | Historical-frequency exploitation | Outcome-conditioned response |
|---|---|---|---|---|---|---|---|---|
| Description (paraphrased) | Always plays Rock, regardless of history. | Mostly plays Paper, with occasional Scissors and rare Rock; whether deviations are patterned or random is unclear. | Follows a fixed Rock → Paper → Scissors cycle. | Never repeats its own previous move. | Consistently copies ego's previous move in the observed rounds, with no other clear pattern. | Tends to counter ego's previous move, especially in rounds 8–19, with earlier exceptions. | Counters ego's most frequent past move; ties favor Scissors, then Paper, then Rock. | Repeats its winning move in the next round. |
Response to Reviewer MHQ6 (3/4)
Model-Specific Blind Spots
Q2.2: Can different LLMs have blund spot on particular opponents model?
TL;DR: Different LLMs do indeed have blind spots when identifying opponent types.
Action: We will add this analysis to the Appendix.
Very interesting question. For the RPS setting, we used 10 types of rule-based bots as opponents, tested each bot for 40 games, and each game had 20 rounds. We asked three LLMs to guess the opponent's type at rounds 10 and 20. We did not use predefined labels; instead, we asked the LLMs to describe it in natural language, and then used GPT-6.1-Sol to judge whether the guess was correct. We report the success rates at rounds 10 and 20 as follows:
Success rates at round 10:
| rule-based bot | Claude Sonnet 5 | Deepseek-v4.1-flash | GPT-5.6-Terra |
|---|---|---|---|
| Always plays the same move | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Favors one move (70%) | 45% (18/40) | 38% (15/40) | 75% (30/40) |
| Chooses R/P/S uniformly at random | 10% (4/40) | 5% (2/40) | 43% (17/40) |
| Cycles through R → P → S | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Copies ego's last move | 93% (37/40) | 100% (40/40) | 95% (38/40) |
| Counters ego's last move | 100% (40/40) | 100% (40/40) | 90% (36/40) |
| Counters ego's move from two rounds ago | 5% (2/40) | 45% (18/40) | 0% (0/40) |
| Counters ego's most frequent move in the last three rounds | 0% (0/40) | 15% (6/40) | 0% (0/40) |
| Repeats after a win; cycles forward after a loss or draw | 68% (27/40) | 73% (29/40) | 25% (10/40) |
| Copies ego's last move in rounds 1–10; counters it in rounds 11–20 | - | - | - |
Success rates at round 20:
| rule-based bot | Claude Sonnet 5 | Deepseek-v4.1-flash | GPT-5.6-Terra |
|---|---|---|---|
| Always plays the same move | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Favors one move (70%) | 53% (21/40) | 60% (24/40) | 83% (33/40) |
| Chooses R/P/S uniformly at random | 20% (8/40) | 0% (0/40) | 55% (22/40) |
| Cycles through R → P → S | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Copies ego's last move | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Counters ego's last move | 100% (40/40) | 100% (40/40) | 100% (40/40) |
| Counters ego's move from two rounds ago | 13% (5/40) | 88% (35/40) | 5% (2/40) |
| Counters ego's most frequent move in the last three rounds | 0% (0/40) | 20% (8/40) | 0% (0/40) |
| Repeats after a win; cycles forward after a loss or draw | 93% (37/40) | 100% (40/40) | 45% (18/40) |
| Copies ego's last move in rounds 1–10; counters it in rounds 11–20 | 100% (40/40) | 98% (39/40) | 75% (30/40) |
Overall, from these two tables, LLMs generally had a higher success rate in guessing at round 20 than at round 10. As you said in question 1, more trajectories allow LLMs to judge the strategy of a particular opponent more accurately.
Regarding your concern about whether different LLMs have blind spots, we also found some interesting conclusions. For example, Deepseek-v4.1-flash has difficulty guessing the correct strategy of an opponent that makes purely random moves, but instead performs well on a case such as “Counters ego's move from two rounds ago,” while GPT-type and claude-type models perform poorly. Similarly, we also found that for a relatively implicit strategy such as “Counters ego's most frequent move in the last three rounds,” all three models performed poorly. For more obvious strategies such as “Copies ego's last move” and “Counters ego's last move,” all three models performed well.
Prompting for More Exotic Opponents
Q2.3: Can the model be given a prompt to introduce it to more exotic opponents models.
TL;DR: Targeted prompts can improve the identification of the specific opponent strategies.
Action: We will add this analysis to the Appendix.
We conducted two types of prompt injection experiments. First, for the case where GPT-type and claude-type models did poorly at considering historical trajectories, we added “You need to consider a longer history” to the prompt and reran the experiment. Here, we report only the round-20 guessing results, as follows:
| rule-based bot | Claude Sonnet 5 | GPT-5.6-Terra |
|---|---|---|
| Counters ego's move from two rounds ago | 47% (19/40) | 40% (16/40) |
| Counters ego's most frequent move in the last three rounds | 5% (2/40) | 2% (1/40) |
We found that compared with the original prompt, targeted prompt injection did indeed bring a noticeable improvement.
Second, for Deepseek-v4.1-flash's inaccurate guesses for purely random moves, we added “You need to distinguish randomness from game strategies” to the prompt. The round-20 guessing result of Deepseek-v4.1-flash was still 0%. We tried several similar prompts; except when explicitly telling it that “the opponent's strategy is to make random moves,” Deepseek-v4.1-flash could not guess it in the other cases.
So from the above experiments, we can see that different models indeed perform differently when facing opponents with different strategies, and specific prompts can improve different models' ability to recognize opponent strategies.
Response to Reviewer MHQ6 (1/4)
Hi Reviewer MHQ6,
Thank you very much for your valuable time and detailed suggestion. We agree that examining OppoMem from qualitative analysis and studies could lead to more interesting discoveries and applications. In the rebuttal below, we provide additional experiments and analyses to address the questions you raised.
Early-Stage Opponent Modeling
Thank you again for taking a close look at our paper. Indeed, opponent type is a very important part of our method. Your question is indeed reasonable: in the early stage of a game, the agent may model the opponent inaccurately because it has only a partial observation history.
First, most of the games we focus on are long-horizon tasks, such as RPS and IPD in the environments we use, and our main optimization objective is the final win or loss of each game. The use of opponent modeling by LLM-based agents in these kinds of scenarios has been shown to significantly improve game ability, such as in Hypothetical minds [1].
Our ablation experiments also show that explicitly asking the agent in the prompt to model the opponent before making a decision can improve the agent's game ability; see the “w/o memory” row in Table 2.
Second, we also paid attention to the issue you mentioned when designing our method. So, before each decision, we let the agent explicitly update the previous round's opponent model based on the latest observation, to make it as accurate as possible, while also allowing it to output that it cannot determine the opponent model when the number of samples is insufficient.
We found a case in RPS against a rule-based bot to show the issue you mentioned. The opponent's strategy is to counter the move that our side has played most frequently in the historical trajectory. The changes in our guesses about the opponent's strategy over 20 rounds are as follows:
This trajectory shows that the agent initially struggles to identify the correct opponent type due to limited interaction history. As the history grows, counterexamples prompt the agent to revise its hypotheses until it identifies the underlying mechanism of the opponent’s strategy. Further observations help confirm and refine this understanding, which then guides the agent’s decisions and memory retrieavl.
[1] Cross L, Xiang V, Bhatia A, et al. Hypothetical minds: Scaffolding theory of mind for multi-agent tasks with large language models[C]//International Conference on Learning Representations. 2025, 2025: 6507-6546.