- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses opponent modeling in imperfect-information games. It tries to find the most likely hierarchical structure of the opponent. It starts by constructing a reduced graph of the opponent's transitions, then tries to find a tree of a given depth that best fits the structure the opponent might have used, with respect to the structural entropy. This additional hierarchical information is then used as input to the actor-critic to improve the training against that specific opponent.
Critical strengths
The main idea of modeling the opponent using a learned hierarchical structure is an intriguing line of research, and I believe it is a good approach for modeling human players.
Critical weaknesses
In line 199, you state: "a standard constructive heuristic" without any reference. It is not clear whether the description that follows is the exact construction taken off-the-shelf or is proposed by the authors as a contribution. I think the description of this construction in Section 4.2 is weak, and a simple example would greatly benefit it. Several of my concerns about this description: Is the tree constructed from the game tree or from the graph constructed in 4.1? If it uses the graph from 4.1, then it is not clear how this general graph is transformed into a tree, as the authors only state, "On read we symmetrize and remove self-loops," but it is not clear how they do it, or even why that would be a good idea. Also, it is not clear to me what the "degenerate two-level tree of entropy " looks like, and whether the algorithm uses this tree only for comparison or modifies it to find deeper trees. Some of these concerns are addressed by pseudocode Algorithm 1, but these should still be explained in the main text.
The paper includes some unnecessary details that are not explained, particularly in Sections 4.3 and 4.5. For example, the exact network sizes used, which are not relevant to the method, or the hyperparameter settings such as 24 context prototypes, , and . I am not saying those details are not important, but if there is no specific theoretical reason why those were (and should be) selected, then they should appear in a hyperparameter section and in the method description, since it is more about hyperparameter tuning than about explaining the method. Ideally, an ablation study could show the effect of changing those hyperparameters, but I do not think it is necessary.
The description of Figure 2 does not seem to correspond fully to the figure itself, as it is not clear what (a), (b), (c), and (d) should correspond to in the figure itself.
The authors use the regime-level adjusted Rand index as their main comparison metric, without formally defining it (they rely only on its informal implications).
It is not clear whether the baselines the authors are using in their experiment are their own, or whether they have been used in prior work. Without these details, it is hard to assess the strength of their approach.
The authors train the Actor-critic, but it does not seem that they evaluate it in any experiment. My understanding is that the goal is both to learn the model and to learn how to play well against it with the actor, but this is not evaluated anywhere. If the actor serves only to guide the opponent's exploration, the authors should state this explicitly.
(minor) Based on the description, it seems the paper is for two-player games only, because line 159 defines a^op_t as the action of a single opponent rather than a collection. If that is true, the authors should state explicitly that they are considering only those games.
(minor) is not defined in Preliminaries, but I assume it is a sequence of actions and observations of the agent .
(minor) I think the NashOpponent in Leduc, in the implementation, is not actually Nash. I did not find any other obvious mistakes. This mistake is not relevant to the paper, as the authors do not use it for any evaluation, so it is not a problem, but it makes me skeptical about other claims in the paper.
Questions for authors
-
You define o_t as the opponent's observation at line 159. Then, in Section 4.1, you work with this concept. Does this mean that the algorithm is aware of the opponent's observation? Is it only during a pretraining phase, when you are not actually playing against the opponent? Otherwise, I do not understand how you get this information.
-
The opponent interaction graph construction uses cosine similarity between two observations. Does the environment define the observations, or are the observations somewhat learned in latent space? Because if they are defined, then the whole merging depends on how the author defined the observations. So the quality of the merging would directly correspond to the quality of the observation definition.
-
In Section 4.2, are you constructing the tree H from the graph you construct in Section 4.1? If so, how do you then remove the cycles from that graph?
Response: graph-to-tree construction, observations, and actor–critic evaluation
We thank the reviewer for the detailed questions. Several arise because our notation obscures what the learner observes and because the method section mixes principles with implementation constants.
1. Observation. In the revision we will replace ot by an explicitly learner-side decision context, specify its components for each environment, and give a concrete record → prototype → edge example.
The observations used for prototype merging are environment-provided feature vectors, not learned latent observations. Cosine merging therefore depends on their scaling and representation.
2. From graph to tree. The encoding tree is built from the interaction graph of Section 4.1, not from the game tree. Cycles in the interaction graph are not removed. Instead, an encoding tree is a nested partition of graph vertices: initially the root contains all vertices and every vertex is a singleton leaf. Bottom-up merges introduce internal nodes grouping vertices; the graph and its cycles remain unchanged.
At each step we choose a same-height merge that most decreases the structural-entropy objective. Thus the initial two-level tree is modified to produce deeper candidate trees; it is not merely a comparison baseline. Reading a directed interaction graph uses W=(A+A⊤)/2 and removes diagonal entries to obtain a well-defined undirected cut volume. This discards direction, which we will state as a limitation. We will cite the constructive heuristic and clarify that our contribution is its opponent-model instantiation and online maintenance. A worked example with a handful of vertices and one merge will be added.
3. Evaluation and baselines. The actor–critic is evaluated, not used only to guide exploration. Figure 11(a,b) reports stationary and switching returns of the learned actor; Section 5.3 additionally reports sample efficiency, success, transfer, and adaptation, while Figure 11(c) separately evaluates opponent-action log-loss. We will move the return comparison into the main text and state this distinction explicitly.
All learned baselines use our implementations under the same PPO trunk and optimization budget: PPO has no opponent encoder; DRQN is recurrent; MBOM and UAOM follow their published opponent-model constructions; SOM-adapted is our learned-constructor adaptation of Cao et al. (2026), not the original LLM-based system. We will label this clearly to avoid overstating it as the original baseline.
4. Presentation and scope. We will move architecture sizes and numerical hyperparameters to Appendix C; retain only the method-defining symbols in Section 4; fix Figure 2’s panel/caption mismatch; and define the level-wise ARI as ARI=max(RI)−E[RI]RI−E[RI], computed independently at each tree level against the corresponding planted partition. The depth-selection lift/cutoff will also be written explicitly.
The current formulation is single-opponent. We will state this scope explicitly. The undefined history symbol will be defined in Preliminaries. Finally, we will inspect and correct/remove the unused NashOpponent implementation and avoid calling any opponent “Nash” without an equilibrium computation. Thank you.