- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors propose a novel task-hypergraph-based policy for reinforcement learning in real-time strategy (RTS) games and evaluate it on Gym--RTS. The proposed method demonstrates better generalization to unseen scenarios than several baseline methods, including CNN, CNN-Reg, DCPG, and GTG
Critical strengths
The paper addresses the important problem of generalization across different RTS initial configurations and offers a novel perspective on policy optimization.
The paper introduces a task-hypergraph-based representation of RTS entities and action spaces, including no-op, move, harvest, and return. This representation is novel to some extent, and the reported results suggest that the resulting reinforcement learning policy yields substantial performance gains. The hypergraph formulation for RTS games is also comprehensive, with detailed descriptions provided in the appendix.
Critical weaknesses
The experiments cover only Maps A–D and appear to focus on changes to individual factors, such as map layouts, building types, or worker types. Broader evaluations are needed to substantiate the paper's generalization claims. For example, the authors should test combinations involving different numbers and types of entities and different resource distributions, including scenarios in which two or more factors vary simultaneously.
The experimental results are reported primarily in terms of win rates, without confidence intervals, variance estimates, or other measures of statistical uncertainty. These are needed to assess the statistical significance and reliability of the reported improvements.
The paper should report training and inference costs, including computational resources and runtime. Such information would enable a more comprehensive comparison with CNN-based and other baseline methods.
Questions for authors
: How does the proposed method perform when multiple configuration factors vary simultaneously, as discussed in W1?
: Are the reported performance improvements statistically significant? Could the authors provide additional experimental results and statistical analyses, as discussed in W2?
: How do the end-to-end runtime and computational costs of the proposed method compare with those of the baselines? Are there significant differences in runtime or resource consumption? Could the observed performance gains be attributed, at least in part, to additional computational overhead, as discussed in W3?
4 comments
Weakness 3: W3.Missing computional cost analysis. The paper should report training and inference costs, including computational resources and runtime. Such information would enable a more comprehensive comparison with CNN-based and other baseline methods.
Response to Weakness 3: All baseline methods, including ours, were trained and evaluated on NVIDIA Tesla T4 GPUs. Each method completed training within 48 hours and evaluation within 4 hours. We omitted the hardware specifications and overall runtimes from the original manuscript because training and evaluation times were broadly comparable across methods.
To assess computational cost more directly, we conducted additional experiments focusing on the recursive and greedy solvers, which we expect to be the main sources of runtime variation within our method.
Solver runtime is determined entirely by the number of entities. More entities generate more task instances, which in turn increases solver runtime. As shown in Table 2, the recursive solver can exhibit exponential time complexity in the worst case, while the greedy solver’s runtime remains relatively stable. When there are many entities, the greedy solver can be used, since the win-rate gap between the two solvers is small, at about 3 percentage points (see Table 1 of the original manuscript).
Thank you for pointing this out. We will include this runtime experiments in the experiment section of the revised manuscript.
Table 2. Solver runtimes across entity counts. For each entity count, we report the average number of task instances and the runtime of the recursive and greedy solvers. The results are reported as mean ± standard deviation over three runs.
| # Entities | # Task Instances | Recursive Solver Time (ms) | Greedy Solver Time (ms) |
|---|---|---|---|
| 10 | 14 | 0.900 ± 0.049 | 0.325 ± 0.200 |
| 20 | 34 | 3.918 ± 0.307 | 1.268 ± 0.325 |
| 30 | 53 | 12.924 ± 5.358 | 1.405 ± 0.348 |
| 40 | 83 | 42.399 ± 40.796 | 1.863 ± 0.462 |
| 50 | 161 | 2630.485 ± 356.275 | 2.597 ± 0.539 |
Question 1: Q1.Compositional generalization: How does the proposed method perform when multiple configuration factors vary simultaneously, as discussed in W1?
Response to Question 1: Question 1 raises the same concern as Weakness 1. Please see our response to Weakness 1.
Question 2: Q2. Statistical significance: Are the reported performance improvements statistically significant? Could the authors provide additional experimental results and statistical analyses, as discussed in W2?
Response to Question 2: Question 2 raises the same concern as Weakness 2. Please see our response to Weakness 2.
Question 3: Q3. Computional overhead: How do the end-to-end runtime and computational costs of the proposed method compare with those of the baselines? Are there significant differences in runtime or resource consumption? Could the observed performance gains be attributed, at least in part, to additional computational overhead, as discussed in W3?
the first point of Question 3: How do the end-to-end runtime and computational costs of the proposed method compare with those of the baselines? Are there significant differences in runtime or resource consumption?
Response to the first point of Question3: The first point of Question 3 raises the same concern as Weakness 3. Please see our response to Weakness 3.
the second point of Question 3: Could the observed performance gains be attributed, at least in part, to additional computational overhead, as discussed in W3?
Response to the second point of Question3: In our evaluation, the recursive solver achieved a win rate approximately 3 percentage points higher than the greedy solver. As you pointed out, this improvement comes at a substantial computational cost (as shown in Table 2). However, in more complex settings, the greedy solver offers a practical trade-off: it reduces substantial solve time at the cost of a slight decrease in win rate. Thank you for pointing this out. We will emphasize this point in the main text of the revised manuscript.
Thank you for the authors' response. My concerns regarding W1 and W2 have been partially addressed. However, considering the novelty and potential impact of this work, I can only increase my rating to 2
Thank you for the feedback. Although the discussion period is nearing its end, we would like to take this opportunity to reiterate the novelty of our work. By transforming states into hypergraph representations, where multi-entity RTS tasks are modeled as hyperedges, our approach captures higher-order interactions beyond pairwise graphs or grid-based representations. This task-centered inductive bias enables the reuse of learned relational structures across changing entity configurations and spatial layouts, supporting strong zero-shot transfer to unseen maps. While we acknowledge that some aspects of our work, including the experimental design, could be improved, we respectfully believe that these limitations do not negate the novelty of our contribution.
We thank the reviewer for taking the time to review our manuscript. We think that there might be some misunderstandings about our paper and hope that our responses can address all your concerns. Below, we respond point by point to the weaknesses and questions raised in the review.
Response to Weakness 1: Our generalization claim is that “this paper studies compositional generalization across entity configurations. The training and test scenarios share the same entity types, action space, interaction rules, and task objective, but differ in their initial entity-type compositions and spatial layouts.” (line 40 of the original manuscript). Our evaluation therefore focuses on these two dimensions (initial entity-type compositions and spatial layouts). In both Map C and Map D, the initial entity-type composition and spatial layout differ simultaneously from those in the training scenarios. We acknowledge that broader evaluations would strengthen the empirical evidence, but the current experiments directly test the two dimensions of variation covered by our claim, including cases in which both vary together.
Response to Weakness 2: We acknowledge that our original experiments lacked sufficient statistical analysis. As shown in Table 1, we conducted additional experiments and report the mean and standard deviation of the results. Thank you for pointing this out. We will include this statistics results in the experiment section of the revised manuscript.
Table 1: Win rates (%) on the training and held-out test maps. Each method plays 40 games on each map. Test WR aggregates all four held-out maps (n= 160); Spatial WR aggregates Maps A and B (n= 80); Compositional WR aggregates Maps C and D (n= 80); and Worst-map WR reports the lowest win rate on a single held-out map (n= 40). Win rates are reported as mean ± standard deviation over three runs.