- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes VLNAPO for full-game RTS control in OpenRA. It combines a vision-language-numerical policy for macro-action generation, prefix-conditioned masks for action validity, and rule-based key-token supervision during PPO training. Experiments show that the proposed action constraints achieve 99.96% joint validity, while Harness supervision improves the win rate from 77% to 90%. The paper also includes ablation studies and evaluations on three unseen maps.
Critical strengths
The paper demonstrates strong performance in full-game OpenRA control, with a 90% win rate compared to 77% without Harness supervision. The results are evaluated across five random seeds, providing reasonable evidence of the system's effectiveness in the studied setting.
Critical weaknesses
-
The proposed framework relies heavily on OpenRA-specific action representations, feasibility constraints, and handcrafted rules. While the results demonstrate an effective RTS agent, it is less clear what generalizable principles or methodological insights emerge from these designs. The broader scientific contribution beyond this particular control setting needs to be better established. The Introduction identifies two gaps concerning invalid action generation and the transfer of external rule guidance into learned policies. However, it does not adequately explain why these limitations are particularly important or why existing approaches are insufficient. In particular, the need to remove external rule systems at inference, rather than simply retaining them, is not well justified.
-
Although the paper distinguishes Harness Engineering from teacher-based distillation, its core training mechanism remains PPO with auxiliary cross-entropy supervision on rule-generated labels. The paper does not establish whether its selective, online supervision offers meaningful advantages over simpler rule-based imitation or existing rule-guided learning methods. This makes its methodological novelty difficult to assess. Section 3.4 motivates Harness Engineering through sparse rewards, yet Appendix A.4 already introduces event-based rewards for construction, production, and economic development. While direct action supervision may provide complementary guidance, the paper does not clearly explain what learning difficulties remain unresolved by reward shaping or why Harness is needed to address them.
Questions for authors
- Which parts of the proposed method are generalizable beyond OpenRA, and what supports this claim?
- What is the advantage of learning rule guidance into the policy rather than retaining the rule engine at inference?
- What does the selective, online supervision in Harness Engineering offer over simpler rule-guided imitation learning methods?
- Given the existing event-based reward shaping, what additional learning difficulties does Harness supervision address?
- How does Harness compare with a no-Harness baseline on unseen maps, and do its benefits hold against different opponent strategies?