- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
本文提出了 Metis 框架,旨在提升卡牌游戏中的战略决策能力,并生成明确的推理过程。该框架结合了混合监督微调(SFT)、带有结构化奖励的组相对策略优化(GRPO),以及利用经行动验证的推理轨迹所进行的迭代自蒸馏。
在“掼蛋”和“斗地主”游戏中的实验表明,该框架在对抗基于规则的对手时,胜率分别达到了 78% 和 76%,优于所评估的其他大语言模型(LLM)及训练基线。消融实验考察了奖励组成部分、推理格式以及蒸馏周期等要素。然而,目前的验证主要集中在游戏表现和行动有效性上,对于推理质量或推理忠实度,尚缺乏充分的直接证据。
Critical strengths
-
The soft-matching reward provides partial credit for matching card types and attributes, offering a practical way to alleviate reward sparsity in large discrete action spaces. The ablation results support its effectiveness.
-
The self-distillation pipeline filters generated reasoning traces through action consistency and an LLM judge, providing a practical approach to constructing CoT supervision without human annotations.
Critical weaknesses
-
The reward function focuses on action matching and formatting rather than reasoning correctness. The generated CoT may simply rationalize decisions instead of reflecting genuine strategic reasoning.
-
The evaluation relies heavily on weak opponents. Metis achieves only a 21% win rate against DouZero (Table 8), questioning its actual strategic competence.
-
The contribution of self-distillation remains unclear. Without compute-matched baselines, the improvements could simply result from additional training iterations and supervision.
-
Only 100 matches are evaluated per configuration, without confidence intervals or multi-seed experiments.
-
The length reward lacks convincing justification. Removing it slightly improves win rates against rule-based opponents in both games (Table 3). More importantly, longer reasoning traces are not necessarily better reasoning.
Questions for authors
-
Without direct evaluation of CoT correctness, what evidence supports the claim of improved strategic reasoning rather than post-hoc rationalization?
-
Why is Metis considered competitive when it wins only 21% of games against DouZero (Table 8)?
-
How can the gains be attributed to self-distillation without controlling for additional training steps and data?
-
Are the reported gains statistically significant across multiple seeds, given only 100 matches per configuration?
-
Why is the length reward necessary when removing it improves win rates against rule-based opponents in both games (Table 3)?