- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SHUGYO adapts a pretrained video diffusion model into an action-conditioned simulator, using separate buttons/mouse conditioning, persistent initial-frame context, and four-step Self-Forcing distillation. It then trains an instruction-following VLA policy inside that simulator using GRPO, a prompted vision–language reward judge, behavioral penalties, and a task curriculum. Validation includes single-chunk prediction benchmarks, simulator-component ablations, a simulator–real comparison across 14 tasks based on the video game Black Myth: Wukon, and policy evaluations in two real games Black Myth: Wukon and Honor of Kings: World.
Critical strengths
- The paper tests whether imagined training improves execution in the real environment. The policy is evaluated through the real game’s observation/action interface after training in the learned simulator, and improvements are reported in the two games. That experiment directly addresses whether the simulator is useful for learning behavior.
- The controlled second-game adaptation comparison provides evidence of reusable learned representations.
Critical weaknesses
- The central outcome measure is not sufficiently specified or independently validated. If the training judge also determines evaluation success, shared judgment errors or adaptation to its preferences could affect the measured gains. This is a possibility requiring validation, not demonstrated reward hacking.
- The experimental design does not establish a robust advantage for the proposed training framework over simpler alternatives. The SFT-versus-full-system comparison is promising, but uncertainty across training runs is unreported and the key curriculum, reward, and generated-experience alternatives are untested. Because the broad world-model-plus-GRPO recipe already exists in closely related work, convincing evidence for the game-specific execution and design choices is especially important.
Questions for authors
- Could you provide some kind of confidence intervals and/or independent training-run variability, second-game trial counts?
4 comments
Response to Reviewer zaUx (2/2)
Q1: Confidence intervals and training-run variability.
Response: We repeated the full RL stage with four random seeds, each using the same checkpoint selection procedure (the VLM judge selects the top-3 candidates, which are then evaluated by humans, and we report their mean). Results are shown below.
| Seed 1 (paper) | Seed 2 | Seed 3 | Seed 4 | Mean ± std | |
|---|---|---|---|---|---|
| Real-game success (human) | 68.6% | 72.1% | 70.0% | 68.6% | 69.8 ± 1.7% |
| Mean VLM-judge score | 0.621 | 0.650 | 0.628 | 0.614 | 0.628 ± 0.016 |
All four seeds exceed SFT (58.6%), and the seed reported in the paper ties for the lowest. The judge-score ranking matches the human-success ranking across seeds, so VLM pre-selection and human evaluation corroborate each other. The table is in the revised Appendix F.3 (Table 10).
With Wilson intervals, SFT reaches 58.6% (95% CI [50.3, 66.4]) and a single RL seed 68.6% ([60.5, 75.7]). Pooling the 560 trials of the four seeds gives 69.8% ([65.9, 73.5]). The difference from SFT has p ≈ 0.011 under a two-proportion test.
Q2: Trial counts on the second game.
Response: On Honor of Kings: World, each task is evaluated over 10 independent trials (100 in total). SFT reaches 52% (95% CI [42.3, 61.5]) and RL 74% ([64.6, 81.6]), with p ≈ 0.001 under a two-proportion test. The trial counts are stated in the revised Appendix F.4.
We hope these responses address the reviewer's concerns. Should any questions remain, we would be grateful if the reviewer could let us know, and we will respond as promptly as possible.
Dear Reviewer zaUx,
We sincerely thank you again for the constructive comments, which led us to add the real-game component ablations, the multi-seed results with confidence intervals, and the judge–human agreement study. As the author–reviewer discussion period is closing soon, we would be grateful if you could let us know whether these responses resolve your concerns about the evaluation protocol and the comparison with simpler alternatives. If you find that the concerns you described as minor or fixable have been addressed, we would kindly ask you to consider raising your Soundness and overall Rating scores. It would be our pleasure to clarify any remaining questions within the remaining time.
Dear Reviewer zaUx,
We understand that this is a busy period, and we sincerely appreciate your time. As the discussion period closes in a few hours, we would be grateful if you could take a brief look at our responses and the revised paper. If our additional experiments have addressed your concerns, we would kindly ask you to consider updating your Soundness and Rating scores. Thank you again for your constructive review.
Response to Reviewer zaUx (1/2)
We sincerely thank Reviewer zaUx for the thoughtful and constructive feedback, which will help improve our work. Our responses are as follows.
Response: All reported real-game success rates are obtained by human evaluation. The VLM judge used in training plays no part in any success decision. Human evaluation is performed by multiple evaluators blind to the model setting, following task-category-specific criteria (Appendix D.4). Shared judgment errors between the evaluator and the reward model therefore cannot enter the reported success rates.
The VLM judge is used only for rewards inside the simulator and for pre-selecting three candidates among the saved checkpoints. Pre-selection is the only coupling between the two, and we verified it. In an early stage of the project, we compared several commercial models (Gemini series and Qwen3.8) with human scores on n = 54 real-game rollouts (criteria in Appendix D.4):
Gemini-3.6-Flash agrees best with humans in ranking. Since GRPO depends only on within-group ranking, ranking agreement matters more than overall bias, which is why we chose it. On 200 human-annotated simulator rollouts sampled during RL training, Gemini-3.6-Flash reaches a Pearson correlation of 0.609 with 83% within 0.2. Moreover, across four RL seeds, the ranking by mean judge score matches the ranking by human success (see Q1 below). The full protocol and agreement analysis are in Sec. 5 and Appendix D.4 (Table 6) of the revision.
Response: We evaluated four simpler alternatives in the real game, each replacing or removing a single component of SHUGYO. All variants are initialized from the same WukongAgent-SFT checkpoint and evaluated by blinded human evaluators on the same 14 tasks (10 trials each). The results are summarized below.
Generated experience and optimizer are both necessary. Replacing either one eliminates the entire RL gain. With experience from Matrix-Game 3.0, continuous navigation even falls below SFT (76.7% vs. 83.3%), which is consistent with the policy fitting dynamics that do not transfer to the real game. Generic generated experience combined with GRPO is therefore insufficient. The gain requires a simulator aligned with the real game's control interface and dynamics. REINFORCE lacks the within-group baseline of GRPO and thus yields high-variance gradients under sparse trajectory-level rewards.
Curriculum and reward shaping act where they are designed to. Each contributes about 5 points. Without the curriculum, atomic tasks still improve but compound tasks drop below SFT (43.3% vs. 45.0%), so staged training is what carries atomic gains over to compound tasks. Without the penalties, the loss concentrates on navigation tasks in which overshooting or failing to stop directly causes failure.
The world-model-plus-GRPO recipe alone thus does not account for the improvement. The gain emerges only from the combination of an action-aligned simulator, group-relative optimization, a staged curriculum, and a stopping-aware reward. Per-category results are reported in the revised Appendix F.2 (Table 9).