- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces MOBACast, a dataset comprising 860 professional matches (approximately 460 hours) across League of Legends, Dota 2, and Honor of Kings. It also introduces a model called MOBA-VL, a two-stage training framework for real-time MOBA commentary that combines supervised fine-tuning on professional commentary with event-localized multi-turn reinforcement learning. The ablation studies and evaluation results demonstrate the effectiveness of the proposed model.
Critical strengths
- A large-scale dataset with 860 professional matches (approximately 460 hours) that could be useful for future research on real-time MOBA game commentary.
- The authors propose a model using GRPO with event-localized credit assignment. The main results and ablation studies demonstrate the effectiveness of the proposed method.
Critical weaknesses
1 My main concern is that MOBA games receive frequent updates, including changes to equipment, characters, and game mechanics, which may cause the benchmark to become outdated over time. Therefore, the long-term usefulness of both the benchmark and the trained model may be limited.
Questions for authors
- The event evaluation allows a 30-second tolerance when matching generated commentary with ground-truth events. Since the task focuses on real-time commentary, I wonder whether this window is too large. Could the authors explain this choice or provide results with smaller time windows?
3 comments
Dear Reviewer,
Thank you for your review. The authors have submitted their rebuttal. Could you please take a moment to review their responses?
Thank you for your time and contribution!
Best,
AC
My concerns and questions have been addressed. I will keep my positive score.
We thank Reviewer Tjjd for the positive and constructive review. Following the suggestions, we introduce several new evaluations, detailed below. Our judge, DeepSeek-V4.1-Flash, was updated on October 17, and the earlier version is no longer available through the API. All new numbers in this response therefore use the updated judge and differ slightly from the paper.
W1. We agree that frequent updates to heroes, items and mechanics put real pressure on how long both the benchmark and a trained model stay current. A model that has never seen a hero can read its name from the draft, but may still not understand what the hero is doing on screen.
Benchmark and model can be refreshed. For LoL and Dota 2, the pipeline is automated and the RL stage needs no human labels beyond telemetry, and only HoK events are annotated by hand, so MOBACast-Bench and the model can be rebuilt from new tournaments. We will release the full pipeline. We acknowledge that a trained model can still become outdated between refreshes. An agent that retrieves the latest patch notes and player statistics at inference time may be a better long-term solution.
Q. We chose ±30 s as a lenient first pass, so that a correct but late summary of a fight is not penalized. For real-time commentary, tighter windows are indeed more informative. We recompute all results from the existing outputs at tighter tolerances, together with a strict rule under which the named killer must be the actual killer. Event latency is the time of the first correct mention minus the event time.
Table R1. Event latency and recall / one-to-one precision (%) under the strict killer rule, macro-averaged over the six clip subsets.
The ranking is the same at every tolerance, and our method has the highest recall and precision even at ±3 s. In the revision, the strict rule with ±3 and ±10 s becomes the main event metric, and ±30 s is kept as a lenient reference.