acceptodds
Under review as a conference paper at ICLR 2027

MOBA-VL: Event-Localized Multi-Turn Reinforcement Learning for Real-Time MOBA Commentary

Abstract

Real-time commentary for Multiplayer Online Battle Arena (MOBA) esports requires a vision-language model (VLM) to narrate a live match second by second, both fluently and accurately. Existing streaming VLMs sound natural but often miss key events such as kills and objectives. To address this limitation, we use game telemetry, which records exactly when each event occurs, as a supervision signal. We introduce MOBA-VL, a 9B-parameter model trained on this signal with event-localized multi-turn reinforcement learning, which rewards the turns that describe each event. We also collect MOBACast, 860 professional matches (about 460 hours) across three MOBA games with word-level timestamped commentary, and MOBACast-Bench, a benchmark from held-out tournaments. On MOBACast-Bench, MOBA-VL achieves the highest Overall score on full matches (63.25 vs. 55.12 for StreamingVLM) and clips (63.45 vs. 56.22 for DeepSeek-V4.1-Flash). Event-localized credit also raises event recall from 34.5 to 42.1 over supervised fine-tuning. Code and data will be released. Anonymous project page: https://moba-vl.github.io.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
vision-language models, streaming video understanding, multi-turn reinforcement learning, event-level credit assignment, real-time esports commentary

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_TjjdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces MOBACast, a dataset comprising 860 professional matches (approximately 460 hours) across League of Legends, Dota 2, and Honor of Kings. It also introduces a model called MOBA-VL, a two-stage training framework for real-time MOBA commentary that combines supervised fine-tuning on professional commentary with event-localized multi-turn reinforcement learning. The ablation studies and evaluation results demonstrate the effectiveness of the proposed model.

Critical strengths
  1. A large-scale dataset with 860 professional matches (approximately 460 hours) that could be useful for future research on real-time MOBA game commentary.
  2. The authors propose a model using GRPO with event-localized credit assignment. The main results and ablation studies demonstrate the effectiveness of the proposed method.
Critical weaknesses

1 My main concern is that MOBA games receive frequent updates, including changes to equipment, characters, and game mechanics, which may cause the benchmark to become outdated over time. Therefore, the long-term usefulness of both the benchmark and the trained model may be limited.

Questions for authors
  1. The event evaluation allows a 30-second tolerance when matching generated commentary with ground-truth events. Since the task focuses on real-time commentary, I wonder whether this window is too large. Could the authors explain this choice or provide results with smaller time windows?
3 comments

We thank Reviewer Tjjd for the positive and constructive review. Following the suggestions, we introduce several new evaluations, detailed below. Our judge, DeepSeek-V4.1-Flash, was updated on October 17, and the earlier version is no longer available through the API. All new numbers in this response therefore use the updated judge and differ slightly from the paper.


W1. We agree that frequent updates to heroes, items and mechanics put real pressure on how long both the benchmark and a trained model stay current. A model that has never seen a hero can read its name from the draft, but may still not understand what the hero is doing on screen.

Benchmark and model can be refreshed. For LoL and Dota 2, the pipeline is automated and the RL stage needs no human labels beyond telemetry, and only HoK events are annotated by hand, so MOBACast-Bench and the model can be rebuilt from new tournaments. We will release the full pipeline. We acknowledge that a trained model can still become outdated between refreshes. An agent that retrieves the latest patch notes and player statistics at inference time may be a better long-term solution.


Q. We chose ±30 s as a lenient first pass, so that a correct but late summary of a fight is not penalized. For real-time commentary, tighter windows are indeed more informative. We recompute all results from the existing outputs at tighter tolerances, together with a strict rule under which the named killer must be the actual killer. Event latency is the time of the first correct mention minus the event time.

Table R1. Event latency and recall / one-to-one precision (%) under the strict killer rule, macro-averaged over the six clip subsets.

ModelMedian latency (s) ↓±3 s±5 s±10 s±15 s±30 s
SFT base3.018.8 / 64.222.7 / 67.327.9 / 72.929.6 / 74.331.0 / 74.5
+ Sequence RL2.325.2 / 68.528.9 / 72.333.8 / 76.535.1 / 77.136.8 / 78.0
+ Event-Localized (Uniform) (ours)2.327.5 / 69.431.3 / 73.135.7 / 76.637.6 / 77.839.9 / 79.4

The ranking is the same at every tolerance, and our method has the highest recall and precision even at ±3 s. In the revision, the strict rule with ±3 and ±10 s becomes the main event metric, and ±30 s is kept as a lenient reference.

Dear Reviewer,

Thank you for your review. The authors have submitted their rebuttal. Could you please take a moment to review their responses?

Thank you for your time and contribution!

Best,

AC

My concerns and questions have been addressed. I will keep my positive score.

Reviewer_rdpKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work is related to real-time commentary for Multiplayer Online Battle Arena (MOBA) e-sports. The authors commented that, although the existing streaming VLMs sound natural, they often miss key events such as kills and objectives. As a result, they propose MOBA-VL, which is a 9B-parameter model trained on this signal with event-localized multi-turn reinforcement learning. Its training consists of a two-stage framework where it combines supervised fine-tuning on professional commentary, together with event-localized multi-turn reinforcement learning. In order to demonstrate the effectiveness of their method, the authors collected 860 professional matches (about 460 hours) from three different MOBA games to create a benchmark, MOBACast. Each word in the commentary is timestamped.

The results show that MOBA-VL achieves the highest overall score on full matches and clips. Event-localized credit also raises event recall over supervised fine-tuning.

Critical strengths

The work addressed an issue of missing key events in real-time commentary for Multiplayer Online Battle Arena (MOBA). The authors propose a method, MOBA-VL, which has a two-stage training framework for real-time MOBA commentary that combines supervised fine-tuning on professional commentary and event-localized multi-turn reinforcement learning. The authors create a benchmark, MOBA, to help experiment with real-time commentary. It is based on 860 professional matches (about 460 hours) across three MOBA games with word-level timestamped commentary. The authors carry out meaningful experiments to show the effectiveness of the proposed method.

Critical weaknesses

[W1] The authors try to convince the readers of the superiority of their method by comparing their method with a prior approach. Without a gold standard for comparison, the pairs of narration do not tell anything or excite the readers.

[W2] According to Figure 2, it is an imbalanced dataset which is dominated by League of Legions (LoL), which is almost 5 times of Dota and 7 times of Honor of Kings (HoK). The result from this biased distribution may provide mislead the observation and come to wrong conclusions.

{W3] It was mentioned that each sentence is scored for gameplay relevance. If a score is above 4 (I have to assume it is good to have a lower scoring as it was not specified), the sentence will be replaced by DeepSeek-V4-Pro to become <silence>. Have these sentences <silence> been randomly checked for correctness? Also, who checked them? how many people? and inter-rater reliability?

[W4] The scores of CC and Fluency are the mean of running 3 times and judged by DeepSeek-V4.1-Flash. Why 3 times? Was it the best outcome after trying with different number of times? How sensitive are CC and Fluency under different of times? And, again, has the team randomly checked the fairness made by the VLM?

[W5] The two tables (Table 1 and 2) show that the method does not always come out as the champion, and CC for Dota2 in both full-match and clip level are not even the second best. Can you comment on what characteristics of these 2 games leading to current performance? Is it due to the imbalance of representation in the dataset?

[W6] Table 5 shows that Uniform leads to better performance than Gaussian. Although the authors commented that "By weighting the whole window equally rather than emphasizing tokens near the anchor, it may better support descriptions spanning several turns and depend less on precise localization" This answer is more a re-write of the observation that an answer.

[W7] The results in Table 3 is an aggregate from three different datasets (games). Because of the imbalance in distribution in your MOBACast, are these values obscured by the over representation of LoL? Are all games have similar results? Or some games better than others? Is it a macro- average? What about micro-average of different games?

Questions for authors

Please refer to the weakness mention in above weakness as well as

  • What is the min, max, average time length of each of the selected matches from each tournaments?
  • There is no future work or limitation discussed in the paper.
5 comments

We thank Reviewer rdpK for the careful review. Following the suggestions, we introduce several new evaluations and experiments, detailed below. Our judge, DeepSeek-V4.1-Flash, was updated on October 17, and the earlier version is no longer available through the API. All new numbers in this response therefore use the updated judge and differ slightly from the paper.


W1. We agree that the qualitative comparison needs an explicit factual reference. We will add timestamped telemetry events and the corresponding human commentary to Figure 1. Our quantitative event evaluation already uses official telemetry as ground truth. Against it under a strict rule at ±3 s, ProAct-VL reports 7.3% of events with 33.9% precision, versus 27.5% with 69.4% for our model.


W2. The imbalance reflects what is available. Professional LoL broadcasts with English commentary and machine-readable telemetry are far more common than HoK ones. Dota 2 broadcasts are frequently paused during matches, which breaks the alignment between video, commentary and telemetry and makes usable data harder to collect. We designed the evaluation so that this imbalance does not drive the conclusions.

Evaluation is balanced across games. The benchmark has separate full matches and clips for each game, every table reports each game separately, and the final Overall gives each game equal weight. Our model ranks first in Overall in every game, on both full matches and clips (Tables 1 and 2).

The RL gains hold in every game. Table R1 reports event recall per benchmark subset under the strict rule at ±3 s, with RL results averaged over 3 training runs.

Table R1. Event recall (%) per subset, strict rule at ±3 s.

SubsetSFT base+ Sequence RL+ Event-Localized (Uniform) (ours)
LoL skirmishes26.238.139.5
LoL teamfights15.817.119.5
Dota 2 skirmishes24.230.833.3
Dota 2 teamfights8.911.315.0
HoK skirmishes22.531.233.9
HoK teamfights15.422.823.9

Our method improves over SFT in all 18 combinations of training run and subset. The gain is as large on Dota 2 and HoK as on LoL, although HoK is only 6% of the RL training windows (954 LoL, 311 Dota 2, 84 HoK). On held-out human-verified events, the locator is within 3 s of the human-labelled mention for 88% of LoL, 88% of Dota 2 and 93% of HoK events.


W3. We first clarify the direction. A higher score means more relevant, and a sentence is replaced by <silence> only when its score is below 4. This removes 27.3% of sentences.

We now add a human check. Two people independently labelled 150 sentences, 50 per game, as relevant or not.

Table R2. Human check of the relevance filter on 150 sentences.

ComparisonAgreement (%)Cohen's κ
Annotator 1 vs annotator 273.30.31
Filter vs annotator 174.70.45
Filter vs annotator 266.70.18
Filter vs the 110 sentences both annotators agree on78.2

Relevance is partly subjective, and the filter agrees with each annotator about as often as the two annotators agree with each other. Of the sentences both annotators would remove, the filter removes 75%. The main error is over-removal, since 21% of the sentences both annotators would keep are also removed.


W4. Why 3 repeats. We fixed 3 repeats in advance for cost and did not tune this number on the results. To test sensitivity, we judged five systems 10 times each on the clips and resampled k of the 10 judgements per unit 2,000 times.

Table R3. CC and Fluency (mean ± sd) when each unit is judged k times.

SystemCC, k = 1CC, k = 3CC, k = 5Fluency, k = 1Fluency, k = 3Fluency, k = 5
MOBA-VL (ours)51.9 ± 0.7751.9 ± 0.4051.9 ± 0.2686.6 ± 0.8486.6 ± 0.4286.6 ± 0.28
GPT-5.6-Luna51.0 ± 0.8051.0 ± 0.4051.0 ± 0.2664.8 ± 0.9664.8 ± 0.4864.8 ± 0.31
DeepSeek-V4.1-Flash48.8 ± 0.8048.8 ± 0.4148.8 ± 0.2773.9 ± 1.1373.9 ± 0.5873.9 ± 0.36
ProAct-VL35.0 ± 0.6234.9 ± 0.3134.9 ± 0.2172.9 ± 0.7672.9 ± 0.3872.9 ± 0.26
StreamingVLM28.7 ± 0.6528.7 ± 0.3328.7 ± 0.2285.6 ± 0.8085.6 ± 0.4285.6 ± 0.27

The mean score does not change with k, only its spread does. With 3 repeats, as in the paper, the sd is at most 0.6 points, far smaller than the gaps between most systems.

Fairness of the judge. The judge is blind to the system. In a blind human study with 7 experienced players on 16 clips, DeepSeek agrees with the raters on 75% to 86% of non-tied judgements.

W5. Our model ranks first in Overall in every game, but we agree that its CC on Dota 2 is not among the best, and we acknowledge that limited data is one cause. Our model has only 9B parameters, while the strongest CC scores on Dota 2 come from much larger models trained on far larger data, such as DeepSeek-V4.1-Flash with 552B parameters and the proprietary GPT-5.6-Luna, which carry more game knowledge. Dota 2 matches are also long, so its 104 hours of video yield only 138 matches, and fewer heroes and less commentary per hero enter training (Table R4).

Table R4. Hero coverage in the training data.

GameMatchesHoursHeroes seen in trainingPicks per heroWords per hero
LoL628334.013745.823,663
Dota 2138104.1107 of 12712.98,063
HoK9423.78111.63,126

On factual event reporting, the RL gains on Dota 2 and HoK are as large as on LoL (Table R1).


W6. We agree, and replace the explanation with a measured one. We added a telemetry anchor with Uniform kernel control and checked, on 91 human-audited events from held-out 2026 matches, how close each anchor is to the actual mention.

Table R5. Anchor and kernel under the strict rule, mean over 3 training runs (%).

Telemetry anchorSemantic locator
Anchor within 3 s of the actual mention72.589.0
Anchor within 9 s of the actual mention97.895.6
Recall at ±3 s, Gaussian kernel24.325.2
Recall at ±3 s, Uniform kernel27.027.5

With a Gaussian kernel, which concentrates credit within a few seconds of the anchor, the semantic locator helps because it is closer to the human-marked mention than the telemetry time (within 3 s for 89.0% versus 72.5% of events). With a Uniform kernel, which spreads credit over ±9 s, both anchors cover the actual mention for over 95% of events, so the locator adds little. In short, the Gaussian kernel is highly sensitive to localization while the Uniform kernel is not, which is why Uniform performs better, and it does so in all 6 paired runs. We will replace the sentence in Sec. 4.3 with this analysis.


W7. Table 3 is a micro average over all pairwise votes. Table R6 splits it by game.

Table R6. Win rate (%) of our model per game.

SettingOpponentLoLDota 2HoKMicro (Table 3)Macro
Full matchGPT-5.6-Luna55.453.157.855.255.4
Full matchDeepSeek-V4.1-Flash65.564.355.563.561.8
Full matchClaude-Haiku-4.570.573.061.769.768.4
Full matchQwen3.5-27B83.895.480.586.286.6
ClipGPT-5.6-Luna66.362.071.766.566.7
ClipDeepSeek-V4.1-Flash73.968.071.771.871.2
ClipClaude-Haiku-4.589.182.082.685.684.6
ClipQwen3.5-27B81.580.087.082.482.8

Our model wins more than half of the comparisons in every game and setting, and micro and macro differ by less than 2 points, so the result is not driven by LoL.


Q1. Durations are video lengths after removing the draft phase.

Table R7. Match duration per tournament (minutes:seconds).

GameTournamentMatchesMinMaxMean
LoL2025 LPL Summer24521:5750:1231:44
LoL2025 MSI7624:3345:1832:05
LoL2025 Worlds8324:0958:1532:14
LoL2026 First Stand4420:2740:2429:57
LoL2026 LCS Lock In5823:0349:0632:47
LoL2026 LPL Winter12222:3545:5632:12
Dota 22025 TI13826:2783:2645:16
HoK2025 KWC945:5925:1715:06

Q2. We will add a section on limitations and future work.

Limitations. The main limitation is the trade-off between perception and memory under real-time constraints. Even with a 9B model on an H100 GPU, 720p input at 1 frame per second is already demanding, while streaming video understanding often benefits from higher resolution and frame rate, which is hard to provide without losing real-time speed. Maintaining long-range memory over a full match is also difficult, and balancing it with dense perception is harder still. In addition, game patches change heroes, items and strategies, so the knowledge learned from past matches can become outdated.

Future work. One promising direction is an agent that retrieves the latest information online at inference time, such as patch notes, player statistics and the hero preferences of each player, and uses it as a reference. This could keep the commentary current and improve its understanding of individual players.

Thank you authors for the extended explanation. Regarding [W3], if the inter-rator reliability is relatively weak, I am not sue whether we can make a proper evaluation when cohen's score is only 0.31 between two humans. As you have said, it is very subjective and has high variability, the authors should come with an assessment to reduce individual bias or other confounding. There are a few things that have been answered and are planned to incorporate the revision; however, the paper has already reached its limit, I am doubtful you have to include your "promises" in response to all reviewers.

Reviewer_SeYGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents MOBA-VL, a 9B vision-language model for real-time MOBA commentary. The model is first fine-tuned on professional commentary and then trained with multi-turn reinforcement learning using game telemetry as supervision. The main idea is to use separate NLI models to score event coverage and locate the turns that describe each event, then adjust token-level advantages around those turns. The authors also introduce MOBACast, a dataset of 860 professional matches across three MOBA games, and a benchmark containing full matches and event-dense clips from held-out tournaments. The reported results show higher event recall than SFT and sequence-level RL, and better overall commentary scores than the evaluated baselines. A periodic cache reset strategy supports real-time inference over long matches on a single GPU.

Critical strengths
  1. The authors study real-time video commentary for MOBA matches and propose MOBA-VL, which combines supervised fine-tuning with multi-turn reinforcement learning. The method uses game event logs as supervision. NLI models assess event coverage and locate the turns that describe each event, and these signals guide local adjustments to token-level advantages.

  2. The authors also introduce MOBACast, a dataset covering three MOBA games, and evaluate commentary quality, event coverage, and inference latency on full matches and event-dense clips. The main ablations compare sequence-level feedback, local feedback anchored to event timestamps, and local feedback guided by semantic localization.

Critical weaknesses

1、The text evaluations in the main tables remove timestamps and turn boundaries, and full matches are scored in 300-second windows. Event matching also allows a ±30-second tolerance, making it difficult to distinguish timely reporting from substantial delays. More seriously, the matching rules accept either the actual killer or a player credited with an assist. If A kills B and C assists, the statement “C killed B” may still count as correct. The evaluation therefore tolerates the kind of attribution error the paper claims to address. The inference latency results do not resolve this issue: the time needed to generate a turn is different from the delay between an event and its first correct mention. The current results show better event coverage under lenient matching rules, but do not establish that events are reported more promptly or accurately.

2、 The main results tables do not include SFT and Sequence RL comparisons using the same backbone, so it is unclear how much of the overall-score advantage comes from domain fine-tuning and how much comes from the proposed event localization mechanism. In Table 4, the best method improves recall from 38.6 to 42.1 over Sequence RL with the same reward, a gain of 3.5 percentage points. The increase from 34.5 to 42.1 also includes the gains from standard RL. In addition, the Telemetry anchor baseline uses a Gaussian kernel, while the best proposed variant uses a uniform kernel. The Gaussian comparison, 38.8 to 40.8, does provide evidence that localization helps. However, without a Telemetry anchor + Uniform control, the effects of semantic localization and kernel choice remain partly confounded in the best configuration. Only averages over three training runs are reported, with no individual run results or uncertainty estimates for the paired differences. This makes it difficult to assess whether the gains are consistent. The experiments therefore do not sufficiently establish that the proposed localization mechanism is a key source of the final performance gains.

3、The NLI scorer and locator are trained on automatically generated labels. Table 11 measures the locator’s agreement with automatic reference labels; it does not assess the reward scorer’s ability to detect factual errors. The paper also does not report the accuracy of the final event extraction pipeline against independent human annotations. The connection between automatic reward and evaluation scores and actual commentary quality therefore lacks independent validation. Appendix A reports a relevant failure mode: exploratory training beyond 100 updates produces repeated paraphrases of the same event, which the existing four-gram repetition penalty may miss. Precision is calculated separately for each claim, allowing multiple claims to match the same ground-truth event. Nonconsecutive repetitions of a correct claim may therefore still count as correct. The one-to-one matching used for recall prevents repetition from directly inflating recall, but unchanged precision does not rule out semantic repetition or a decline in output quality. Without independent factual checks and measurements of semantic repetition in the RL outputs, the evidence for reward reliability and improved commentary quality remains insufficient.

Questions for authors
  1. Does Uniform retain its recall advantage over Sequence RL under tighter temporal tolerances, such as 3, 5, or 10 seconds, when players credited with assists are no longer accepted as correct killer matches? Could the authors recompute precision and recall from the existing outputs and report the time difference between each event and its first correct mention, including cases where the outcome is announced before it occurs?

  2. Could the authors add a Telemetry anchor + Uniform control to separate the effects of semantic localization and kernel choice? Please also report the main quality metrics for SFT, Sequence RL, and the final method using the same backbone, along with the paired gains of Uniform over Sequence RL and their variability across the three training runs. These comparisons would help determine how much the proposed method contributes and whether its gains are consistent.

  3. Have the NLI reward scorer and the final event extraction pipeline been validated against independent human annotations? If so, what are the main error types and error rates? Given the semantic repetition reported in the appendix, could the authors report the repetition rate in outputs from the checkpoint at update 100 and check whether the conclusion that precision is maintained still holds after repeated claims about the same event are deduplicated?

6 comments

We thank Reviewer SeYG for the detailed review. Following the suggestions, we introduce several new evaluations and experiments, detailed below. Our judge, DeepSeek-V4.1-Flash, was updated on October 17, and the earlier version is no longer available through the API. All new numbers in this response therefore use the updated judge and differ slightly from the paper.


W1. We agree that the lenient rule cannot separate prompt and correct reports from delayed or misattributed ones. We now use a strict rule, under which the named killer must be the actual killer and an assisting player no longer counts.

Tighter tolerances and latency. Table R1 reports recall and one-to-one precision under the strict rule, macro-averaged over the six clip subsets as in Table 4. Event latency is the time of the first correct mention minus the event time, separate from computation time.

Table R1. Event latency and recall / one-to-one precision (%) under the strict killer rule.

ModelMedian latency (s) ↓±3 s±5 s±10 s±15 s±30 s
SFT base3.018.8 / 64.222.7 / 67.327.9 / 72.929.6 / 74.331.0 / 74.5
+ Sequence RL2.325.2 / 68.528.9 / 72.333.8 / 76.535.1 / 77.136.8 / 78.0
+ Event-Localized (Uniform) (ours)2.327.5 / 69.431.3 / 73.135.7 / 76.637.6 / 77.839.9 / 79.4

Our method has the highest recall and precision at every tolerance, and RL lowers the median delay from 3.0 s to 2.3 s.

Timestamps and 300 s windows in the text metrics. CC and Fluency judge what is said, so systems with different speaking rhythms are compared on the same text. The 300 s windows are chosen to evaluate overall content and long-range fluency, since longer passages reveal repetition and abrupt transitions that short ones hide.


W2. Same backbone. Table R2 reports SFT and sequence RL on the full-match split, with RL results as mean ± sd over 3 training runs.

Table R2. Full-match results of the SFT and RL checkpoints.

ModelCC ↑Fluency ↑DA ↑Overall ↑
SFT base54.479.853.762.63
+ Sequence RL54.981.253.163.08 ± 0.58
+ Event-Localized (Uniform) (ours)56.382.052.763.67 ± 0.52

Most of the gain over external systems comes from domain SFT, which already reaches 62.6. CC is judged against one human caster's commentary, so it depends heavily on what that caster chose to say. In this open-ended task, the full-match metrics compare long-range commentary rather than individual events within a few seconds. Reporting each event correctly and promptly is what localized credit optimizes, so its effect shows mainly in the event metrics (Table R1).

Telemetry anchor with Uniform kernel. We trained this control with 3 runs. Table R3 extends Table 4 with it, under the strict rule at ±3 s.

Table R3. Credit-assignment ablation with the new control, strict rule at ±3 s, mean over 3 training runs.

MethodRecall ↑One-to-one precision ↑Fluency ↑
SFT base18.864.284.6
+ Sequence RL25.268.585.0
+ Telemetry-Localized (Gaussian)24.366.285.6
+ Telemetry-Localized (Uniform)27.067.686.2
+ Event-Localized (Gaussian)25.266.084.2
+ Event-Localized (Uniform) (ours)27.569.485.9

We thank the reviewer for suggesting this control, which we had not considered and which changed our understanding of the method. We agree that the two factors were confounded. With a Gaussian kernel, which concentrates credit within a few seconds of the anchor, the semantic locator helps because it is closer to the human-marked mention than the telemetry time (within 3 s for 89.0% versus 72.5% of events, W3). With a Uniform kernel, which spreads credit over ±9 s, the locator adds little. This new finding shows that the kernel and the locator play different roles, and we will revise the claim accordingly. In short, the Gaussian kernel is highly sensitive to localization while the Uniform kernel is not, and the NLI scorer reward improves recall over SFT in every variant.

Run-level results.

Table R4. Recall (%) at ±3 s under the strict rule, per training run.

MethodRun 1Run 2Run 3Mean ± sd
SFT base18.818.8
+ Sequence RL23.427.125.225.2 ± 1.5
+ Telemetry-Localized (Gaussian)22.924.725.324.3 ± 1.0
+ Telemetry-Localized (Uniform)25.928.227.127.0 ± 0.9
+ Event-Localized (Gaussian)24.124.027.625.2 ± 1.7
+ Event-Localized (Uniform) (ours)27.125.030.427.5 ± 2.2

Runs with the same index share a data seed. The telemetry anchor with Uniform kernel beats sequence RL in every run.

W3. Locator. We replace the automatic references with a human audit on 2026 matches that appear in no training data, including the scorer's. Without seeing the locator output, we marked by hand the second at which the human commentary mentions each of 94 events. The locator finds 91 of them.

Table R5. Locator time versus the human-labelled mention time, on held-out 2026 matches.

SubsetEventsWithin 1 s (%) ↑Within 3 s (%) ↑
All9176.989.0
League of Legends5979.788.1
Dota 21770.688.2
Honor of Kings1573.393.3

Scorer. Instead of testing the scorer in isolation, we check whether training with it makes the outputs more factual, directly against telemetry. Precision at ±3 s rises from 64.2 (SFT) to 69.4 (Table R1), and the named killer is correct in 73.3% of matched kill claims versus 62.7% for SFT.

Event extraction and matching. We manually checked both steps against the commentary, without knowing which system produced it.

Table R6. Human check of the event evaluation pipeline.

StepSampleSFT base+ Sequence RLOursAll
Extraction is correct (%)90 claims90.080.090.086.7
Matching verdict agrees with human (%)90 event and system pairs90.090.093.391.1 (κ = 0.82)

Semantic repetition. DeepSeek-V4.1-Flash marks parts of the commentary that only repeat what was said earlier, without knowing which system produced it. Deduplication counts repeated claims of the same event as wrong, which is the one-to-one precision used in Table R1.

Table R7. Repetition and the effect of deduplication on precision.

ModelRepeated content (% of words) ↓Largest precision drop after deduplication, over all tolerances (points) ↓
ProAct-VL8.2
SFT base1.10.5
+ Sequence RL1.30.6
+ Event-Localized (Uniform) (ours), update 1001.20.4

Q1. All numbers are recomputed from the existing outputs. Under the strict rule, our method keeps its recall advantage over sequence RL at ±3, ±5, ±10, ±15 and ±30 s, and Table R1 reports the median time from each event to its first correct mention.

For mentions before the event, 18.2% of the first correct mentions of sequence RL come more than 3 s early, compared with 14.9% for SFT and 13.8% for ours. Table R8 no longer counts such early mentions as correct, and our method still leads.

Table R8. Recall (%) under the strict rule, counting only mentions from 3 s before to k s after the event.

Modelk = 3 ↑k = 5 ↑k = 10 ↑
SFT base18.820.924.6
+ Sequence RL25.226.829.0
+ Event-Localized (Uniform) (ours)27.529.532.0

Q2. Table R3 adds the telemetry anchor with Uniform control, Table R2 reports the main quality metrics with the same backbone, and Table R4 reports every training run. On the full matches, the Overall gains of our method over sequence RL for the three runs are +1.40, +0.49 and −0.14.


Q3. Table R6 gives the error rates of extraction and matching against human checks. The 12 extraction errors are 4 claims the commentary does not make, 4 misheard names, 3 cases where nobody died, and 1 wrong team. These errors are not biased toward our model. Table R7 gives the repetition rate of the update 100 checkpoint and shows that precision changes by at most 0.6 points after deduplication.

Thank you for the additional experiments. The new results alleviate some of my evaluation concerns, but two central issues remain:

  1. The independent benefit of semantic localization remains insufficiently established. With the Telemetry + Uniform control, the semantic locator adds only approximately 0.5 percentage points in recall, with paired gains of +1.2, −3.2, and +3.3 across the three runs. The original comparison changed both the anchor and the kernel, so it could not isolate the contribution of semantic localization. The additional results also do not establish a consistent incremental benefit.
  2. The timeliness claim still has measurement and interpretation issues. The paper assigns an event claim the start time of its grouped commentary line, which may precede the actual mention. The effect of this rule on evaluation under tight temporal tolerances needs clarification. Furthermore, the rebuttal interprets a reduction in median |d| as a reduction in reporting delay, but a smaller absolute temporal error does not directly establish earlier reporting. I therefore maintain my original rating of 2 (Weak Rejection) for now.
Reviewer_SWEZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces MOBA-VL, a 9B streaming commentary model trained through SFT on professional MOBA broadcasts and event-localized multi-turn RL. Telemetry-derived event hypotheses are scored by an NLI model, while a separate locator finds the commentary turn where support for an event increases most. A bounded, zero-mean adjustment redistributes token advantages around this anchor without changing the rollout's scalar reward. The paper also introduces MOBACast and a held-out-tournament benchmark, and uses periodic cache rebuilding to support streaming on a hybrid-attention backbone. Reported evaluations cover commentary quality, event precision/recall, latency, and an exploratory game-control application.

Critical strengths
  1. The central RL comparison controls important confounds. The SFT initialization, scalar reward, training windows, and optimization settings are held fixed across sequence-level RL, telemetry anchoring, and learned localization.

  2. The paper tests a concrete streaming implementation rather than assuming real-time operation. Cache rebuilding addresses the hybrid backbone's inability to evict recurrent state token by token.

Critical weaknesses
  1. The event metric establishes broad coverage more clearly than timely, non-redundant factual commentary. The paper measures computational latency separately, but a timely computation can still produce a delayed or repetitive event description. The narrower event-latency and redundancy behavior should be evaluated directly.
  2. The relationship between the credit-assignment gain and the headline full-match result is incomplete. The main CC/Fluency/DA tables compare the final domain-trained model against other systems, whereas Table 4 isolates RL variants on an event-focused clip evaluation. Reporting the SFT and sequence-RL checkpoints under the same full-match metrics would show how much of the headline gain is attributable to the proposed localization rather than domain SFT or other differences. Moreover, the benchmark has only six full matches, Table 4 reports means without run-level variability, and locator validation uses automatic references with some matches not necessarily held out from the scorer's training corpus. These limitations do not negate the result, but reduce confidence in its magnitude and robustness across matches.
Questions for authors
  1. Do the localized-RL gains persist with tighter event tolerances, signed event-delay reporting, and one-to-one precision matching or a separate semantic-repetition metric? Please include the sequence-RL baseline under those same rules.
  2. Can SFT, sequence RL, and localized RL be compared on the main full-match quality metrics with training-run variability and match-level uncertainty? This would connect the controlled ablation to the headline overall improvement.
  3. Can the locator be audited on human-verified events from matches disjoint from both scorer and locator training, including HoK? How sensitive are the downstream gains to the ±9-second kernel support and the 20% advantage-adjustment bound?
2 comments

We thank Reviewer SWEZ for the constructive review. Following the suggestions, we introduce several new evaluations and experiments, detailed below. Our judge, DeepSeek-V4.1-Flash, was updated on October 17, and the earlier version is no longer available through the API. All new numbers in this response therefore use the updated judge and differ slightly from the paper.


W1. We agree that low computational latency does not guarantee timely or non-redundant commentary. We now measure both directly in the same setting as the credit assignment ablation (Sec. 4.3, Table 4), with 300 telemetry events, 3 decoding seeds, and 3 training runs for each RL variant.

Event latency. For each telemetry event, d is the time of the first correct mention minus the event time. We use a strict rule, so the named killer must be the actual killer and not an assisting player. "Within k s" is the recall at ±k s (Table R5).

Redundancy. We use DeepSeek-V4.1-Flash to find parts of the commentary that only repeat what was said earlier. It does not know which system produced the commentary. We report how much of the commentary, measured in words, is labelled as repeated.

Table R1. Event latency and redundancy under the strict killer rule, macro-averaged over the six clip subsets.

ModelMedian |d| (s) ↓Within 3 s (%) ↑Within 5 s (%) ↑Within 10 s (%) ↑Within 15 s (%) ↑Repeated content, % of words ↓
ProAct-VL3.27.38.29.39.68.2
SFT base3.018.822.727.929.61.1
+ Sequence RL2.325.228.933.835.11.3
+ Event-Localized (Uniform) (ours)2.327.531.335.737.61.2

RL reports events sooner. The median delay drops from 3.0 s to 2.3 s.

RL does not make commentary more repetitive. Repeated content stays around 1% for all our variants.


W2. The main tables (Tables 1 to 3) are designed for comparison across systems. Their gain over external systems mainly comes from MOBACast and domain training, which is one of our contributions. To separate this from RL, we evaluate the SFT and sequence RL checkpoints on the full-match (long) split of MOBACast-Bench. RL results are mean ± sd over 3 training runs.

Table R2. Full-match results of the SFT and RL checkpoints.

ModelCC ↑Fluency ↑DA ↑Overall ↑
SFT base54.479.853.762.63
+ Sequence RL54.981.253.163.08 ± 0.58
+ Event-Localized (Uniform) (ours)56.382.052.763.67 ± 0.52

Our method improves Overall over SFT by 1.03 points, with a gain in all three training runs. It also improves over sequence RL by 0.59 points, and scores higher in 5 of the 6 matches.

These metrics are not designed to isolate event-localized credit. Following StreamingVLM, CC is judged against the human commentary rather than the video, since judging full matches from frames would be very costly. CC thus rewards matching what the human caster said, not reporting each event correctly and on time. The latter is what event-localized RL targets, and we measure it with event metrics against official telemetry (Table 4, Tables R1 and R3).

Six full matches. The full-match split is meant for overall quality. Event metrics use the clip split, which covers 40 games across three titles with 300 telemetry events, and every number in Tables R1 and R3 comes from it. On the full matches, our method scores higher than sequence RL in 5 of the 6 matches.

Run-level variability. Table R3 reports event recall under the strict rule at 3 s for each training run.

Table R3. Event recall (%) at 3 s under the strict killer rule, per training run.

ModelRun 1Run 2Run 3Mean ± sd
SFT base18.818.8
+ Sequence RL23.427.125.225.2 ± 1.5
+ Event-Localized (Uniform) (ours)27.125.030.427.5 ± 2.2

Our method improves over SFT in every run, and in all 18 combinations of training run and benchmark subset. We will add run-level sd to every row of Table 4.

Locator validation. We replace the automatic references with a new human audit on 2026 matches that appear in no training data, including the scorer's. Without seeing the locator output, we marked by hand the second at which the human commentary mentions each of 94 events. The locator finds 91 of them, and Table R4 compares its time with the human label.

Table R4. Locator time versus the human-labelled mention time, on held-out 2026 matches.

SubsetEventsWithin 1 s (%) ↑Within 3 s (%) ↑
All9176.989.0
Kills7677.688.2
Objectives1573.393.3
League of Legends5979.788.1
Dota 21770.688.2
Honor of Kings1573.393.3

Q1. Table R5 reports the strict rule at tighter tolerances, where a claim counts as correct only if it is within ±k s of the event. Precision is one-to-one, so repeated claims of the same event count as wrong. Under the same rule and tolerance, it is at most 0.2 points lower than per-claim precision, since only 0.3% to 0.6% of correct claims repeat an event.

Table R5. Event recall and one-to-one precision (%) under the strict killer rule, macro-averaged over the six clip subsets as in Table 4.

(a) Recall

Model±3 s ↑±5 s ↑±10 s ↑±15 s ↑
SFT base18.822.727.929.6
+ Sequence RL25.228.933.835.1
+ Event-Localized (Uniform) (ours)27.531.335.737.6

(b) One-to-one precision

Model±3 s ↑±5 s ↑±10 s ↑±15 s ↑
SFT base64.267.372.974.3
+ Sequence RL68.572.376.577.1
+ Event-Localized (Uniform) (ours)69.473.176.677.8

Our method has the highest recall and precision at every tolerance, and its recall gain over SFT is about 8 points at every tolerance, including ±3 s. The signed median delay is +0.5 s for SFT, −0.5 s for sequence RL and −0.2 s for ours, so our reports are not systematically late or early. The semantic repetition metric is in Table R1.


Q2. Table R2 reports this with run-level sd. Across the six matches, averaged per match rather than per game (mean ± sd over matches), our method gains +0.59 ± 0.66 over sequence RL, higher in 5 matches, and +0.84 ± 1.81 over SFT, higher in 4.


Q3. The human audit on held-out 2026 matches, including Honor of Kings, is in Table R4. For sensitivity, we train one extra run with the kernel support halved (σ = 1.5, ±4.5 s) and one with the bound doubled (0.4).

Table R6. Sensitivity to the kernel support and the advantage bound under the strict killer rule.

SettingRecall at ±3 s (%) ↑Recall at ±5 s (%) ↑Recall at ±10 s (%) ↑
SFT base18.822.727.9
+ Sequence RL23.426.731.2
Ours, default (±9 s, bound 0.2)27.131.135.3
Ours, support halved (±4.5 s)24.128.233.9
Ours, bound doubled (0.4)22.727.032.4

RL rows here use the same data seed (run 1). Both changes lower recall, and precision at 3 s drops by 7 to 8 points. Every setting stays well above SFT in recall, and the default works best.

Comments on the paper

1 comment

We thank all reviewers and the AC for their time. We appreciate that Reviewer SeYG raised their score from 2 to 3 after the discussion and that Reviewer Tjjd confirmed their concerns were resolved.

Strengths highlighted by reviewers. Reviewers Tjjd, rdpK, and SeYG considered MOBACast a useful dataset for future research, with 860 professional matches and word-level timestamped commentary across three games. Reviewers rdpK and SeYG highlighted the value of using game telemetry to address missed key events in streaming VLMs. Reviewer SWEZ highlighted our controlled RL comparisons and practical streaming implementation, including cache rebuilding for the hybrid-attention backbone.

Concerns addressed during the discussion. We conducted additional experiments and revised the paper to address the reviewers' concerns.

  1. Event accuracy and timeliness (SeYG, SWEZ, Tjjd). We adopted stricter event metrics (±3 s, strict killer matching, one-to-one precision). Our method still improves over SFT in recall (18.8 to 27.5) and precision (64.2 to 69.4). We further checked reporting delays using 1 s timestamps and measured semantic repetition.

  2. Method effectiveness (SeYG, SWEZ, rdpK). We added the suggested telemetry-anchor control with the Uniform kernel. This clarified the different roles of the locator and kernel. We also compared SFT, Sequence RL, and our method on full matches, reported run-level variability, and tested sensitivity to the kernel settings. We revised Section 4.3 accordingly.

  3. Human validation (SeYG, SWEZ, rdpK). We manually checked the locator on held-out 2026 matches, with 89.0% of located events falling within 3 s of human annotations. We also validated the event extraction and matching pipeline against human annotations, achieving 91.1% agreement for event matching (κ = 0.82). We extended the training relevance-filter check to 11 annotators and clarified that the earlier κ of 0.31 was unrelated to model evaluation.

  4. Dataset balance and judge validation (rdpK). We reported results separately for all three games, compared micro and macro averages, tested judge stability across different numbers of repeats, and conducted a blind human evaluation. These results address concerns about dataset imbalance and evaluation bias.

Paper revisions. We have included the new results, updated Table 4 and Figure 1, and added a limitations and future work paragraph in the revised paper. The main text remains within the page limit.

We thank the reviewers again for their constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.