- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes GroundedPlay, a lifetime-aware memory framework for streaming sports commentary. It maintains persistent entity memory, evolving event memory, and episodic memory to support player identification, action recognition, and timely commentary generation. Reciprocal Semantic Grounding (RSG) enables bidirectional interactions between player identities and actions, while Evidence-Completion Coupling (ECC) integrates event understanding with completion prediction. Experiments on NBA Streaming and MatchTime-SID demonstrate improvements in response timing, commentary quality, and processing throughput.
Critical strengths
The paper addresses a practical and challenging problem of real-time sports commentary requiring accurate player identification, event understanding, and timely responses.
The proposed framework integrates different semantic memory lifetimes with bidirectional interactions between identity, action, and event completion.
Extensive experiments demonstrate substantial improvements over existing baselines, particularly in CIDEr, identity accuracy, and event response timing.
Critical weaknesses
- Limited methodological novelty. The three-level memory organization, attention-based semantic interactions, and gated fusion mechanisms largely combine established techniques. The main contribution is their task-specific integration rather than a fundamentally new learning mechanism.
- Insufficient generalization. Evaluation is limited to basketball and soccer commentary, without demonstrating whether lifetime-aware memory generalizes to broader streaming video understanding tasks.
- Weak cross-domain evidence. Performance on MatchTime-SID remains relatively low (CIDEr 27.155, identity accuracy 5.556%, and response time difference 12.561 s), raising concerns about robustness beyond basketball.
- Limited analysis of memory efficiency. The paper does not sufficiently quantify memory consumption, retrieval overhead, or scalability to substantially longer video streams.
- Limited evidence for the episodic memory contribution. Removing episodic memory only slightly reduces CIDEr (320.016 to 312.693), suggesting that this component provides relatively modest benefits compared with entity and event memory.
Questions for authors
See the above
4 comments
Response to Reviewer mKvq (Part 2/2)
W5. Limited evidence for the episodic memory contribution.
Removing episodic memory only slightly reduces CIDEr (320.016 to 312.693), suggesting that this component provides relatively modest benefits compared with entity and event memory.
Response:
Episodic memory complements entity and event memory by retaining completed-event history for subsequent commentary. When it is removed, the model still has access to player identity evidence and current-action information. Its ablation therefore measures the additional value of historical context.
To examine this contribution more directly, we inspected the original play-by-play records and identified 128 annotations (3.47% of the 3,693 test annotations) that were the later event in an explicit action chain, such as a steal or rebound followed by a running shot, or an offensive rebound followed by a putback. On this subset, CIDEr increases from 313.447 without episodic memory to 327.237 with it, yielding a gain of 13.790, compared with 7.323 over the full test set. Although the aggregate improvement is modest, the larger gain on explicitly linked events supports episodic memory’s complementary role in describing plays that depend on preceding actions.
The authors have addressed my concerns. Therefore, I increase my score.
Thank you for your thoughtful review and for updating your assessment. We are glad that our responses have addressed your concerns.
Response to Reviewer mKvq (Part 1/2)
We thank the reviewer for the careful assessment and constructive suggestions.
W1. Limited methodological novelty.
The three-level memory organization, attention-based semantic interactions, and gated fusion mechanisms largely combine established techniques. The main contribution is their task-specific integration rather than a fundamentally new learning mechanism.
Response:
We agree that attention and gating are established building blocks. However, characterizing GroundedPlay solely as their task-specific integration overlooks how it couples memory retention, semantic grounding, and completion prediction during streaming inference. GroundedPlay uses persistent identity evidence to interpret ongoing actions, and the evolving action state helps determine when to respond. The completion predictor also guides which visual evidence is incorporated into that state.
Our methodological contribution lies in these dependencies and their associated retention rules, which determine what information persists across events and how it informs event understanding and response timing. Table 2 tests the consequences of changing the retention policies, while Table 3 examines the interaction directions. These controlled ablations support the proposed architectural and algorithmic design. The novelty claim concerns this organization of memory and inference, rather than the underlying attention or gating operators.
W2. Insufficient generalization.
Evaluation is limited to basketball and soccer commentary, without demonstrating whether lifetime-aware memory generalizes to broader streaming video understanding tasks.
Response:
This paper specifically studies streaming sports commentary, as stated in the title and contributions. We evaluate basketball and soccer to examine the proposed design across two sports, with ablations testing the effects of memory retention and semantic interactions. Evaluation beyond sports would be valuable, but would address the broader applicability of the approach rather than the task-level contribution investigated here. The abstract presents this broader applicability as a promising direction, not as an established experimental finding. Our results support lifetime-aware memory and reciprocal semantic interactions as an effective design for streaming sports commentary.
W3. Weak cross-domain evidence.
Performance on MatchTime-SID remains relatively low (CIDEr 27.155, identity accuracy 5.556%, and response time difference 12.561 s), raising concerns about robustness beyond basketball.
Response:
The absolute scores on MatchTime-SID should be considered alongside baseline performance under the same evaluation protocol. GroundedPlay achieves 3.75× the CIDEr of VideoLLM-Online, the strongest evaluated baseline on this metric. Using StreamMind as a shared reference, GroundedPlay achieves approximately 21.5× its CIDEr on NBA_Streaming and 9.4× on MatchTime-SID. These comparisons show substantial relative improvements in both sports. GroundedPlay also achieves the lowest response time difference and the highest identity and action accuracies among the evaluated methods on MatchTime-SID.
The lower absolute scores on MatchTime-SID may partly reflect differences between the two datasets. Compared with NBA_Streaming, MatchTime-SID contains more varied commentary and a larger player vocabulary. Its training and validation timestamps are automatically aligned, while test timestamps are manually aligned. These differences may contribute to the lower absolute scores. Despite the improvements over the evaluated baselines, player identification and response timing remain limitations requiring further improvement.
W4. Limited analysis of memory efficiency.
The paper does not sufficiently quantify memory consumption, retrieval overhead, or scalability to substantially longer video streams.
Response:
GroundedPlay bounds memory growth through memory-specific retention rules. Entity memory stores at most 128 entries per game. Event memory maintains a fixed-size recurrent state and action distributions, which are updated during the current event and reset at predicted completion. Episodic memory uses bounded storage and retrieves at most three completed outputs from the preceding 30 seconds.
We measured storage usage and retrieval overhead on the NBA_Streaming test set. The average retrieval time per call was 2.732 ms for entity memory and 1.155 ms for episodic memory. Their peak stored-data sizes were 0.514 MiB and 81.31 KiB, respectively. These measurements show low storage and retrieval costs on the evaluated streams. The fixed capacity limits keep memory storage bounded even for substantially longer video streams.