acceptodds
Withdrawn from ICLR 2027

SAW: Sink-Aware Weighting for Effective Video Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video reasoning requires linking evidence across time, yet providing frames spanning a video does not ensure that a multimodal large language model (MLLM) uses that evidence. We investigate frame-level attention sink, where response tokens concentrate attention on a small subset of frames. Our analysis shows that hallucinated tokens exhibit lower frame-attention entropy and higher peak attention than non-hallucinated tokens. We further derive an entropy-dependent bound on the single-frame approximation error of an attention head's visual contribution. Motivated by these findings, we propose Sink-Aware Weighting (SAW), which reduces the training influence of tokens with relatively concentrated frame attention. SAW converts frame-attention entropy into bounded token weights for supervised fine-tuning losses and reinforcement-learning advantages, while preserving the backbone architecture. With an 8B backbone, SAW achieves 64.99% average accuracy across five video question-answering benchmarks, 2.79 percentage points above the strongest evaluated open-weight baseline with complete results and comparable to GPT-4o (64.86%). On VideoHallucer, it improves pair-level accuracy by 7.60 percentage points over the strongest evaluated baseline. These results support attention-based reweighting for video reasoning and hallucination reduction. Data, code, and models will be made publicly available.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Video Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dSpMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SAW, a sink-aware weighting framework for video reasoning. The paper observes that hallucinated reasoning tokens tend to exhibit lower frame-attention entropy and higher peak attention, and uses frame-attention concentration as a proxy for attention sinks. Based on this signal, SAW assigns continuous token-level weights to both SFT losses and GRPO advantages, downweighting tokens whose attention is relatively concentrated on a small number of frames. The paper additionally constructs SAW-CoT by correcting hallucinated reasoning traces. SAW achieves strong results across several video reasoning and hallucination benchmarks.

Critical strengths
  1. Strong empirical results. SAW consistently improves over the Qwen3-VL-8B backbone on all five video QA benchmarks and achieves the best average performance among the evaluated open-weight models with complete results.

  2. Clear and well-designed experiments. The paper separately evaluates supervision refinement, sink-aware SFT weighting, standard GRPO, and sink-guided advantage modulation, making the contribution of each component relatively clear.

  3. Simple and practical method. SAW uses frame-attention concentration to compute bounded token weights and applies the same idea to both SFT and RL. It does not require any change to the backbone architecture, making the method easy to apply to existing models.

Critical weaknesses
  1. The definition of the sink score is not well justified. SAW treats low frame-attention entropy as sink-like behavior, but concentrated attention is not always bad. Some questions can be answered from only a few key frames, so focusing on these frames may be correct. More importantly, the score cannot tell whether the model is focusing on irrelevant frames or the correct key frames. Both cases receive a high sink score and are downweighted. The authors also acknowledge that attention concentration alone cannot distinguish useful focus from spurious sinks.

  2. The causal link between more distributed attention and better performance is not established. The paper shows that SAW makes frame attention more distributed and also improves accuracy, but this is only a correlation. It is still unclear whether the accuracy improvement is really caused by the more distributed attention, or by other changes introduced during training.

  3. The method generally favors more distributed frame attention without considering what type of evidence each question requires. Some questions need information from many frames, while others may only depend on a few local frames or object details. In the latter case, sparse attention can be correct rather than problematic. The bounded weighting may reduce the negative effect, but it does not solve this basic ambiguity.

Questions for authors

see weakness

Reviewer_vxMUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • SAW uses frame-attention entropy to assign bounded token weights during SFT and GRPO, reducing the training influence of tokens with relatively concentrated attention.
  • The pipeline also introduces SAW-CoT, a corpus of model-generated, corrected, and answer-filtered reasoning traces.
  • Evaluation on Qwen3-VL-8B covers five video QA benchmarks and VideoHallucer, with component, weighting-scope, and hyperparameter ablations.
Critical strengths
  • Simple, reusable training mechanism: The same weighting rule applies to both SFT and RL without changing the architecture or adding inference procedures.
  • Evidence beyond data refinement: With the SFT checkpoint and RL corpus fixed, SAW weighting improves the three-benchmark average by 2.00 points over GRPO. This supports an optimization benefit beyond corrected supervision.
Critical weaknesses
  • Limited methodological advance and mechanistic support: The main contribution is a heuristic entropy-based weighting rule. Attention concentration cannot distinguish legitimate focus from spurious sinks, and the theoretical bound does not establish a connection to hallucination. The evidence therefore supports a training heuristic more strongly than the proposed explanation.
  • The broader benefit remains insufficiently established: Evaluation uses one 8B backbone without multi-seed uncertainty or attention-collection overhead measurements. Comparisons against simpler weighting controls and a complete separation of data, SFT, and RL effects are needed to establish the method’s distinct value and likely impact.
Questions for authors
  • Clarification: How were the clipping radii and probe layer selected, and was a separate validation set used?
  • Assessment-relevant comparison: Does SAW outperform shuffled attention weights or other bounded weighting schemes with matched weight distributions? This would establish whether the attention signal itself is consequential.
  • Assessment-relevant comparison: Does SGAM improve over GRPO when both start directly from the original backbone, without SAW-CoT or weighted SFT?
  • Clarification: How stable are the gains across training seeds, and what additional training time and memory does attention collection require?
Reviewer_K3k2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces sink-aware weighted training for video reasoning. Based on the observation that hallucinated tokens tend to exhibit more concentrated frame attention, this paper proposes using frame-attention entropy to weight token-level SFT losses and GRPO advantages. This paper also reformulates videoR1 [A] cot data by correcting and filtering model-generated reasoning traces. Experiments on different video question-answering benchmarks and VideoHallucer show improvements in video understanding and hallucination reduction.

[A] Feng, K., Gong, K., Li, B., Guo, Z., Wang, Y., Peng, T., Wu, J., Zhang, X., Wang, B., & Yue, X. (2025). Video-R1: Reinforcing video reasoning in MLLMs.

Critical strengths

[+] The paper studies a relevant problem in video reasoning: providing multiple frames does not ensure that the model uses the available evidence. The token-level analysis offers a useful perspective on this issue.

[+] The proposed weighting scheme is simple and can be applied to both SFT losses and GRPO advantages without changing the backbone architecture.

[+] Table 3 provides controlled comparisons on the same SFT corpus and, separately, with the same RL initialization and data. These results support the incremental benefit of the proposed weighting.

[+] SAW improves performance across all five video QA benchmarks over its backbone and achieves a notable gain on VideoHallucer.

Critical weaknesses

[-] While Table 3 shows gains from attention-based token weighting, comparisons with shuffled weights and alternative signals, such as predictive entropy, are missing. It remains unclear whether the gains specifically depend on frame-attention concentration. These comparisons would help further clarify the benefits of the proposed weighting scheme.

[-] While Section 4.2 acknowledges that frame relevance varies across reasoning steps, the paper lacks validation across different reasoning token types. Logical reasoning may use previously extracted evidence, making concentrated frame attention difficult to interpret. A breakdown by token type, together with video-masking experiments before and after evidence extraction, would help distinguish attention sinks from valid reasoning over prior evidence.

[-] The relationship between the proposed frame-level attention concentration and the attention sinks studied in StreamingLLM [B] is not clearly explained. It would be helpful to clarify whether they share the same underlying mechanism and how the proposed measure distinguishes attention sinks from legitimate focus on relevant frames.

[-] The benefit of hallucination correction is not isolated from using a stronger teacher to provide CoT (Gemini-3.1pro). SAW-CoT combines Gemini-3.1pro regeneration, correction, and answer filtering. Comparing corrected and uncorrected Gemini traces on the same retained samples would help readers to better understand the contribution of explicit hallucination correction.

[B] Xiao, G., Tian, Y., Chen, B., Han, S., & Lewis, M. (2024). Efficient streaming language models with attention sinks.

Questions for authors

See weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.