- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2(+1)
Read review
Paper summary
This paper proposes carrier–receiver computation sharing to accelerate the language decoder prefill stage of video-language models. The method computes Attention and FFN updates only for selected carrier tokens, while receiver tokens reuse the corresponding carrier updates. The author reports accuracy close to dense inference, with relative efficiency gains.
Critical strengths
-
The paper distinguishes between retaining a token’s representation and independently computing its update. Sharing updates while preserving separate residual states provides a potential alternative to token removal.
-
The experiments cover three model families and three established video understanding benchmarks.
Critical weaknesses
-
My primary concern is the completeness of the paper and the scope of the work. The presentation is difficult to follow, particularly because the method lacks sufficient motivation to establish the need for carrier–receiver sharing. In addition, there is substantial unused space around the tables on pages 7 and 8, and the main text does not reach nine pages. Taken together, these issues considerably weaken my impression of the manuscript’s completeness.
-
The evidence for the carrier assignment strategy is limited. The method assumes that similarity in projected hidden features identifies tokens whose operator updates can be shared. However, input similarity does not necessarily imply update similarity. The paper would benefit from providing evidence and observation on this point.
-
Part of the experiment is incomplete. For example, accuracy is evaluated at 32 frames, while the experiments at other frame counts report latency only. Consequently, the results do not establish whether quality preservation remains consistent as the visual sequence grows.
Questions for authors
-
What evidence supports the assumption that tokens with similar hidden states require similar Attention or FFN updates?
-
How does carrier–receiver sharing compare with independently optimized visual token skipping methods at matched computational budgets or measured latency? The zero-update ablation uses a schedule calibrated for sharing, which may not provide a fair comparison with skipping.
5 comments
Response (2/3)
W2 & Q1: Evidence supporting the assumption that input similarity implies update similarity.
To verify that input feature similarity indicates update similarity, we conducted two analyses on Qwen3-VL-8B using 70 representative videos: 40 from MVBench covering all tasks and 30 from VideoMME covering short, medium, and long durations.
1. Correlation between Input Similarity and Update Similarity
We measured Spearman rank correlations between input-feature cosine similarity and operator-update cosine similarity across sampled actual carrier–receiver pairs, using inputs and updates from the same shared layer. We computed correlations for each video and shared layer, averaged across layers within each video, and then averaged across videos separately for each benchmark.
Table A: Correlation between Input Similarity and Update Similarity
| Dataset | Operator | Spearman Correlation |
|---|---|---|
| MVBench | Attention | 0.7052 |
| VideoMME | Attention | 0.6985 |
| MVBench | FFN | 0.8939 |
| VideoMME | FFN | 0.8422 |
Taking FFN as an example, every token in a layer is processed by the same function:
The same function is applied to every token. Within a bounded input neighborhood,
where measures how much the FFN can amplify input differences. When this amplification is moderate, similar inputs produce similar updates, motivating computation sharing.
A concrete example occurs in MVBench video action_antonym:027. At FFN layer 34, receiver token 336 is assigned to carrier token 547. Their input cosine similarity is 0.9500, and the cosine similarity between the receiver's independently computed reference update and the carrier's update is 0.9449. The carrier-to-receiver update norm ratio is 0.9840, showing that the updates are close in both direction and magnitude. Copying the carrier update yields a normalized squared error of 0.1087, compared with 1 for a zero update.
A similar observation holds for Attention. In the same video, at Attention layer 23, receiver token 265 is assigned to carrier token 256. The receiver's independently computed update and the carrier's update have a cosine similarity of 0.9502, and the carrier's update is only 1.1% larger in magnitude. Sharing this update yields a normalized squared approximation error of 0.1008.
The consistent positive correlations support the premise that similar inputs yield similar updates and validate our design.
2. Update Approximation Error
We measured the distance between each receiver's true dense update and its assigned shared update. Keeping the selected carrier set unchanged, we compared our input-feature-based matching with random assignment from the same pool of valid carriers.
Table B: Update Approximation Error (Our Matching vs. Random Matching)
| Dataset | Operator | Match Error (Ours) | Random Error | Relative Error Reduction |
|---|---|---|---|---|
| MVBench | Attention | 0.689 | 0.995 | 30.8% |
| VideoMME | Attention | 0.695 | 0.996 | 30.2% |
| MVBench | FFN | 0.383 | 1.405 | 72.7% |
| VideoMME | FFN | 0.503 | 1.493 | 66.3% |
Compared to random assignment within the same carrier pool, our matching strategy consistently lowers the approximation error, providing empirical support for our assignment mechanism. This discussion is now in Section 4.6.
Response (3/3)
W3: More Experiments.
To verify quality preservation on longer visual sequences, we evaluated accuracy at 128 and 256 frames on Qwen3-VL-8B and Qwen2-VL-7B.
Table C: Accuracy (Dense vs. Ours)
| Model | Dataset | Frames | Dense Acc. | Ours Acc. | Δ (pp) |
|---|---|---|---|---|---|
| Qwen3-VL-8B | VideoMME | 128 | 67.81 | 66.63 | -1.19 |
| Qwen3-VL-8B | VideoMME | 256 | 69.96 | 68.89 | -1.07 |
| Qwen3-VL-8B | LongVideoBench | 128 | 65.00 | 64.85 | -0.15 |
| Qwen3-VL-8B | LongVideoBench | 256 | 66.12 | 65.82 | -0.30 |
| Qwen2-VL-7B | VideoMME | 128 | 60.81 | 59.63 | -1.19 |
| Qwen2-VL-7B | VideoMME | 256 | 61.93 | 60.78 | -1.15 |
| Qwen2-VL-7B | LongVideoBench | 128 | 56.25 | 56.39 | +0.15 |
| Qwen2-VL-7B | LongVideoBench | 256 | 56.32 | 56.69 | +0.37 |
Accuracy remains stable as frame count increases. Beyond these frame-scaling results, we conducted additional experiments to further evaluate our method, including calibration on STAR, scaling to a larger model, and analysis of Attention sharing efficiency. Please see the Global Response for a summary of these experiments and the corresponding manuscript updates.
Q2: Fair comparison with independently optimized visual token skipping methods.
Our original zero-update ablation used a schedule calibrated specifically for sharing. To compare against independently optimized skipping, we evaluated against Op-Skip (Luo Z, et al., 2026), a representative visual skipping method. Since an off-the-shelf implementation for our specific datasets and models was unavailable, we adapted their official codebase for Qwen3-VL-8B and Qwen2-VL-7B, utilized their official operator risk assessment for optimal layer selection, and aligned their FLOPs budget with ours.
Table D: Share vs. Independently Optimized Op-Skip at Matched FLOPs
| Model | Dataset | Dense Acc. | Op-Skip Acc. | Ours | Δ (Ours vs Skip) | Matched FLOPs (Ours / Op-Skip) |
|---|---|---|---|---|---|---|
| Qwen3-VL-8B | MVBench | 66.60% | 66.08% | 66.63% | +0.55 pp | 58.19% / 58.49% |
| Qwen3-VL-8B | VideoMME | 62.48% | 59.81% | 61.81% | +2.00 pp | 58.23% / 58.53% |
| Qwen2-VL-7B | MVBench | 65.03% | 62.23% | 64.90% | +2.68 pp | 56.59% / 56.55% |
| Qwen2-VL-7B | VideoMME | 56.11% | 53.11% | 55.44% | +2.33 pp | 56.68% / 56.65% |
Our method consistently achieves higher accuracy than Op-Skip under equivalent computational budgets. This demonstrates the benefit of preserving individual residual states and sharing updates over simply skipping operators entirely.
Response (1/3)
Dear Reviewer vY5G,
We thank you for the detailed review. Based on your feedback, we have conducted additional experiments and revised the manuscript to improve clarity.
W1: Motivation, presentation, and completeness of the manuscript.
1. Clarifying motivation: Our original motivation, as described in the Introduction, is to reduce redundant Attention and FFN computation during decoder prefill. We pursue this goal by exploiting redundancy in operator updates: visual tokens can retain distinct information while requiring similar updates. Carrier–receiver sharing therefore separates retaining a token's representation from independently computing its update. Carriers compute the original operator updates, and receivers add the assigned updates to their own residual states. This reduces repeated computation while preserving individual token representations, positions, and K/V entries. The additional analysis in W2 and Q1 below provides direct evidence linking input similarity to update similarity, further supporting the rationale for carrier–receiver sharing.
2. Improving presentation: We have conducted new experiments on scaling, generalization, and sharing mechanics. We have incorporated the core findings into the main text and appendix, and revised the manuscript layout to make it more compact and easier to read.