acceptodds
Under review as a conference paper at ICLR 2027

LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs

Abstract

The fundamental challenge in scaling Video Large Language Models (Video LLMs) to long-form video lies in managing the explosion of visual-token context length. Existing strategies predominantly focus on "post-hoc" token reduction–reducing visual tokens after feature extraction to alleviate the LLM's computational overhead. While these methods effectively reduce the number of visual tokens, we observe that the primary latency bottleneck then shifts from the LLM to the expensive per-frame processing of the vision encoder. To address this, we introduce LiteFrame, a strong, yet highly efficient video encoder backbone for Video LLMs. To train LiteFrame, we propose Compressed Token Distillation (CTD), a novel training framework that teaches a compact student vision encoder to directly predict information-dense, spatio-temporally compressed representations produced by a large teacher vision model, effectively bypassing redundant computation. When coupled with further Language Model Adaptation (LMA), this approach results in a new latency-accuracy Pareto frontier–compared with InternVL3-8B, LiteFrame provides a 35% reduction in end-to-end latency while processing 8 more frames and improves average video understanding accuracy across multiple benchmarks. Our results demonstrate a new potential path to unlocking longer-form video understanding under fixed compute budgets.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video LLMs, Efficient Video Encoders

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_9tiHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper one efficient VLM for video understanding, where an efficient video encoder compress the token within a lightweight visual backbone. CTD, a new training framework is proposed to transfer spatio-temporally dense information from a teacher to a compact student. The expriments on effectiveness and efficiency demonstrate the superiority of the proposed method.

Critical strengths

This may be the first work that integrates spatio‑temporal compression natively inside the vision backbone.

Compressed Token Distillation is proposed to transfer the spatio-temporally dense information from a teacher to a compact student.

The motivation and writing is clear.

The experiments demonstrate the effectivenesses and efficiency of the proposed model.

Critical weaknesses

The evaluations should be more solid, Only video benchmarks are evaluated. What about the performances on the other benchmark datasets, such as images, or document video benchmark dataset.

Traditional frame sampling strategy differs from the proposed litframe. Can the two strategies considerded togehter?

All main experiments are confined to the InternVL model family. What about the performances on other VLMs?

Questions for authors

Check the detailed comments in the weaknesses part.

4 comments

We sincerely thank Reviewer 9tiH for carefully evaluating our work and providing constructive feedback. We are particularly grateful that you recognized (1) the novelty of integrating spatio-temporal compression directly into the vision backbone, (2) the strong experimental results and (3) our clear motivation and writing.

[R3W1] Evaluation Scope

Thank you for the suggestion and we appreciate the sentiment that evaluations should be as comprehensive as possible. That being said, we believe our experiments are quite comprehensive in targeting our main research area of “efficient video understanding” and cover 8 strong video benchmarks: Video-MME, MLVU, LongVideoBench, MMBench-Video, LVBench, MVBench, TVBench, and HLVid. Together, these evaluate short and long video understanding, multiple-choice and free-form QA, and high-resolution video perception. There is some coverage of text reading through OCR questions in Video-MME. However, we report aggregate benchmark results rather than a dedicated analysis of document-video understanding. We agree that document-centric evaluation is a valuable extension but we leave that as future work.

Since LiteFrame was trained on video data, as acknowledged in the Limitations section, image evaluation is outside the current experimental scope. Extending our training and evaluation to image understanding is a promising direction, yet the current results do not establish whether replacing the original encoder with LiteFrame preserves image-understanding performance. We would like to emphasize that LiteFrame achieves a state-of-the-art score on HLVid, which requires fine-grained spatial perception via zero-shot spatial resolution scaling (see Section 5.2.3).

[R3W2] LiteFrame with Uniform Frame Sampling

Thank you for raising this question. Yes, LiteFrame supports both standard uniform frame sampling and the proposed dense clip sampling. We originally used dense clip sampling to help our video encoder exploit temporal continuity between frames.

To directly evaluate uniform frame sampling, we apply it to LiteFrame with 64, 128, and 256 input frames. As shown in the table below, uniform sampling performs comparably to dense clip sampling, with a small improvement of 0.3-0.4 percentage points in average accuracy.

These results show that LiteFrame works well with both sampling strategies, suggesting robustness to the larger inter-frame intervals encountered when uniformly sampling long videos. We have added further results and discussion to Appendix I.

MethodFramesSampling StrategyLatency (ms)Video-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
LiteFrame64Uniform147.161.765.466.253.661.7
LiteFrame64Clip147.161.364.165.154.761.3
LiteFrame128Uniform268.564.168.368.455.164.0
LiteFrame128Clip268.563.667.666.757.063.7
LiteFrame256Uniform525.664.869.669.858.565.7
LiteFrame256Clip525.664.768.570.557.765.4

[R3W3] Other VLMs

Thank you for raising this concern and we do agree that evaluation in multiple VLM families would be nice. We focused our experiments on InternVL3-8B because (1) it represents a standard, widely adopted architecture for modern Video LLMs (frame-wise ViT + MLP+LLM) and (2) distilling the student on large-scale video datasets is computationally demanding. Appendix F provides some form of generalization, transferring to InternVL3-2B using the LiteFrame encoder, already distilled from InternVL3-8B. Note that these two models have different ViT weights each. Extending our method to other model families (e.g. Qwen, Molmo) is a promising way to strengthen our paper, which we intend to explore in future work, but due to compute limitations, we are unable to provide this analysis in this rebuttal.

Reviewer_9tiHOpenReview ↗

Comments after reading the rebuttal

The first two comments are well addressed.

The comments about the performances on the other VLMs are not well addressed with no experimental results provided. InternVL3-8B is one baseline VLM, which performs inferiorly to current modern VLMs. The performance gains over InternVL3-8B may not be obtained based on other VLMs. The experiments are highly suggested to demonstrate the effectiveness of the proposed LiteFrame.

Thank you for taking the time to read our response and for confirming that our responses addressed your first two concerns. We agree that validating LiteFrame beyond the InternVL family is important for establishing the generalizability of our method. That being said, full validation of LiteFrame on a new VLM family requires distilling a new student encoder followed by Language Model Adaptation, which is infeasible within the rebuttal period given our computational resources.

Nevertheless, LiteFrame's gains rely on two critical components: (1) WAP serving as an effective training objective for distillation and (2) the LLM benefiting from frame scaling under compressed visual tokens. As a result, we believe showing that these two findings transfer to other VLM families is a partial, yet strong indicator of whether the full method would transfer. In the tables below, we test both properties on Qwen3-VL-8B-Instruct. Please check the plots (Fig. 14) in Appendix K.

The evaluations demonstrate the following:

(1) WAP consistently outperforms average pooling. Under a 16$\times$ compression ratio, WAP achieves higher accuracy across four benchmarks, increasing average accuracy from 54.5% to 55.4% (+0.9%p) at 64 frames and from 58.1% to 59.1% (+1.0%p) at 128 frames.

MethodFramesVisual TokensVideo-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
Qwen3-VL-8B-Instruct (No comp.)64819266.869.768.661.266.6
+ Average Pooling (16$\times$)6451252.059.654.052.454.5
+ WAP (Ours, 16$\times$)6451253.360.754.852.755.4
Qwen3-VL-8B-Instruct (No comp.)1281638469.173.172.762.669.4
+ Average Pooling (16$\times$)128102455.064.057.356.058.1
+ WAP (Ours, 16$\times$)128102456.365.058.756.459.1

(2) WAP enables effective frame scaling under fixed visual token budgets. Across budgets of 1024, 2048, 4096, and 8192 visual tokens, WAP (16$\times$) allows 16$\times$ more input frames and improves average accuracy by 2.0-4.0%p over the uncompressed baseline. Notably, WAP (16$\times$) with 256 frames (2048 tokens) already matches the uncompressed baseline with 32-frame (4096 tokens) using half the visual tokens.

MethodFrames at Each Budget1024 Visual Tokens2048 Visual Tokens4096 Visual Tokens8192 Visual Tokens
Qwen3-VL-8B-Instruct (No comp.)8 / 16 / 32 / 6455.159.162.866.6
+ WAP (16$\times$)128 / 256 / 512 / 102459.1 (+4.0)62.8 (+3.7)66.5 (+3.7)68.6 (+2.0)

In summary, these results confirm that LiteFrame’s core compression primitive and frame scaling behavior generalize well to Qwen3-VL-8B-Instruct; while this still does not show whether the full LiteFrame method transfers, we think it is a strongly positive indicator and hope that it at least partially addresses the reviewer’s concern. We are also extending these evaluations to other VLM families. We will report the results here as they become available.

Following up on our Qwen3-VL-8B results, we have now completed evaluations on Gemma4-E4B-it, which show the same two findings.

(1) WAP consistently outperforms average pooling. At 64 frames and 16× compression, WAP improves average accuracy from 49.0% to 50.1% (+1.1 percentage points), with improvements in all four evaluation settings.

MethodFramesVisual TokensVideo-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
Gemma4-E4B-it (No comp.)641638456.159.960.754.157.7
+ Average Pooling (16$\times$)64102446.153.548.747.649.0
+ WAP (Ours, 16$\times$)64102448.354.949.747.750.1

(2) WAP enables effective frame scaling under fixed visual-token budgets. Across the four evaluated budgets, 16$\times$ WAP processes 16$\times$ more frames and improves average accuracy by 2.2–6.1 percentage points over the uncompressed baseline.

MethodFrames at Each Budget1024 Visual Tokens2048 Visual Tokens4096 Visual Tokens8192 Visual Tokens
Gemma4-E4B-it (No comp.)4 / 8 / 16 / 3244.047.951.455.4
+ WAP (16$\times$)64 / 128 / 256 / 51250.1 (+6.1)53.3 (+5.4)56.0 (+4.6)57.6 (+2.2)

Notably, WAP with 128 frames achieves 53.3% using 2,048 visual tokens, exceeding the uncompressed baseline with 16 frames (51.4% using 4,096 tokens) while using half the visual tokens. These results provide further empirical support for LiteFrame’s two design principles beyond InternVL. The updated table and plots are included in Appendix K.

We hope these additional results help address your remaining concerns. If you have any further questions, we would be happy to clarify them before the discussion period ends. We sincerely appreciate your time and thoughtful consideration.

Reviewer_kL16OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LiteFrame, a lightweight visual encoder for VideoLLMs, addressing the visual encoding latency bottleneck when effective post-hoc token reduction methods are applied. Its core method, Compressed Token Distillation (CTD), distills WAP-compressed features from a large teacher encoder into a lightweight student encoder. Extensive experiments demonstrate that LiteFrame achieves competitive video understanding performance with low latency, particularly when processing large numbers of frames.

Critical strengths
  1. Well-motivated approach. The paper identifies visual encoding as an important efficiency bottleneck in VideoLLMs, particularly when effective LLM-side token reduction methods are employed.
  2. Strong experimental results. Extensive experiments demonstrate competitive performance and reduced latency compared to existing post-hoc token reduction methods and efficient visual encoders, particularly as the number of frames increases. Ablation studies further validate the effectiveness of compressed token distillation over conventional distillation.
Critical weaknesses
  1. Limited novelty. The core contribution is applying compressed feature distillation to lightweight visual encoders. While the experimental results are solid, the overall approach is relatively straightforward, raising concerns about methodological novelty.
  2. Insufficient analysis of distillation effectiveness. The paper does not adequately justify WAP as the distillation target over simpler alternatives such as average pooling, nor does it analyze the consistency between student-predicted and teacher-generated representations. Moreover, the effects of the student architecture and distillation target are not clearly disentangled.
  3. Limited generalizability and scalability evaluation. Most experiments rely on InternVL3-8B, leaving the effectiveness of LiteFrame across different MLLM backbones unclear. Further evaluations with different teacher and student model sizes are also needed, particularly given the reported training instability of smaller students.
  4. Potential loss of query-relevant information. Since token compression is performed within the visual encoder without access to downstream queries, it remains unclear whether important query-relevant information may be discarded. Further analysis is needed to understand this potential limitation, especially for fine-grained video understanding tasks.
Questions for authors

Please see Weaknesses

3 comments

We greatly appreciate Reviewer kL16’s thoughtful review and positive assessment. We are particularly grateful that you highlighted (1) the strong motivation for addressing visual encoding efficiency and (2) our strong experimental results, supported by comparisons with post-hoc token reduction methods and ablation studies.

[R2W1] Methodological Novelty

Thank you for recognizing the experimental results. We agree that feature distillation, pooling, and temporal convolutions themselves are established techniques. However, our contributions stem from how the supervision target and student computation are designed together for efficient video understanding.

Specifically, CTD (Compressed Token Distillation) trains the student to directly predict the teacher’s spatio-temporally compressed output. This differs from reconstructing the teacher’s dense features through an auxiliary decoder or applying compression only after dense feature extraction. Combined with progressive token reduction in the student, LiteFrame significantly reduces computation during both visual encoding and LLM prefilling. We believe that comprehensive ablations in the manuscript support these design choices and their effectiveness. Furthermore, while we respect the reviewer’s opinion on methodological novelty, we see the straightforwardness of our approach as a strength, rather than a limitation. Our experimental results demonstrate that careful application of simple ideas can outperform far more complex techniques in the literature.

[R2W2] WAP as the distillation target / Representation consistency / Ablative Analysis

Our manuscript already includes quantitative results directly addressing these questions in the Appendix:

  • Appendix C.4 (Table 8) compares WAP with average pooling as a distillation target. At 128 frames, WAP achieves 60.4% average accuracy after LMA, compared with 59.0% for average pooling.
  • Appendix E (Table 10) quantitatively demonstrates representation consistency between the student output and the compressed teacher representations. Cosine similarity reaches 0.811 and linear CKA reaches 0.646.
  • Appendix C.5 (Table 9) provides further ablative analysis that isolates the effect of using WAP as the distillation target. This compares CTD against vanilla distillation followed by post-hoc WAP, obtaining 64.1% vs. 61.1% average accuracy at 256 frames. Furthermore, in Table 6, for the exactly same compressive student architecture, we compare two training objectives, where CTD + LMA outperforms RTD + LMA (63.4% vs 61.5%).

[R2W3] Generalizability & Scalability evaluation

Thank you for raising this point. We focused our main experiments on InternVL3-8B because (1) it represents a standard, widely adopted architecture for modern Video LLMs (frame-wise ViT + MLP+LLM) and (2) distilling the student on large-scale video datasets is computationally demanding. Appendix F demonstrates some form of generalization, transferring to InternVL3-2B using the LiteFrame encoder (which is already distilled from InternVL3-8B). Note that these two models have different ViT weights each.

Regarding the student size, we explored 22M and 6M parameter students and observed training instability under the initial training setup. In our subsequent experiments, we found that replacing the LayerNorm with RMSNorm and using random weight initialization (instead of clipping the teacher’s weights) resolved the instability. However, the resulting models did not improve the accuracy-latency Pareto frontier under the tested settings. We have added the discussion to Appendix J.

[R2W4] Query-relevant information

We completely agree that traditional token selection or pruning approaches (which permanently drop tokens) run a significant risk of discarding critical query-relevant information. However, LiteFrame avoids this risk in three ways:

(1) LiteFrame does not perform discrete hard selection or pruning of the patches. Instead, it aggregates all local regions through learned spatio-temporal operations. This ensures that no patch of the video is permanently discarded, allowing every region to contribute to the compressed representation.

(2) Our WAP supervision target is weighted by the teacher’s attention. Since the teacher ViT is already aligned with the LLM, this teaches the student to aggregate and compress highly salient features that are expected to be useful for downstream reasoning, rather than dropping them.

(3) The LMA stage explicitly fine-tunes the student’s latent space alongside the LLM. This further guarantees that query-relevant visual information is preserved with the LLM’s needs.

Moreover, Appendix B.1 and Figure 8 demonstrates the effectiveness of LiteFrame for the fine-grained short video benchmarks (MVBench and TVBench), achieving 28% and 63% latency reduction, respectively.

Dear Reviewer kL16,

We truly appreciate your positive assessment and your thoughtful comments. We would like to kindly follow up on our rebuttal, where we clarified our contributions and analyses of WAP as the distillation target, representation consistency, and ablations, along with a discussion of smaller student backbones and the preservation of query-relevant information. We have also added results on Qwen3-VL-8B-Instruct in Appendix K, showing that the core components of LiteFrame transfer to another VLM family, which provides partial support for the generalizability of LiteFrame.

We hope our responses have addressed your concerns. If you have any remaining questions, we would be happy to clarify them before the discussion period ends. We sincerely appreciate your time and support.

Warm regards,

Authors

Reviewer_CkRxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LiteFrame, an efficient vision encoder for Video LLMs that addresses the computational bottleneck of conventional post-hoc visual token compression. The authors introduce Compressed Token Distillation (CTD), which trains a lightweight vision encoder to directly predict compressed teacher representations, followed by Language Model Adaptation (LMA). Experiments across multiple video understanding benchmarks demonstrate improved accuracy–latency trade-offs, enabling the model to process substantially more frames with lower inference latency.

Critical strengths

The paper addresses an important efficiency bottleneck in Video LLMs, highlighting the computational overhead of vision encoding that remains after post-hoc token compression. And the proposed framework is relatively simple and effective, combining a lightweight spatiotemporal encoder with compressed token distillation. The experiments demonstrate promising efficiency improvements across multiple benchmarks, supported by several ablation studies.

Critical weaknesses
  1. The experimental comparisons primarily focus on conventional post-hoc token compression methods, such as ToMe, PruMerge, and FastVID, while overlooking several recent approaches that also address vision encoding efficiency. For example, EarlyTom (CVPR 2026) performs token compression within the vision encoder, and MeToM (CVPR 2026) reduces redundant computation through metadata-guided token merging. FlashVID (ICLR 2026) is another relevant and competitive baseline for spatiotemporal token compression. These methods are closely related to the motivation of LiteFrame, and the absence of comparisons makes it difficult to assess the proposed method's advantages over recent state-of-the-art approaches.

  2. The main experiments compare methods using substantially different numbers of input frames (e.g., 32 frames for InternVL3-8B versus 256 frames for LiteFrame). While such comparisons are reasonable for evaluating accuracy–latency trade-offs, they do not clearly isolate the benefits of the proposed encoder from those of processing more frames. Although the paper includes some fixed-frame comparisons, more comprehensive evaluations under identical frame budgets would help establish the effectiveness of the proposed representation learning. Moreover, the evaluation uses different frame sampling strategies (uniform sampling for baselines versus dense clip sampling for LiteFrame), introducing an additional confounding factor. A more controlled evaluation is needed to better attribute the reported improvements to the proposed method.

Questions for authors

I think the authors should strengthen the experimental evaluation by including recent encoder-side efficiency methods and conducting more controlled comparisons under matched frame counts, sampling strategies, and computational budgets. This would provide more convincing evidence for the effectiveness of LiteFrame and clarify its advantages over existing approaches.

4 comments

We sincerely thank Reviewer CkRx for taking the time to evaluate our work and providing constructive feedback. We are particularly grateful that you recognized (1) the importance of the vision-encoding bottleneck, (2) the simplicity and effectiveness of our framework, and (3) the promising efficiency improvements across multiple benchmarks.

[R1W1] Comparison with Recent Compression Baselines

Thank you for the suggestion—we agree that including these additional baselines strengthens our evaluation. As a result, we evaluated EarlyTom (CVPR 2026) and FlashVID (ICLR 2026) on InternVL3-8B by reproducing their official implementations. We could not include MeToM (CVPR 2026) because their code was not publicly available. As shown in the table below, LiteFrame still maintains the best performance on the latency-accuracy Pareto frontier. Please check the updated Pareto frontier plot in Figure 13 in Appendix H. LiteFrame processes 128 frames with 63.7% average accuracy at 272ms, compared with 61.9% at 493.1ms for EarlyTom using 64 frames. The comparison with FlashVID shows a similar gain.

We would like to emphasize that although EarlyTom performs merging within the vision encoder, vision encoding remains a bottleneck: at 128 frames, vision encoding takes 665.3ms and LLM prefilling takes 288.6ms. EarlyTom still requires dense early-layer computation in the original heavy ViT and incurs additional compression operations. LiteFrame, in contrast, addresses this bottleneck by directly predicting compressed tokens with a lightweight encoder. These results further support the effectiveness of LiteFrame for frame scaling under a fixed latency budget. We have added these results and discussion to Appendix H, which will be included in the main text of the final version.

MethodFramesVision (ms)LLM (ms)Total (ms)Video-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
InternVL3-8B840.0167.3208.459.060.062.254.859.0
+ FastVID32161.763.0224.858.762.564.052.659.5
+ EarlyTom32171.299.8271.558.962.463.952.659.5
+ FlashVID32275.360.0335.658.362.663.654.359.7
+ LiteFrame (Ours)6454.894.9150.161.064.265.753.661.1
InternVL3-8B1674.0329.3403.661.964.066.456.562.2
+ FastVID64310.695.4406.259.363.465.154.460.6
+ EarlyTom64332.3160.3493.160.764.268.454.261.9
+ FlashVID64513.887.4601.559.563.466.754.261.0
+ LiteFrame (Ours)128105.3166.6272.663.966.866.757.263.7
InternVL3-8B32144.5669.8814.565.667.469.658.765.3
+ FastVID128625.8168.9794.960.466.169.355.662.9
+ EarlyTom128665.3288.6954.562.667.171.056.764.3
+ FlashVID128992.9155.41148.661.366.469.757.063.6
+ LiteFrame (Ours)256204.0327.4532.365.168.570.758.365.7

[R1W2-1] Efficiency Gains under Identical Frame Budgets

Thank you for pointing out the distinction between these two evaluation settings. LiteFrame is designed for frame scaling: processing more frames within a fixed latency budget. It cuts the cost of both visual encoding and LLM prefilling, and the saved compute budget can then go toward processing additional frames. A fixed-frame-count evaluation holds the number of frames constant, so the saved compute budget goes unused and one of the method's primary benefits then doesn't show up. For this reason, we believe a fixed-latency comparison is the fairer way to evaluate LiteFrame. This also directly aligns well with how such models are deployed in practical, real-world scenarios.

That being said, we agree that fixed-frame comparisons provide a complementary assessment of this trade-off. We therefore report the 32-frame results below. LiteFrame reduces latency from 814.5ms to 91.9ms (8.9$\times$ faster), while average accuracy decreases from 65.3% to 57.7%. Our frame scaling experiments evaluate whether this computational saving can be converted into a better accuracy-latency trade-off, as shown in Table 3.

MethodFramesLatency (ms)Video-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
InternVL3-8B32814.565.667.469.658.765.3
LiteFrame3291.957.460.459.653.257.7

[R1W2-2] Robustness across Frame Sampling Strategies

Thank you for raising this concern. The additional experiments show that LiteFrame’s accuracy-latency advantage persists when both models use the same sampling strategy. We evaluate both InternVL3-8B and LiteFrame under uniform frame sampling and dense clip sampling. As shown in the table below, LiteFrame achieves higher average accuracy and lower latency under both strategies while processing 8$\times$ more frames. This confirms that our reported advantage does not hinge on a specific frame sampling strategy.

Additionally, the uniform sampling experiments show small improvements in average accuracy for LiteFrame across the tested frame budgets. This suggests that LiteFrame can handle larger frame intervals encountered when uniformly sampling long videos. We have added these results and discussion to Appendix I.

MethodFramesSampling StrategyLatency (ms)Video-MME (w/o subs)Video-MME (w/ subs)MLVULongVideoBenchAverage
InternVL3-8B8Uniform206.059.060.062.254.859.0
+ LiteFrame64Uniform147.161.765.466.253.661.7
InternVL3-8B16Uniform401.361.964.066.456.562.2
+ LiteFrame128Uniform268.564.168.368.455.164.0
InternVL3-8B32Uniform809.765.667.469.658.765.3
+ LiteFrame256Uniform525.664.869.669.858.565.7
InternVL3-8B8Clip206.053.855.156.352.754.5
+ LiteFrame64Clip147.161.364.165.154.761.3
InternVL3-8B16Clip401.357.758.560.255.057.9
+ LiteFrame128Clip268.563.667.666.757.063.7
InternVL3-8B32Clip809.761.163.163.357.161.2
+ LiteFrame256Clip525.664.768.570.557.765.4

Dear Reviewer CkRx,

Thank you again for your constructive feedback on our submission. We wanted to gently follow up on our rebuttal, where we addressed your concerns with additional experiments.

  • [R1W1] Comparison with recent baselines: We reproduced EarlyTom and FlashVID on InternVL3-8B. LiteFrame still achieves the best latency-accuracy Pareto frontier. Please check the updated Pareto frontier plot in Figure 13 of Appendix H.

  • [R1W2-1] Identical frame budgets: We added a 32-frame comparison. We also explained why the fixed-latency setting better reflects frame scaling, the primary goal of LiteFrame.

  • [R1W2-2] Frame sampling strategy: We evaluated both InternVL3-8B and LiteFrame under the same sampling strategy (uniform and dense clip sampling). LiteFrame achieves higher accuracy at lower latency in both settings (Appendix I).

Please let us know if our responses have addressed your concerns, or if there are any remaining questions we can help clarify before the discussion period ends. We would be grateful to hear your thoughts.

Warm regards,

Authors

Thank you for your detailed response. I agree with the your responses to most of the original two weaknesses and no longer consider the lack of baselines and the unfair sampling strategy to be major issues. Based on this, I tend to raise my score to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.