acceptodds
Under review as a conference paper at ICLR 2027

CueKFS: Agentic Cue-Driven Keyframe Selection for Long Video Understanding

Abstract

Keyframe selection (KFS) has long produced compact video summaries for browsing and retrieval, and representative frames for thumbnails. More recently, when conditioned on a question, KFS provides an alternative to uniform sampling for long-video question answering by selecting frames that are more relevant to the question. Most methods rank frames by similarity to the question. Yet a relevant frame may score poorly when the question combines subjects or moments that no single frame shows, or requires implicit information absent from its wording. Other methods try to break down the question into subqueries, but suffer from inaccurate decomposition due to their static initial context. Therefore, we propose CueKFS, a training-free method that reformulates question–frame matching as comparing frames against a set of dynamically generated visual cues. From an initial set of salient frames, we decompose the question into cues. Each cue concurrently probes the video to navigate to its own evidence. A reasoning VLM then agentically revises the cue set against its evidence to re-explore the video. CueKFS then allocates the budget across the surviving cues. Across three benchmarks, CueKFS establishes state-of-the-art results in all 27 evaluated settings with available prior results, achieving budget-averaged gains of up to +4.54% over the previous baseline and a median of only two VLM calls. We further provide a detailed behavioral analysis of CueKFS, showing that agentic cue refinement drives active re-exploration of the video, yielding relative similarity gains of up to 92% over the initial context.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Keyframe Selection, Long Video Understanding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_JafDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CueKFS is a training-free keyframe selection method for long-video QA. A reasoning VLM decomposes the question into visual cues using a compact video overview. Each cue retrieves relevant frames via SigLIP2 similarity, after which the VLM refines the cues (keep/rewrite/split/drop) based on the retrieved evidence. The refined cues guide a second retrieval stage, with the frame budget allocated across cues. CueKFS consistently outperforms WFS-SB on Video-MME, MLVU, and LongVideoBench across three downstream VLMs and three frame budgets, requiring a median of only two VLM calls.

Critical strengths
  1. Well-motivated approach. Refining cues based on retrieved evidence addresses the limitations of static question decomposition.
  2. Training-free and efficient. Batched cue refinement keeps VLM calls low, with consistent gains across 27 settings, including with a strong downstream VLM (GPT-5.5).
  3. Thorough analysis. The paper provides extensive ablations, behavioral analyses (coverage shift and action frequency), and a candid discussion of failure cases.
Critical weaknesses
  1. Potential test-set tuning. Hyperparameters appear to be tuned on Video-MME, which is also used for evaluation in Table 1. This contributes approximately 1.7 points to the reported gain, raising concerns about generalization.
  2. Unfair comparisons. CueKFS uses GPT-5.5 with access to the question, answer options, and multiple frames, while most baselines rely only on embedding models. Cost-matched comparisons with related methods (T*, MSJoE, VideoAgent) and a direct GPT-5.5 frame-selection baseline are missing. Per-question latency and API cost are also unreported.
  3. Limited novelty and uncertain gains. The method primarily combines existing techniques and heuristics, with approximately 15 hyperparameters. Gains at k=32 are small, with no variance or significance analysis to establish their reliability.
Questions for authors
  1. Were the hyperparameters in Table 3 tuned on the same Video-MME test set used in Table 1? If so, how does performance change with held-out tuning or default settings?
  2. How much of the gain depends on GPT-5.5 as the reasoning VLM? Could you reproduce Table 1 using Qwen3.6-27B for cue reasoning across all downstream models, and include a baseline where GPT-5.5 directly selects k frames from the overview?
  3. How does CueKFS compare with T*, MSJoE, and VideoAgent under matched VLM costs? Could you report per-question latency, input image counts, and API cost relative to WFS-SB?
  4. How does performance change when answer options are excluded from cue decomposition and refinement?
  5. Could you report variance across runs or statistical significance, particularly at k=32, where improvements are small?
5 comments

Weakness 1 / Question 1. Generalization of tuning

The original Video-MME release provides only a public test split, so the sensitivity sweep in Table 3 uses the same split as the evaluation in Table 1. Even before hyperparameter tuning, CueKFS outperforms WFS-SB, the strongest Video-MME baseline in Table 1 with the GPT-5.5 reader. At the same frame budgets, CueKFS averages 81.04% with Qwen3.6-27B and 81.52% with GPT-5.5, compared with WFS-SB’s 78.64%.

The results on MLVU and LongVideoBench already demonstrate cross-benchmark generalization. With the Video-MME settings frozen and no tuning on either benchmark, CueKFS achieves 81.95% on MLVU and 77.51% on LongVideoBench, outperforming WFS-SB by 2.42 and 4.06 points, respectively.

To further test whether the selected hyperparameters generalize with held-out tuning, we repeat the full one-factor-at-a-time sweep from Table 3 on CG-Bench mini, a benchmark unused in our original experiments. We use the same selection criterion, averaging accuracy across frame budgets. The sweep selects the same values as Video-MME except for the deduplication thresholds. The cosine similarity threshold stays at 0.98, while the temporal window increases from 1 s to 3 s. We compare both CueKFS configurations and WFS-SB on the same CG-Bench mini split at k=16 with the GPT-5.5 reader.

SelectorDeduplication thresholds (similarity, time)CG-Bench mini long-acc. (%)
WFS-SBN/A54.0
CueKFS with frozen Video-MME settings(0.98, 1 s)57.3
CueKFS with settings selected on CG-Bench mini(0.98, 3 s)57.5

The frozen configuration already outperforms WFS-SB by 3.3 points at the same frame budget. Repeating the sweep changes accuracy by only 0.2 points, supporting the transfer of the Video-MME configuration to CG-Bench mini.


Weakness 2 / Questions 1 and 2. Default settings, Qwen3.6-27B setting, and direct GPT-5.5 selection

To separate the effect of the VLMs, we evaluate Qwen3.6-27B with both the pre-sweep settings and the final Table 1 settings. In the direct-selection control, GPT-5.5 receives the same question, answer options, and overview as CueKFS and selects 16 frames from that overview, which is capped at 48 frames.

BenchmarkReaderUniform sampling (%)WFS-SB (%)CueKFS with default settings (%)CueKFS with Table 1 settings and Qwen (%)Direct GPT-5.5 (%)CueKFS with Table 1 settings and GPT-5.5 (%)
Video-MMEGPT-5.577.1178.7480.6382.5277.4183.30
Video-MMEQwen2.5-VL57.5661.3061.8565.4857.8566.26
Video-MMEInternVL362.5665.4866.5268.4862.8569.26
MLVUGPT-5.567.7480.3680.4081.5168.0882.29
MLVUQwen2.5-VL58.3768.3568.9570.7958.6971.57
MLVUInternVL367.0775.6276.7276.3667.3977.14
LongVideoBenchGPT-5.567.4673.9074.8776.5167.7677.26
LongVideoBenchQwen2.5-VL56.3263.3561.2263.9556.6264.70
LongVideoBenchInternVL356.2560.8160.9963.4356.5464.17

At the default settings, Qwen3.6-27B retains most of the gain over WFS-SB obtained with GPT-5.5 as the reasoner. On Video-MME at k=16, CueKFS with Qwen gains 1.89 points over WFS-SB with the same GPT-5.5 reader. Before the sweep, switching only the reasoner to GPT-5.5 adds 0.48 points on average across frame budgets (Table 3). With the final settings, the GPT-5.5–Qwen gap is +0.78 points On Video-MME at k=16.

Direct GPT-5.5 selection provides only a small gain over uniform sampling. On Video-MME with the GPT-5.5 reader, direct selection scores 77.41%, compared with 77.11% for uniform sampling and 78.74% for WFS-SB. A 48-frame overview can miss the specific objects, actions, or moments a question asks about. Embedding model can search the full candidate pool for that evidence, whereas direct selection can only rank the frames it already sees. With the same reasoner, reader, starting overview, and output budget, CueKFS gains 5.89 points over direct selection by refining its cues and retrieving evidence by embedding model.

Weakness 2 / Question 3. VLM cost and selector latency

On Video-MME at k=16, we fix GPT-5.5 as the reader and use Qwen3.6-27B on one H200 for VLM-based selection. We report accuracy and VLM cost for each method’s selection workload, using VLM runtime for Qwen and API charges for GPT-5.5. CueKFS uses the Table 1 settings, and A.I.R. retains its default analysis image cap and batches its per-frame requests. Because the selectors all use an embedding or relevance-extraction stage, we time selection from decoded frames and precomputed features or relevance scores to the selected frame IDs. VLM time covers prefill and decoding, including VideoAgent’s caption generation. Selector latency also covers image encoding, retrieval, MSJoE’s learned sampler, and T*’s detector search.

MethodQwen calls/qQwen input / output tokens/qQwen input images/qQwen VLM time (s/q)Selector latency after embeddings (s/q)GPT-5.5 selector API ($/q)Acc. with Qwen (%)Acc. with GPT-5.5 (%)
WFS-SB00 / 0000.2078.7478.74
T*13,811 / 160322.324.30.037879.3079.81
MSJoE1685 / 19282.202.30.019178.4178.81
A.I.R.19 (batched)≈16,700 / ≈550193.14.10.110778.6779.19
VideoAgent8≈4,700 / ≈774≈89.19.70.047579.1579.67
CueKFS2≈10,100 / ≈320≤80 with one refinement batch5.025.90.074382.5283.30

CueKFS scores +3.37 points over VideoAgent at 5.9 vs. 9.7 s/question of selector latency after embeddings. Relative to WFS-SB, CueKFS adds 5.7 s/question of selector time and gains 3.78 points with Qwen. Its margins over the strongest alternative are +3.22 points with Qwen and +3.49 points with GPT-5.5 as the selector.


Question 4. Performance without options

On Video-MME, we remove the answer options from the selector's prompts.

Selectork=8 (%)k=16 (%)k=32 (%)
WFS-SB76.2278.7480.96
CueKFS, question only79.7882.7883.85
CueKFS, question and options80.6783.3084.15

At k=16, removing the options reduces CueKFS accuracy by 0.52 points, leaving it 4.04 points ahead of WFS-SB. Options can be essential to cue decomposition and refinement when they specify what evidence to retrieve. For example, in “Which of the following did not happen in the video?”, the question alone leaves the candidate events unspecified.

Weakness 3 / Question 5. Novelty and significance of the gains

We respectivefully disagree with the comment of "primarily combines existing techniques and heuristics". Our novelty is that CueKFS dynamically adjusts the visual evidence for retrieval under our proposed cue-wave calculation and budget allocation. Specifically, CueKFS inspects the frames found for each cue, revises or splits that cue, and searches the full candidate pool again (lines 211–224). This differs from T*, which changes where its detector searches for the initial targets, and MSJoE, which combines the initial queries for its sampler (lines 088–098). It also differs from VideoAgent’s use of captions to formulate subsequent retrieval queries. Our Peaks and Plateaus give the reasoner evidence from brief and sustained matches (lines 191–209). After the second search, Cue Budget Allocation preserves distinct evidence by distributing the final frame budget across cues.

Table 2 measures these contributions under controlled settings. Adding refinement improves average accuracy by 1.78 points. Within refinement, replacing overview frames with cue-wave evidence adds 2.94 points. Allocating frames across the resulting cues adds 2.38 points over global top-k selection. On long videos, the final cues find stronger matches in the full candidate pool than in the overview, with a 53% higher median SigLIP2 score. Refinement also changes which frames the cues cover, with a median Jaccard overlap of 0.65 between the initial and final frame sets (Tables 5–6). Moreover, in our response to the other reviewers, replacing our adjacent-frame segmentation with WFS-SB’s one changes accuracy from 80.63% to 80.18%, and removing CD, IS, and BO changes it to 79.48%. These further support the effectiveness of our design choices.

With all these evidences, we believe our innovation is clear, motivated, and effective.

Regarding the gains on k=32, we want to hightlight that our gains on k=32 exceed three points on two benchmarks, which is not small. We also want to mention that a larger frame budget gives the reader more chances to see relevant evidence despite imperfect selections, which makes the difference between methods relatively less significant on k=32 by design. Even though, CueKFS still improves over WFS-SB by 3.19 points on Video-MME and 3.22 points on LongVideoBench with GPT-5.5. On MLVU, the gain with InternVL3 is 2.84 points.

To assess variance across runs and statistical significance at k=32, we compare three independent Video-MME runs with the GPT-5.5 reader and pair the accuracy differences between WFS-SB and CueKFS by video.

WFS-SB accuracy (mean ± SD, %)CueKFS accuracy (mean ± SD, %)Paired gain (points)95% CI (points)p-value
80.93 ± 0.1584.14 ± 0.24+3.21[+1.84, +4.58]3.76 × 10⁻⁶

The paired difference across runs is +3.21 points. The original video-level paired analysis also establishes statistical significance at k=32. Its 95% confidence interval of [+1.85, +4.59] points excludes zero.

Reviewer_7wCLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CueKFS, a training-free keyframe selection method for long video understanding. The authors introduce a cue-driven framework that decomposes questions into visual cues and iteratively refines them using retrieved evidence and a reasoning VLM. The selected frames are then allocated under a limited frame budget for downstream video question answering. Experiments on 3 long-video benchmarks show consistent improvements over existing keyframe selection methods.

Critical strengths
  1. The paper addresses a timely and important problem in long video understanding, where processing long videos requires substantial visual tokens and computational resources. Efficient keyframe selection is an important direction for reducing this overhead.

  2. The idea of dynamically refining visual cues based on retrieved evidence is interesting and provides a reasonable motivation beyond previous question-frame similarity matching. The results in tab.1 are promising, showing consistent improvements across multiple benchmarks and downstream VLMs.

Critical weaknesses
  1. The initial clip segmentation strategy is not well justified. The authors use adjacent-frame embedding similarity and wavelet-based transition detection to segment videos into clips, but it is unclear how robust this heuristic approach is across different video dynamics. In particular, how effectively can the proposed method capture temporal transitions or fine-grained actions based on frame-level similarity? The reviewer believes that comparisons with other scene detection methods would help clarify the advantage and necessity of this design.

  2. The motivation behind the initial visual cue construction principles is not sufficiently explained. The authors introduce 3 principles (CD / IS / BO), but the underlying intuition and design philosophy remain unclear. Is there any hierarchy behind these principles? It is also unclear why these particular rules are sufficient to cover diverse video dynamics and temporal reasoning requirements. The reviewer believes that further justification would strengthen this part of the contribution.

  3. While the results in tab.1 are promising, it is unclear whether the comparison is sufficiently fair in terms of computational cost. The proposed framework relies on a large embedding model (siglip2-giant) and an additional reasoning VLM (gpt-5.5 or qwen3.6-27B) for cue generation and refinement. Even though the method achieves better accuracy under the same frame budget, it is not obvious what practical advantages it provides considering the additional computational overhead. The authors emphasize the small number of reasoning-VLM calls, but this alone cannot represent the actual inference cost. A detailed cost breakdown, including end-to-end latency or throughput, will be needed to establish a fairer comparison with existing approaches.

Questions for authors
  1. Overall, while the authors attempt to introduce a structured agentic framework for keyframe selection, ironically, the proposed pipeline itself becomes computationally heavy, relying on an additional large reasoning VLM. This raises concerns about its actual efficiency and practical benefits. Furthermore, the method involves several heuristic design choices, but lacks sufficient analysis of their underlying motivations and effectiveness in capturing video dynamics.

  2. As a minor comment, the reviewer found Fig.1 difficult to follow even after several iterations to the method section. It is unclear how the illustrated frames contribute to answering the question and how the intermediate cues and evidence are used throughout the pipeline. The reviewer encourages the authors to improve the figure with clearer explanations and annotations.

8 comments

Weakness 1. Clip segmentation

Since a question can combine several subjects or leave relevant details implicit, changes in question–frame similarity need not coincide with visual transitions (lines 040–046). CueKFS therefore uses adjacent-frame similarity for the boundaries, whereas WFS-SB uses question–frame similarity (lines 143–153). We test this choice on Video-MME at k=16 by substituting the WFS-SB segmentation module into the Table 2 configuration while holding the rest of CueKFS fixed.

MethodSegmentationQA accuracy (%)
CueKFS with WFS-SB segmentationQuestion–frame similarity80.18
CueKFSAdjacent-frame similarity80.63

Our adjacent-frame segmentation improves accuracy by 0.45 points over the WFS-SB segmentation module.


Weakness 2. Cue construction rules

The rules serve complementary roles in retrieval.

  1. Concrete and Distinct (CD) makes each cue specific enough to match a frame and distinct from other cues.
  2. Instance-Separated (IS) keeps repeated events, action stages, or different subjects independently retrievable by assigning each instance its own cue.
  3. Beyond the Overview (BO) expands the evidence they can target. Since a sparse overview can omit relevant events, BO allows cues to seek them across the full video (lines 155–162 and 173–174). Refinement then checks those proposals against retrieved frames and can revise or drop them (lines 211–224).

To measure what the rules contribute jointly, we remove them from the cue-generation prompt (lines 719–727) and hold the rest of the Table 2 configuration fixed on Video-MME at k=16.

MethodQA accuracy (%)
CueKFS without CD, IS, and BO79.48
CueKFS80.63

The −1.15-point change when removing the rules isolates their joint contribution to accuracy with the retrieval and refinement procedures held fixed.


Weakness 3 / Question 1. Computational cost and efficiency

CueKFS computes frame embeddings once per video and reuses them across questions. This is the same as prior works. Using VLM in selection is common also a practice in related works, including A.I.R. and VideoAgent. On Video-MME at k=16, we fix the GPT-5.5 reader and use Qwen3.6-27B on one H200 for VLM-based selection. We retain each selector’s workload and report median per-question costs. The selector timer starts from decoded frames and precomputed embeddings or relevance scores, including WFS-SB’s question-conditioned scores, and ends when the 16 frames are selected. It includes image encoding for the selector VLM, captioning for VideoAgent, and retrieval.

MethodQwen input / output tokens/qQwen VLM time (s/q)Selector latency after precomputation (s/q)Accuracy (%)
WFS-SB0 / 000.278.74
A.I.R.≈16,700 / ≈5503.14.178.67
VideoAgent≈4,700 / ≈7749.19.779.15
CueKFS≈10,100 / ≈3205.025.982.52

As shown above, CueKFS gains 3.37 points over VideoAgent while reducing selector latency by 3.8 s/question. A.I.R.’s batched verification is 1.8 s/question faster than CueKFS, but with 3.85 points lower accuracy. Therefore, our CueKFS reaches a good trade-off between efficiency and effectiveness. We are not computationally heavy.

Question 1. Analysis of design choices

We respectivefully disagree with the comment of "lacks sufficient analysis of their underlying motivations and effectiveness in capturing video dynamics". Our method achieves SOTA on different lengths of videos with different dynamic across three benchmarks. The motivation is that we want to retrieve frames by a set of visual cues that dynamically changes. Specifically, CueKFS inspects the frames found for each cue, revises or splits that cue, and searches the full candidate pool again (lines 211–224). This differs from T*, which changes where its detector searches for the initial targets, and MSJoE, which combines the initial queries for its sampler (lines 088–098). It also differs from VideoAgent’s use of captions to formulate subsequent retrieval queries. Our Peaks and Plateaus give the reasoner evidence from brief and sustained matches (lines 191–209). After the second search, Cue Budget Allocation preserves distinct evidence by distributing the final frame budget across cues.

Table 2 measures these contributions under controlled settings. Adding refinement improves average accuracy by 1.78 points. Within refinement, replacing overview frames with cue-wave evidence adds 2.94 points. Allocating frames across the resulting cues adds 2.38 points over global top-k selection. On long videos, the final cues find stronger matches in the full candidate pool than in the overview, with a 53% higher median SigLIP2 score. Refinement also changes which frames the cues cover, with a median Jaccard overlap of 0.65 between the initial and final frame sets (Tables 5–6). Moreover, replacing our adjacent-frame segmentation with WFS-SB’s one changes accuracy from 80.63% to 80.18%, and removing CD, IS, and BO changes it to 79.48%. These further support the effectiveness of our design choices.

With all these evidences, we believe our analysis is comprehensive, and our motivations and effectiveness are clear and significant.


Question 2. Clarify Figure 1

Thanks for the suggestion. In our future revision, Figure 1 will trace one question from cue retrieval through refinement to the final frame set. Enlarging the cue-wave panel and using consistent cue labels, as in Figure 3, will show which frames inform each refinement action. We will mark the selected frames that support the answer and consider adding a reader panel to make that final step explicit.

AuthorsOpenReview ↗

Request for an Update | Other Reviewers Recommend Acceptance

We wanted to follow up on our rebuttal. We believe it addresses each of the points you raised. We added ablations for clip segmentation and the cue construction rules, reported selector latency and VLM costs, and outlined how we’ll clarify Figure 1.

Reviewer AMAs has maintained a rating of 3 (Weak Acceptance), and Reviewer JafD raised their rating to 3 (Weak Acceptance) after confirming that our rebuttal addressed all of their concerns.

Given the evidence in our rebuttal and the other two reviewers’ recommendations, could you explain what specific concerns still warrant a rating of 2 (Weak Rejection) in your assessment? If our rebuttal has resolved your concerns, we ask that you update your rating to 3 (Weak Acceptance).

Reviewer_AMAsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CueKFS selects video keyframes without additional training. A VLM generates visual cues and revises them using evidence retrieved by a frozen image and text encoder. The method allocates a fixed frame budget across cues. Experiments on three benchmarks, three answering models, and three frame budgets show gains over the reported baselines.

Critical strengths
  1. Table 1 shows consistent gains across evaluation settings. With GPT-5.5 as the answerer, improvements over WFS-SB averaged across frame budgets range from 2.42 to 4.06 percentage points.
  2. Table 2 shows benefits from cue refinement and budget allocation with Qwen3.6-27B as the reasoning VLM. Batching reduces median VLM calls from 11 to 2 while maintaining comparable accuracy: 81.04% versus 80.95%.
Critical weaknesses
  1. VideoAgent (Wang et al., 2024, Sections 3.2 to 3.4) already generates retrieval queries from captions of previously observed frames. CueKFS adds direct visual feedback and frame budget allocation. A comparison using the same models and budgets would help isolate the gains from these designs. Reference: https://arxiv.org/abs/2403.10517
  2. The Table 3 configuration uses an overview budget of 48 images and up to 32 evidence images per refinement batch. The paper reports VLM call counts; total latency and visual token use would make the efficiency assessment more complete.
Questions for authors
  1. How does CueKFS compare with a VideoAgent style baseline using the same models and input and output budgets?
  2. How does the full pipeline compare with WFS-SB in latency and visual token use?
  3. Which data split was used to select the Table 3 configuration?
2 comments

Weakness 1 / Question 1. Compare with VideoAgent

VideoAgent uses a separate VLM captioning step to convert retrieved frames into text before generating new queries. CueKFS passes the retrieved frames directly to the reasoner (lines 211–224), avoiding the captioning overhead discussed in lines 051–054 and preserving visual details that captions can omit. Whereas VideoAgent retrieves the highest-scoring frame, CueKFS uses Peaks and Plateaus to surface evidence for both brief and sustained matches (lines 191–209).

On Video-MME, both selectors use Qwen3.6-27B, the same SigLIP2 encoder and candidate frames, and the same question and answer options. We cap each selector at 8,000 input and 350 output tokens per question across all calls, including VideoAgent’s captioning. CueKFS uses its final configuration with the visual context capped to fit this budget. Both selectors pass 16 frames to the same GPT-5.5 reader.

SelectorVideo-MME accuracy (%)
VideoAgent-style selector78.93
CueKFS81.93

CueKFS gains 3.00 points over the VideoAgent-style selector under the same input, output, and frame budgets. Additionally, the controlled ablations in Table 2 already isolate two contributions within CueKFS. With the reasoner, reader, and frame budgets fixed, cue-wave evidence improves average accuracy by 2.94 points over generic scene frames, and Cue Budget Allocation improves it by 2.38 points over global top-k.


Weakness 2 / Question 2. Cost and compare with WFS-SB

Relative to WFS-SB, CueKFS adds a reasoning VLM that decomposes the question into separate retrieval cues and refines them using retrieved frames. Each cue can recover evidence for an individual subject or an implicit detail that WFS-SB's whole-question scoring can miss (lines 040–046 and 073–084).

Starting from decoded frames and precomputed embeddings, it takes 5.9 s/question for CueKFS and 0.2 s/question for WFS-SB on one H200. CueKFS uses up to 48 overview images and 32 evidence images for one refinement batch, or about 9,000 visual tokens at 448 × 256 when all 80 slots are used. With the same GPT-5.5 reader and 16-frame budget on Video-MME, CueKFS gains 3.78 points over WFS-SB using the final settings with Qwen3.6-27B. This significant performance improvement with moderate increased latency is a worthwhile trade-off.


Question 3. Table 3 protocol

The public Video-MME release contains only a test split, so Table 3 assesses sensitivity on the same split used in Table 1. We want to clarify that CueKFS already outperforms the strongest baseline before the hyperparameter sweep (81.52% vs. 78.64% with GPT-5.5 as the reasoner), and the selected settings reach 82.70%. The final configuration also generalizes beyond Video-MME. With no tuning on MLVU or LongVideoBench, CueKFS outperforms WFS-SB by 2.42 and 4.06 points, respectively.

Reviewer_AMAsOpenReview ↗

comment

The rebuttal addresses my main concern about the VideoAgent comparison. On Video-MME, CueKFS outperforms the VideoAgent-style baseline by 3.0 percentage points under matched token and frame budgets. The new latency and token measurements also clarify the accuracy–cost trade-off with WFS-SB. My remaining concerns are the scope of the matched comparison, which covers one benchmark and one frame budget, the lack of end-to-end latency measurements, and hyperparameter selection on the Video-MME test split. The gains before tuning and on other benchmarks support the paper’s contribution. Given the remaining limitations, I maintain my rating of 3 (Weak acceptance).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.