acceptodds
Under review as a conference paper at ICLR 2027

Beyond Frame Selection: Decoupled Adaptive Evidence Selection for Video Understanding with MLLMs

Abstract

In video understanding, MLLMs often need to reason over visual content whose relevance to a question varies over time. Existing frame selection and visual-token pruning methods typically rely on a unified relevance ranking or fixed selection granularity, overlooking that visual-text and ordinary visual content can exhibit different relevance distributions and contribute differently to different questions. We introduce DAES, a training-free framework for Decoupled Adaptive Evidence Selection. DAES uses reflection-based cues to disentangle visual-text and ordinary visual evidence at the patch level, and estimates their query-conditioned relevance through Value-space similarity between query tokens and visual patches in an MLLM. To make the two evidence types directly comparable, we calibrate their relevance against type-specific null distributions. Under a fixed token budget, DAES adaptively selects evidence types and timestamps according to calibrated relevance scores. This evidence-level selection enables broader temporal coverage under fixed token budget while preserving question-relevant information. For efficient selection, we use a lightweight MLLM as the evidence selector while reserving a larger MLLM for final reasoning. Experiments on subtitle-rendered Video-MME-v2 and LongVideoBench demonstrate consistent improvements in video understanding performance under the same token budget.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Models, Video Understanding, Evidence Selection, Visual Token Efficiency, Text-Visual Disentanglement

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_N6zhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DAES, a training-free framework for selecting video evidence under a fixed visual-token budget. Instead of selecting complete frames, it separates each frame into ordinary visual and visual-text evidence using reflection-based cues. It estimates option-conditioned relevance in an MLLM’s Value space, calibrates scores using evidence-specific null distributions, and selects evidence across answer options and timestamps. Evaluation on subtitle-rendered Video-MME-v2 and LongVideoBench shows improvements over frame-selection baselines. The paper also includes component ablations, an original-video experiment, and transfer to different answer models.

Critical strengths
  1. A well-motivated alternative to whole-frame selection. Selecting visual-text and ordinary visual evidence independently allows the model to retain different information at different timestamps without paying for every complete frame. The type-specific calibration addresses the problem that raw relevance scores for these evidence types may not be directly comparable. Together, these choices form a coherent approach to allocating limited visual context, rather than simply increasing the number of input frames.
  2. Meaningful gains with useful supporting experiments. With the same Qwen3.5-9B answerer and comparable visual-token budgets, DAES improves over the strongest reported frame-selection baseline by 4.63 percentage points on Video-MME-v2 and 2.19 points on LongVideoBench. The component ablations provide evidence that the main design choices help. Reusing the selected evidence with Qwen3.5-27B and InternVL3.5-8B also produces gains, suggesting that the selected evidence is useful beyond a single answerer.
Critical weaknesses
  1. The benefit of V/T decoupling and its broader applicability are not sufficiently established. The Full32 ablation changes both V/T decomposition and selection granularity, so it does not establish whether explicit V/T separation is better than simpler token selection. The main evaluation also uses subtitle-rendered multiple-choice videos, while the original-video experiment compares only against uniform sampling. The Q-only ablation does not evaluate open-ended answering. Please evaluate a clearly specified question-only adaptation on VideoEval-Pro, an open-ended long-video QA benchmark. Include matched full-frame and type-agnostic token-selection baselines, controlling the selector, relevance computation, candidate pool, answerer, and token budget. This would clarify whether V/T decoupling itself remains useful without constructed subtitles or predefined answer options.
  2. Evidence selection may depend on how answer options are written and ordered. DAES computes option-specific relevance and cycles through options when selecting evidence. However, evidence matching a distractor may not help distinguish the correct answer, potentially wasting budget or making selection sensitive to option wording and order. Please test meaning-preserving distractor paraphrases and option permutations only in the selector input, keeping the video, question, and answerer inputs fixed. Reporting changes in selected evidence and final QA performance would establish whether the gains are robust to these irrelevant changes in presentation.
  3. The practical efficiency trade-off is unclear. Matching the answerer’s visual-token budget does not account for DAES’s additional processing of original and reflected frames or its option-conditioned scoring. Covering more timestamps may also increase answerer-side visual encoding before masking. Without end-to-end measurements, it is unclear whether the accuracy gains justify the additional computation. Please report latency and peak memory against the main baselines on the same hardware, separating preprocessing, evidence selection, and answering. Distinguish one-time calibration and reusable per-video work from per-question costs, including the case where multiple questions share a video.
Questions for authors

See weakness.

6 comments

W1. V/T decoupling, original-video inputs, and open-ended QA

Controlling for patch-level selection and answer options.

We agree that the original Full32 ablation changes both evidence typing and selection granularity. We therefore introduce Unified Patch, which ranks individual patches by option-conditioned relevance without V/T separation, using the same model configuration and visual-token budget (Tables 19–20).

MethodVideo-MME-v2 (%)LVB (%)
Unified Patch38.5671.05
DAES40.8873.37
Gain (pp)+2.32+2.32

DAES consistently outperforms Unified Patch on both benchmarks, demonstrating the benefit of V/T decoupling beyond generic patch-level selection.

We further conduct a controlled ablation on VideoEval-Pro to examine whether this benefit depends on predefined answer options. Unlike multiple-choice QA, evidence selection here is guided solely by the question, without candidate answer options. All methods use the same Qwen3.5-2B selector and Qwen3.5-9B answerer.

MethodVideoEval-Pro Accuracy (%)
Full-frame Selection (no V/T)37.94
Unified Patch (no V/T)38.63
DAES40.26
Gain over Unified Patch (pp)+1.63

DAES further improves over Unified Patch by 1.63 pp, showing that the benefit of V/T decoupling persists without predefined answer options. Together, these results support the contribution of V/T decoupling beyond both generic patch-level selection and option-conditioned relevance estimation.

Original RGB with competitive baselines.

We further extend the original-video evaluation to multiple baselines at 16, 32, and 64 FFE (Table 3). Results at 32 FFE are summarized below:

MethodOriginal Video-MME-v2 (%)Original LVB (%)
Uniform3230.0064.40
AKS33.6366.79
Q-Frame31.8467.31
WFS-SB32.9169.18
VTR-VLM31.9467.24
QCA32.3868.59
FlashVID32.1667.99
V-CAST29.3168.74
DAES34.5670.53

Original RGB inputs contain no additionally rendered subtitles, although naturally occurring scene text and embedded subtitles remain. DAES outperforms the strongest competing baseline by 0.93 pp on Video-MME-v2 and 1.35 pp on LVB. These results demonstrate that its benefits also extend to original video inputs, although the gains are smaller than with subtitle rendering.

Open-ended QA.

DAES mainly focuses on decoupling Visual and Textual Evidence and adaptively selecting relevant patches based on Query–Patch relevance in the MLLM Value space. Since evidence selection can be guided directly by the question, it does not inherently require predefined answer options. We evaluate this capability on VideoEval-Pro. The model receives only the video and question and generates free-form answers, evaluated against reference answers using GPT-6.1 Sol for semantic correctness.

MethodVideoEval-Pro Accuracy (%)
Uniform3227.31
AKS36.93
Q-Frame37.16
WFS-SB34.76
VTR-VLM33.51
V-CAST33.28
DAES40.26

DAES outperforms Q-Frame by 3.10 pp and Uniform32 by 12.95 pp, demonstrating its effectiveness in open-ended video QA.

W2. Sensitivity to option order and distractor wording

Tables 4(b) and 22 report the results under randomized option order and meaning-preserving distractor paraphrases. Importantly, these perturbations are applied only to the selector inputs, while the answerer inputs remain unchanged, allowing us to isolate their effects on evidence selection.

Appendix B.3 provides representative paraphrases, including active/passive reformulations (e.g., "The boy is hugged by his sister" to "His sister hugs the boy"), synonym substitutions (e.g., "closes the door" to "shuts the door"), and minor word-order changes (e.g., "A crisper and drier texture" to "A drier and crisper texture"). These transformations preserve the original meaning while changing the surface wording.

SettingAccuracy (%)Change (pp)Patch Overlap (%)Timestamp Overlap (%)
Original DAES40.88———
Randomized option order40.72-0.1695.2595.79
Distractor paraphrases40.59-0.2989.3288.96

The accuracy drops are small (0.16 and 0.29 pp), while the selected patches and timestamps maintain high overlap with the original DAES selections. These results demonstrate that DAES is robust to option-order and distractor-wording perturbations, both in terms of final QA performance and evidence selection stability.


W3. End-to-end efficiency, memory, and reusable computation

We have added a comprehensive efficiency comparison in Table 7 of the revised manuscript. All methods are evaluated on the same RTX PRO 6000 Blackwell (96 GB), using the BF16 Qwen3.5-9B answerer with batch size one. We report the latency of three stages: Selector (including selector-side visual encoding and selection), Vision (answerer-side visual encoding), and LLM (prefill and decoding). For end-to-end latency, Q-Level measures independent per-question inference, while V-Level amortizes shared video features across four questions from the same video.

MethodAcc. (%)Selector (s/Q)Vision (s/Q)LLM (s/Q)Q-Level E2E (s/Q)V-Level E2E (s/Q)Peak Mem. (GB)
Uniform3232.160.0000.2101.3472.2041.59821.20
AKS35.002.2150.2001.3689.8574.04125.64
Q-Frame34.660.7030.4891.3505.1132.59225.16
WFS-SB36.002.9050.4671.3655.6344.78525.96
VTR-VLM36.254.6430.1741.36920.6546.38643.67
QCA35.282.2150.2071.3699.9074.07525.64
FlashVID37.471.5970.0001.3268.0903.24726.01
V-CAST32.311.4050.0001.3147.3113.01924.43
Unified Patch38.561.5151.1391.39111.3444.24426.81
DAES (2B)40.881.6670.5141.3358.4713.56127.59

DAES achieves the highest accuracy among the compared methods, with competitive end-to-end latency and moderate memory consumption. Its broader temporal coverage (59.34 vs. 32 timestamps per question) incurs additional answerer-side visual encoding compared with Uniform32 (0.514 vs. 0.210 s/Q), but contributes to an 8.72 pp accuracy improvement under the same visual-token budget.

Compared with Unified Patch, DAES further improves accuracy by 2.32 pp while reducing answerer-side visual encoding time from 1.139 to 0.514 s/Q. This is consistent with its more selective temporal coverage (59.34 vs. 255.77 timestamps per question). Despite slightly higher selector latency, DAES achieves lower end-to-end latency at both Q-Level (8.471 vs. 11.344 s/Q) and V-Level (3.561 vs. 4.244 s/Q), with comparable peak memory consumption.

Regarding reusable computation, video-level feature sharing reduces DAES's amortized latency from 8.471 to 3.561 s/Q, while query-dependent selection and answer generation remain independent. These results provide a comprehensive evaluation of DAES's accuracy–efficiency trade-off.

Reviewer_N6zhOpenReview ↗

Response

Thank you for the detailed rebuttal and additional experiments. The new results address some of my original concerns, and I have three remaining clarification questions:

  1. For VideoEval-Pro setup, were the videos evaluated in their original form without additional subtitle rendering? Did all methods use the same visual-token budget?

  2. For the original-video and VideoEval-Pro experiments, were the null distributions reused from the main experiments or constructed separately? If reconstructed, what data were used for calibration?

  3. In Table 7, the sum of Selector, Vision, and LLM latency is much smaller than the Q-Level end-to-end latency. What accounts for this difference?

Overall, I appreciate the authors' efforts to address my concerns. These clarifications would help me better assess the new results.

Reviewer_FyYGOpenReview ↗
Rating
1(−1)
Confidence
3(−1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Decoupled Adaptive Evidence Selection (DAES), a training-free framework for video understanding under a fixed visual token budget. The method separates visual and textual evidence at the patch level using reflection-based cues, estimates option-aware patch relevance through Value-space similarity, and applies evidence-specific null calibration to make the relevance scores of different evidence types comparable. It then performs option-balanced evidence selection across timestamps and candidate answers. The authors evaluate DAES on Video-MME-v2 and LongVideoBench, reporting improvements over several frame-selection baselines under comparable token budgets. Additional experiments include component ablations, token-budget analysis, and cross-model generalization.

Critical strengths

1. Novel Evidence Selection Strategy. The authors explicitly consider textual information as a distinct type of evidence for video evidence selection, which is a novel aspect compared with conventional frame-selection methods. Furthermore, the proposed reflection-based decomposition effectively distinguishes textual evidence from ordinary visual evidence without relying on OCR or external tools.

2. Effective Evidence Selection Strategy. Instead of simply selecting the top-k evidence candidates based on relevance scores, the proposed method adopts an option-balanced cyclic selection strategy, selecting evidence across different answer options until the token budget is exhausted. This design encourages balanced evidence coverage across candidate answers.

Critical weaknesses

1. Misalignment Between Challenges and Contributions. The paper emphasizes that different queries require different types of evidence. However, the proposed method mainly focuses on separating textual and visual evidence, which does not sufficiently address the broader challenge presented in the introduction.

2. Overly Benchmark-Oriented Method Design. The proposed method heavily relies on predefined answer options for evidence selection. However, in real-world scenarios, users typically ask open-ended questions without providing candidate answers. Benchmarks should serve as tools to evaluate a method's capabilities rather than dictate its design. This dependency significantly limits the method's practical applicability and generalizability.

Questions for authors

In addition to the concerns raised in the Critical Weaknesses section, I have the following questions:

1. Insufficient Ablation Studies. Why are sensitivity analyses missing for several key hyperparameters, such as the top 5% textual seeds, bottom 40% visual seeds, reflection threshold (2.75), and selected MLLM layers? Additionally, can the authors isolate the contribution of Visual/Textual Decoupling from that of patch-level selection?

2. Limited Baseline Comparisons. How does DAES compare with more recent and competitive frame-selection methods, as well as query-aware visual-token selection approaches?

3. Limited Token Budget Comparisons. The main comparisons use a fixed token budget equivalent to 32 frames. How does DAES compare with other methods under different budgets, such as 16 and 64 frames?

4. Computational Efficiency. What is the computational overhead of the proposed method? Can the authors report evidence selection time and end-to-end inference latency compared with existing methods?

5. Limited Cross-Model Evaluation. Although the paper provides results using Qwen3.5 and InternVL3.5, comprehensive comparisons across different MLLM architectures remain limited. Can the authors evaluate DAES with additional models to demonstrate its generalizability?

5 comments

W1. Alignment between the stated challenge and contribution

As noted in the Conclusion, DAES currently focuses on two broad evidence types, Visual Evidence and Textual Evidence, while exploring richer evidence categories offers an interesting direction for future work. Within this scope, our work addresses a specific evidence-allocation problem: the two evidence types exhibit distinct query-relevance distributions and contribute differently to different questions. DAES addresses this problem by separately modeling the two evidence types, calibrating their relevance scores, and adaptively allocating a fixed visual-token budget across evidence types and timestamps. This allows DAES to move beyond unified relevance ranking and perform more fine-grained evidence selection under a limited token budget.


W2. Dependence on predefined answer options and open-ended applicability

We need to clarify that DAES focuses on decoupling Visual Evidence and Textual Evidence, and adaptively selecting relevant patches based on Query–Evidence relevance in the MLLM Value space. This mechanism allows evidence selection to be conditioned directly on the question, without necessarily relying on predefined answer options. To investigate its applicability to open-ended QA, we further evaluate DAES on VideoEval-Pro, where the model receives only the video and question and generates free-form answers. Generated answers are evaluated against reference answers by GPT-6.1 Sol for semantic correctness.

MethodVideoEval-Pro Accuracy (%)
Uniform3227.31
AKS36.93
Q-Frame37.16
WFS-SB34.76
VTR-VLM33.51
V-CAST33.28
DAES40.26

DAES outperforms Q-Frame by 3.10 pp and Uniform32 by 12.95 pp, demonstrating its effectiveness in open-ended video QA without predefined answer options.


Q1. Hyperparameter sensitivity and the contribution of V/T decoupling

Hyperparameter sensitivity. We have added a comprehensive hyperparameter sensitivity analysis in Appendix B.5, Table 25, covering textual/visual seed ratios, the Peak-Z threshold, and relevance-layer selection. The results show that DAES is relatively stable to seed-ratio and threshold variations, while the choice of relevance layers has a more noticeable impact. Our default configuration achieves the best overall performance among the tested settings.

Contribution of V/T decoupling. To further examine the contribution of V/T decoupling beyond patch-level selection, we introduce Unified Patch, which ranks all patches directly using Query-Patch token relevance without separating Visual and Textual Evidence. It shares the same selector, Value-space relevance computation, candidate frames, answerer, and visual-token budget as DAES (Appendix B.2, Tables 19–20).

Selection methodVideo-MME-v2 (%)LongVideoBench (%)
Unified Patch38.5671.05
DAES40.8873.37
Gain (pp)+2.32+2.32

DAES consistently outperforms Unified Patch on both benchmarks, demonstrating the benefit of explicitly separating and allocating Visual and Textual Evidence, beyond generic patch-level relevance selection.


Q2. Recent frame-selection and visual-token baselines

We further introduce three recent baselines covering both frame-level and token-level selection: QCA for frame selection, and FlashVID and V-CAST for token selection.

At approximately 32 FFE, the subtitle-rendered results are:

MethodSelection TypeVideo-MME-v2 (%)LongVideoBench (%)
Uniform32Frame sampling32.1665.37
AKSFrame selection35.0068.29
Q-FrameFrame selection34.6668.06
WFS-SBFrame selection36.0069.63
VTR-VLMFrame selection36.2571.18
QCAFrame selection35.2869.56
FlashVIDToken selection37.4767.39
V-CASTToken selection32.3169.78
DAES (2B)Evidence selection40.8873.37

All methods use the same answerer and comparable answerer visual-token budgets. DAES achieves the best performance on both benchmarks.

Q3. Comparisons under 16- and 64-frame budgets

We further evaluate DAES under 16, 32, and 64 frame-equivalent (FFE) budgets, using both subtitle-rendered and original Videos (Table 3). Original Videos contain no additionally rendered subtitles, while naturally occurring scene text remains. Each entry reports subtitle-rendered / original accuracy (%):

MethodMME 16 FFEMME 32 FFEMME 64 FFELVB 16 FFELVB 32 FFELVB 64 FFE
Uniform28.38 / 27.4432.16 / 30.0034.72 / 32.6962.08 / 60.7365.37 / 64.4068.44 / 67.99
AKS32.50 / 31.2835.00 / 33.6337.25 / 34.0966.42 / 65.5968.29 / 66.7970.23 / 69.04
Q-Frame32.75 / 30.0934.66 / 31.8437.09 / 32.9466.34 / 64.3268.06 / 67.3170.31 / 68.96
WFS-SB33.34 / 30.7536.00 / 32.9138.97 / 34.7267.69 / 67.1769.63 / 69.1871.80 / 70.91
VTR-VLM32.91 / 30.3436.25 / 31.9436.91 / 33.6965.67 / 65.2271.18 / 67.2470.38 / 69.93
V-CAST28.19 / 26.1932.31 / 29.3137.03 / 32.9466.64 / 66.8769.78 / 68.7471.58 / 70.08
DAES38.72 / 33.5940.88 / 34.5641.66 / 35.7871.13 / 69.2673.37 / 70.5373.00 / 72.18

DAES achieves the best accuracy across all tested budgets and input settings. It maintains clear advantages even at 16 FFE, demonstrating robustness to reduced token budgets. Notably, it also improves performance on original videos, suggesting that DAES can exploit naturally occurring scene text and subtitles without additional rendering.


Q4. Selection overhead and end-to-end inference

Table 7 in the revised manuscript reports selector latency, end-to-end (E2E) latency, and peak GPU memory on an RTX PRO 6000 Blackwell (96 GB).

MethodAccuracy (%)Selector Latency (s/Q)E2E Latency (s/Q)Peak Mem. (GB)
AKS35.002.2154.04125.64
Q-Frame34.660.7032.59225.16
WFS-SB36.002.9054.78525.96
VTR-VLM36.254.6436.38643.67
QCA35.282.2154.07525.64
FlashVID37.471.5973.24726.01
V-CAST32.311.4053.01924.43
DAES (2B)40.881.6673.56127.59

DAES achieves the highest accuracy among the compared methods, with competitive E2E latency and moderate memory consumption. Its latency is lower than AKS, WFS-SB, VTR-VLM, and QCA, while delivering higher accuracy. These results demonstrate a favorable trade-off between selection overhead, inference efficiency, and video understanding performance.


Q5. Generalization to additional MLLM architectures

We further add Gemma 4 E4B and GLM-4.6V-Flash to the answerer-transfer experiments (Table 5). DAES consistently improves over Uniform32 across four MLLM architectures, further demonstrating its generalizability across different models.

AnswererUniform32 (%)DAES (%)Gain (pp)
Qwen3.5-9B32.1640.88+8.72
Qwen3.5-27B36.3147.09+10.78
InternVL3.5-8B28.1635.41+7.25
Gemma 4 E4B25.0031.56+6.56
GLM-4.6V-Flash30.3839.31+8.93

Thank you for your constructive review. As the rebuttal period is approaching its end, we would appreciate it if you could take a look at our response and additional experiments and let us know whether they have addressed your questions.

We look forward to your feedback and would be happy to provide any further clarifications if needed.

Reviewer_FGz6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses the input information selection problem in long video understanding. Different from the general frame-level selection approaches, the paper introduces a fine-grained evidence selection that works on patch level. The proposed method, DAES, first distinguishes the visual evidence and textual evidence appeared in the video. Then, according to each option, it determines the relevance of these evidences respectively. Finally the visual ranking and the textual ranking are calibrated and the input sequence is determined. The evaluation is conducted on VideoMME-v2 and LongVideoBench, where improvements over the uniform baseline and other frame selection methods are demonstrated. Further ablation study and answerer generalization experiments are provided to support the effectiveness of the method.

Critical strengths
  1. The proposed decoupling of visual evidence and textual evidence addresses a practical yet overlooked issue in long video understanding, and the solution of using refection sensitivity for the decoupling is elegant. The authors indicate the importance of such decoupling nicely via the demonstration in figure 5.
  2. The proposed method achieves decent performance improvement on (one of) the most recent benchmark VideoMME-v2, which indicates its effectiveness.
Critical weaknesses

None. However, there are several minor issues which I will raise in the question.

Questions for authors
  1. If I understand it correctly, the proposed method uses both the MirrorCLIP (for frame decomposition) and the Qwen3.5-0.8B/2B selector (for measuring relevance). These models should be stated clearly in the main paper as implementation details. Moreover, does a selector larger than Qwen3.5-2B yield better selection?
  2. What is the inference speed of DAES compared to frame-level selection methods like AKS?
  3. In equation (6), for the two notations vi,k,glv^l_{i,k,g} and vt,p,glv^l_{t,p,g}, what do ii and tt stand for?
  4. According to table 1 and table 4, a large model Qwen3.5-27B achieves a larger improvement from the Uniform32 baseline than a smaller Qwen3.5-9B model. Is this a coincidental result, or could the authors provide an explanation to this?
  5. I don't fully understand the decomposition quality section. Could the authors explain how these scores are obtained more specifically?
5 comments

Q1. Which models perform decomposition and relevance estimation? Does a larger selector help?

We need to clarify that DAES performs both evidence decomposition and relevance estimation within a frozen Qwen3.5 selector, without requiring an additional CLIP or MirrorCLIP model. MirrorCLIP motivates the use of reflection sensitivity as a cue, which we extend to patch-level evidence decomposition for MLLMs. Specifically, the selector's visual encoder processes both original and horizontally reflected frames. After spatial realignment and local smoothing, we compute the normalized L2 feature discrepancy in Eq. (1), identify high-confidence seeds, and construct visual and textual prototypes. The same selector then performs Option-aware Patch Relevance in the MLLM Value space.

Our default configuration uses Qwen3.5-2B as the selector and Qwen3.5-9B as the answerer, with Qwen3.5-0.8B as a lightweight selector variant. Section 3.2 further clarifies the connection to MirrorCLIP. To investigate the effect of selector capacity, we additionally compare three selector sizes in Appendix B.4, Table 24, while keeping the answerer fixed to Qwen3.5-9B:

SelectorVideo-MME-v2 Accuracy (%)
Qwen3.5-0.8B39.34
Qwen3.5-2B40.88
Qwen3.5-9B41.69

The 9B selector improves accuracy by 0.81 points over the 2B variant, indicating that a larger selector can provide further gains. Nevertheless, the 2B selector achieves comparable performance with a substantially smaller model, offering a favorable balance between model capacity and efficiency. We therefore retain Qwen3.5-2B as our default selector.

Q2. How fast is DAES compared with AKS?

Table 7 in our revised manuscript compares the accuracy and latency of different methods, measured on the same hardware with Qwen3.5-9B as the answerer.

MethodAccuracy (%)E2E Latency(s/Q)
AKS35.004.041
Q-Frame34.662.592
WFS-SB36.004.785
VTR-VLM36.256.386
QCA35.284.075
FlashVID37.473.247
V-CAST32.313.019
Unified Patch38.564.244
DAES (2B)40.883.561

DAES achieves the highest accuracy with competitive inference latency, outperforming AKS and several other baselines in both accuracy and latency. Further details are provided in Appendix A.9.

Q3. What do i and t denote in Eq. (6)?

Here, (i) denotes the candidate answer option, and (t) denotes the sampled video timestamp. Thus, vi,k,gℓv^\ell_{i,k,g} is the Value vector of query token k in the question–option query for option i, whereas vt,p,gℓv^\ell_{t,p,g} is the Value vector of visual patch p at timestamp t. Here, ℓ\ell denotes the MLLM layer and g the key–value head. These definitions are now explicit immediately after Eq. (6) in the revised manuscript.


Q4. Why does the 27B answerer obtain a larger gain than the 9B answerer?

We use the same frozen evidence selected by the 2B selector for both answerers, without recomputing relevance or evidence selection for the larger model (Appendix A.7):

AnswererUniform32 (%)DAES (%)Gain (pp)
Qwen3.5-9B32.1640.88+8.72
Qwen3.5-27B36.3147.09+10.78

The larger gain may indicate that the 27B answerer can make better use of the selected evidence, allowing improvements in evidence quality to translate into greater accuracy gains despite its higher baseline.


Q5. How are the decomposition-quality scores computed?

Appendix A.6 provides the evaluation details. We use rendered subtitle regions as partial annotations to evaluate whether DAES correctly identifies Textual Evidence at the patch level.

We report two metrics. Area coverage measures the proportion of annotated subtitle area captured by DAES, with Micro averaging across all frames and Macro averaging per frame. Patch recall measures the proportion of subtitle-containing patches identified as Textual Evidence, evaluated at three overlap thresholds: any overlap, at least 25%, and at least 50%.

MetricDefinitionResult (%)
Micro area coverageOverall subtitle area coverage86.10
Macro area coverageAverage per-frame coverage79.62
Patch recall, any overlapSubtitle overlap >0>083.59
Patch recall, at least 25%Subtitle overlap ≥25%≥25\%87.57
Patch recall, at least 50%Subtitle overlap ≥50%≥50\%92.77

These are positive-only coverage and recall metrics. Since the annotations do not cover all natural scene text, signs, or interface text, unannotated predicted textual patches cannot necessarily be treated as false positives.

Thank you for your constructive review. As the rebuttal period is approaching its end, we would appreciate it if you could take a look at our response and additional experiments and let us know whether they have addressed your questions.

We look forward to your feedback and would be happy to provide any further clarifications if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.