- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper proposes D$^3$-Observe, a training-free VideoQA framework with two routes. The per-question route derives rule-based evidence axes from the question and all answer options, then supplements uniformly sampled frames using lightweight visual cues. The shared route jointly answers several questions about the same long video using one uniformly sampled 32-frame context. Both routes complete observation allocation before answering directly from visuals. The paper evaluates accuracy, amortized input costs, and the effects of replacing visual inputs with textual representations.
Critical strengths
-
The motivation is somehow relevant. Question relevance does not necessarily identify the visual evidence needed to distinguish answer options. Preserving such evidence under a limited observation budget is a worthwhile objective. Actually, the problem settings have some practical impacts.
-
The experimental controls and uncertainty estimates are useful. The fixed-input interface study includes a same-backbone textual control, and the video-level bootstrap accounts for dependence among questions from the same video.
Critical weaknesses
-
The central selection mechanism is insufficiently specified. The manuscript does not explain the complete rules for constructing evidence axes, the operational definitions and normalization of the visual cues, their weights, or the criteria governing adaptive budgets and coverage constraints. Crucially, it remains unclear how semantic distinctions between answer options change the ranking of RGB frames. Appendix C still expresses these essential operations as unexpanded functions. Consequently, the proposed core mechanism cannot be adequately reproduced or assessed.
-
The claimed methodological novelty is not sufficiently established. “Delayed commitment” operationally means fixing the frame set before a direct visual answering call, which also describes the uniform-frame baseline. The textual-interface experiments assess the consequences of particular intermediate representations; they do not isolate the effect of avoiding premature answer commitment. The shared route implements joint answering over uniform frames. A substantive technical advance therefore depends heavily on the difference-aware selector, whose distinctive mechanism remains underdefined.
-
The shared-route comparisons do not adequately isolate the source of the gains. On Video-MME and LVBench, the method uses uniform 32-frame contexts without difference-aware selection. Comparing joint question batches against per-question uniform inference simultaneously changes observation amortization and the textual context available to the model. Table 7 provides a related MLVU ablation, but its replacement policy is insufficiently described, and equivalent controlled comparisons are not reported across benchmarks. The reported reductions establish transmitted-frame savings under the chosen workload, while complete computation savings—including local decoding and selection—remain insufficiently characterized.
-
The ablations leave a consequential interaction unexplained. In Table 7, difference coverage improves accuracy by 4.22 points with shared observation enabled, but its effect becomes −0.05 points when sharing is disabled. Since the shared route itself bypasses difference-aware selection, this interaction requires results stratified by original route eligibility and precise descriptions of all four configurations. The current analysis does not establish when the proposed selector helps or explain the claimed complementarity.
-
The presentation obscures the algorithm, and the mechanism analysis is inadequate. The method interleaves task definitions, accounting conventions, workload routing, and abstract principles before making the computation clear. Meanwhile, the qualitative examples primarily document inputs and successful answers, without showing how option distinctions caused useful frames to be added. Direct evidence-coverage measurements, controlled option-conditioned selection comparisons, and actual selection traces are needed to substantiate the paper’s central explanation.
Questions for authors
Despite this paper points out a potentially practical problem for Video-VLMs in some industrial settings, its presentation cannot support a well-written paper and the methodology leaves many points unexplained.Therefore I cannot recommend any accept to this submission since it still need lots of technical and writing improvements.