- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether frozen Video Language Models (VideoLLMs) already encode information about evidence readiness, i.e., whether sufficient visual evidence to answer a given question has been observed. The authors train linear probes on hidden representations of frozen VideoLLMs to predict evidence readiness, and evaluate the resulting signals across multiple models and video benchmarks. They further introduce Readiness Gating, a simple inference-time strategy that uses the predicted readiness scores to determine when to answer or which answer to select. Experimental results suggest that evidence readiness can be decoded from frozen representations and that readiness-based gating can improve answer accuracy in several evaluated settings.
Critical strengths
Evaluation Across Multiple Models and Datasets. The paper evaluates its proposed approach across multiple VideoLLMs and video datasets, providing a relatively broad experimental assessment.
Critical weaknesses
-
Excessive AI-assisted writing and poor readability. The manuscript exhibits extensive signs of AI-assisted writing, which significantly undermine its readability. Moreover, many experiments appear to be defensive analyses designed to anticipate potential reviewer concerns, rather than hypothesis-driven investigations. The writing and overall presentation raise concerns about the extent of substantive human revision and critical oversight.
-
Limited methodological novelty. The paper does not introduce a fundamentally new algorithm, optimization mechanism, or model architecture. Instead, it primarily relies on existing techniques, including linear probing of frozen representations and threshold-based gating. The methodological contribution is therefore limited.
-
Insufficient differentiation from prior work. What is the unique contribution of this paper compared with StreamReady [1], which has already investigated evidence readiness and answer timing in streaming video understanding? The authors should more clearly articulate the novelty and significance of their work beyond existing readiness-aware approaches.
-
Insufficient evaluation in realistic streaming scenarios. It remains unclear whether the proposed method can reliably handle continuous video streams in which the correct answer changes over time. For instance, RTV-Bench [2] introduces Multi-Timestamp Question Answering (MTQA), where answers evolve as the visual scene changes. Evaluating the proposed method under such dynamic conditions would provide stronger evidence of its robustness and practical applicability.
Reference [1] Azad, S., Vineet, V., & Rawat, Y. S. (2026). StreamReady: Learning What to Answer and When in Long Streaming Videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) [2] Xun, S., Tao, S., Li, J., et al. (2025). RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video. Advances in Neural Information Processing Systems (NeurIPS)
Questions for authors
- Novelty and differentiation from prior work. What is the key contribution of this paper compared with StreamReady [1]? Given that the proposed approach mainly relies on standard linear probing and threshold-based gating, what distinguishes it from existing readiness-aware methods?
- Generalization to realistic streaming scenarios. Can the proposed method reliably handle continuous video streams where answers change over time? For example, RTV-Bench [2] includes Multi-Timestamp Question Answering (MTQA) tasks that require tracking evolving visual information. How well would the proposed method generalize to such scenarios?
Reference [1] Azad, S., Vineet, V., & Rawat, Y. S. (2026). StreamReady: Learning What to Answer and When in Long Streaming Videos. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) [2] Xun, S., Tao, S., Li, J., et al. (2025). RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video. Advances in Neural Information Processing Systems (NeurIPS)
4 comments
Response to Reviewer 3K9a (2/3)
W3 / Q1. Difference from StreamReady.
The two papers ask different questions, which Section 2 now makes explicit:
| StreamReady | This paper | |
|---|---|---|
| Question | How to train a system that knows when to answer | Whether an unmodified model already represents that the evidence has arrived |
| What is trained | A readiness token and head in an added memory-and-retrieval module | One linear readout; the model stays frozen |
| Readiness labels | Pseudo evidence timestamps | Benchmark evidence timestamps, checked against human raters |
| Added inference cost | The added module | One dot product |
StreamReady shows that a trained readiness mechanism helps. We show that the state such a mechanism needs is already present in frozen models: it is linearly readable in seven models, depends on the question on identical frames, and is distinct from confidence and correctness.
We could not run StreamReady, because its weights are not released and its benchmark releases annotations only. Both papers report OVO-Bench, so we place them side by side on its forward group (Appendix R): three tasks, of which step recognition and clue reveal ask when to respond and the third asks for a count, where only the backbone acts. StreamReady scores 58.8 there, using no OVO-Bench labels. Under OVO-Bench's own protocol, a frozen Qwen3-VL-8B with our linear readout scores 58.6 with a readout trained on our other benchmarks and only its cutoff set on 8 labelled OVO videos, and 63.4 with the readout fitted on OVO labels from other videos. The plain backbones start level (49.9, against 48.7 for OVO's published Qwen2-VL-7B, the backbone StreamReady builds on). A linear readout of a frozen model therefore reaches the level of a trained readiness module on these tasks.
This is not a controlled comparison, since the backbones and the supervision differ. The controlled comparisons are with the released triggers we could run: MMDuet and Dispider in Section 4.5 and on SPOT-Bench, and Ctrl-SPOT, the best published SPOT-Bench system, in our reply to W4. We also compare the forward group only. On the backward group, and so on the overall average, StreamReady's memory module puts it ahead: those tasks need memory of earlier video, which a readout does not provide.
Response to Reviewer 3K9a (3/3)
W4 / Q2. Realistic streams, including answers that change.
Following your suggestion, we evaluated the readout on RTV-Bench: 552 streams of up to 3.1 hours, in which 1,052 question groups ask the same question several times with different correct answers.
Generalisation. A Qwen3-VL-8B probe fitted only on our other benchmarks reads RTV-Bench at 0.629 [0.618, 0.642] AUROC with its controls near chance (raw visual features 0.51, shuffled labels 0.48, elapsed time 0.55, confidence 0.57), retaining 92% of a probe refitted on RTV-Bench (0.681).
Tracking. Across 487 answer changes, the score rises while the new evidence is in view (+0.084 [+0.065, +0.103]) and returns to its earlier level afterwards (−0.009 [−0.028, +0.009]).
Selection. Our rule returns the answer from the highest-readiness window so far, which assumes a fixed answer, so we restrict it to windows ending in the last 32 seconds (a bound chosen on development videos; on test videos, overall accuracy is flat for any bound from 24 to 64 seconds). RTV-Bench asks each question at the moment its answer's evidence appears, so it never tests what happens once that moment has passed, which is when choosing a window matters. We therefore built a protocol from its annotations that asks each question again at later times. Asked 16 seconds or more after a change, on test videos:
| Rule (accuracy) | Qwen3-VL-8B | InternVL3.5-8B |
|---|---|---|
| Current 16-second window | 0.467 | 0.453 |
| Highest readiness so far (unbounded rule) | 0.435 | 0.404 |
| Highest readiness in the last 32 seconds | 0.497 | 0.493 |
That is +6.1 [+3.6, +8.7] and +8.9 [+6.5, +11.6] points over the unbounded rule, and +3.0 [+1.4, +4.6] and +4.0 [+2.5, +5.5] over the current window. We added the bounded rule to Section 3.3, the results to Sections 4.3 and 4.4, and Appendix S.
Complete streaming systems. Beyond RTV-Bench, the revision adds two standard streaming benchmarks, each scored by its own code. On SPOT-Bench (NeurIPS 2026 Datasets and Benchmarks), a system must decide every second whether to respond. The system's own trigger and our readout come from the same forward pass and are tuned on the same labelled videos, cross-fitted by video because SPOT-Bench has no training split; the readout also fits a linear direction on those labels, while the own trigger has only its decision rule tuned. T-F1@5 (timing-weighted F1), action boundaries / point of no return:
| System | Own trigger, tuned on SPOT labels | Readout, one fit for both tasks | Gain, both tasks |
|---|---|---|---|
| Qwen3-VL-8B | 30.29 / 16.24 | 36.36 / 22.05 | +5.93 [4.62, 7.24] |
| Qwen2.5-VL-7B | 27.86 / 14.38 | 33.58 / 20.76 | +6.05 [4.86, 7.23] |
| Ctrl-SPOT (best published entry) | 32.21 / 21.52 | 39.98 / 27.10 | +6.67 [5.31, 8.05] |
| MMDuet (released code) | 26.27 / 19.29 | 31.65 / 20.93 | +3.51 [2.33, 4.71] |
| Dispider (released code) | 22.62 / 13.87 | 28.61 / 17.16 | +4.64 [3.41, 5.95] |
When Ctrl-SPOT acts on the readout live, on all 608 videos, the gain is +5.06 [3.78, 6.38]. On OVO-Bench, the readout decides when to respond on the forward tasks (see our reply to W3). Both are in Section 4.5 and Appendix R.
Thank you again for your review. All new results are included in the revised version. We hope they address your concerns, would be grateful if you took them into account in your final assessment, and would be happy to answer any follow-up questions.
Dear reviewer 3K9a, We would like to thank you again for your time and effort.
We hope that our rebuttal has addressed your concerns. With the discussion period nearing its end, we would appreciate hearing your thoughts.
We would be happy to address any remaining concerns or provide further clarification. If our responses have resolved your concerns, we would greatly appreciate it if you could reconsider your score.
Response to Reviewer 3K9a (1/3)
Dear Reviewer 3K9a,
Thank you for your review and for acknowledging the breadth of our evaluation. We especially thank you for pointing us to RTV-Bench, which led us to extend our selection rule to streams whose answer changes; the extension is now part of the method. In short: on SPOT-Bench, a linear readout of a frozen model's state, used as the response trigger, beats the own trigger of each of five systems by +3.5 to +6.7 T-F1@5 with one readout for both tasks (+1.3 to +5.0 with one readout per task); StreamReady trains a module to decide when to answer, whereas we show that frozen models already encode the state such a module needs; and the readout transfers to RTV-Bench without retraining and follows the evidence as the answer changes.
W1. Writing and presentation.
Thank you for these comments on the writing and on how the analyses are presented; the revised version addresses each of them. We improved the readability of the entire paper by hand, with shorter sentences, one idea per sentence, one name for each concept, and one or two numbers per claim in the main text, moving robustness details to the appendix. We also made the logic of the analyses explicit. The Introduction now states four hypotheses: readiness is linearly readable in a frozen model, it depends on the question, it is distinct from confidence and correctness, and it helps decide when to answer. Each subsection of Section 4 now opens with its hypothesis, prediction, test and result, so that every control appears as the test its hypothesis requires, since a probing result can only be interpreted next to controls for what the probe itself can learn (Hewitt & Liang, 2019). We designed the experiments and verified every claim and number ourselves.
W2. Methodological novelty.
The paper's main contribution is a result about what video-language models represent: an unmodified model already encodes whether the evidence for the current question has arrived, separately from its confidence and correctness. The measurement design makes this result credible, and a training-free policy puts it to use. To choose which answer to return, the policy needs no threshold: it returns the answer from the window with the highest readiness score, and only deciding when to act uses a cutoff. We deliberately add no new architecture or training method: a linear probe can only read information the model already represents, so a positive result is evidence about the model. It also works inside complete systems: used as the response trigger on SPOT-Bench, with the same labelled videos as each system's own trigger, it raises timing-weighted F1 by +3.5 to +6.7 points with one readout for both tasks (+1.3 to +5.0 with one readout per task) on five systems, including MMDuet, Dispider and the best published entry, at the cost of one dot product (table in our reply to W4). The Introduction now states this contribution type and lists the SPOT-Bench result among the contributions.