- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CARDANO, a training formulation for streaming video language models that learns when to stop watching by predicting whether further observation would change the current answer. It trains a lightweight commit head using self-supervised targets obtained by comparing the model's predictions on partial and full videos. Experiments across multiple video benchmarks show that CARDANO reduces the fraction of video watched while preserving accuracy and outperforming causal fixed-budget baselines at matched frame budgets.
Critical strengths
- Relevant and pratical problem.
- Lightweight and training-efficient mechanism.
- Extensive benchmark evaluations
Critical weaknesses
-
The novelty over existing consistency-based early-exit methods is insufficiently established, and the independent contribution of the stopping mechanism remains unclear.
-
The experimental comparisons and efficiency analysis do not adequately isolate the benefits of adaptive stopping from answerer training and computational overhead.
-
Several methodological explanations, implementation inconsistencies, and reproducibility issues weaken confidence in the reported conclusions.
Questions for authors
-
The novelty of CARDANO requires stronger differentiation from existing adaptive inference and consistency-based stopping methods. [1] already trains a lightweight classifier using agreement between intermediate and final predictions, which closely resembles CARDANO's self-supervised target. [2] also learns adaptive stopping decisions over video frames, while [3] addresses when sufficient evidence has arrived to answer questions in streaming videos. What is the fundamental technical contribution beyond transferring an existing consistency-based stopping objective from network depth to temporal observation? More direct comparisons and a clearer discussion of these closely related methods would strengthen the novelty claim.
-
The evaluation does not sufficiently separate the contribution of answerer training from that of the commit head. The abstract reports improvements of 8.4 and 3.8 points over the untrained full-video backbone, but these gains may primarily result from prefix-based answerer training rather than adaptive stopping. The trained answerer's whole-video accuracy is not explicitly reported in the main tables, making it difficult to assess the actual accuracy cost or benefit of stopping. Could the authors add a "trained answerer, whole clip" baseline for every model and benchmark, and distinguish improvements from answerer fine-tuning and stopping? The comparison with tuned confidence thresholds also shows very small accuracy differences in several settings. A global confidence or patience policy fitted on the same training trajectories, with paired confidence intervals, would provide a more convincing comparison.
-
The stopping policy's claim of requiring no threshold tuning deserves clarification. Although the commit head uses a fixed decision threshold of 0.5, its positive-class weight controls how aggressively the model stops, and the paper deploys a particular operating point selected from four trained heads. How was this configuration selected, and which data were used? Since class weighting can effectively shift the decision boundary, the distinction from threshold-based stopping is less clear than suggested. Furthermore, the self-supervised target is defined by agreement with the full-video prediction rather than correctness. Could the authors separate false commits caused by inaccurate head predictions from cases where the full-video prediction itself is incorrect, and report their frequencies across benchmarks?
-
The released code and evaluation protocol contain substantial inconsistencies that limit reproducibility. The provided implementation has missing dependencies, including "stage_data" and "imagine_head", while "cardano/commit_head.py" uses "math.log" without importing "math". Several training settings also differ from the paper, including the default future-window offset, the direction of the KL objective, feature normalization, and tie-breaking. More importantly, the data loaders use custom evaluation partitions rather than the official benchmark test splits, and the MVBench evaluation includes videos overlapping with the answerer training data. The released materials also lack trained weights, trajectory dumps, and scripts needed to reproduce the main tables.
-
Some methodological explanations and practical efficiency claims require stronger evidence. Section 3.2 infers limited predictability of future video information from a negative correlation between consecutive representation differences, but this does not establish that answer-relevant future information is unpredictable. The adjacent-window representation similarities reported in Appendix F also suggest a more nuanced interpretation. Could the authors provide stronger evidence supporting this mechanism and clarify the content-prediction baseline? Furthermore, Appendix J shows that CARDANO incurs 29-35% more wall-clock time than a fixed-budget method processing the same number of windows, and its relative runtime can exceed that of full-video inference. Since efficiency is a central motivation, comparisons at matched wall-clock latency, rather than only matched frames, would better establish the practical benefit.
-
Several numerical, presentation, and bibliographic inconsistencies should be corrected. Tables 1 and 7 report slightly different CARDANO results for apparently identical settings, while some figure captions disagree with the corresponding visualizations or experimental descriptions. The qualitative examples also emphasize early successful commits without representing typical later stopping decisions. The bibliography contains several incorrect publication years and author lists, including the citation of STAR as a 2024 paper rather than its NeurIPS 2021 publication. Several references also retain outdated preprint information despite formal publication.
[1] Schuster, Tal, et al. "Confident Adaptive Language Modeling." NeurIPS. 2022.
[2] Ghodrati, Amir, Babak Ehteshami Bejnordi, and Amirhossein Habibian. "FrameExit: Conditional Early Exiting for Efficient Video Recognition." CVPR. 2021.
[3] Azad, Shehreen, Vibhav Vineet, and Yogesh S. Rawat. "StreamReady: Learning What to Answer and When in Long Streaming Videos." CVPR. 2026.