- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses state underspecification in streaming video understanding, where standard answer supervision often does not force models to distinguish event order, counts, or changing relationships. To tackle this, it proposes Visible-State Supervision (ViSS), which fine-tunes Qwen3-VL-8B using source-grounded probes covering event identity, order, accumulation, relation changes, and evidence sufficiency. At test time, Test-Time Causal View Search (TVS) adaptively selects among recent, event, and history views without modifying the trained model. Experiments on StreamingBench and OVO-Bench include a carefully matched temporal-supervision baseline, three paired training seeds, fixed-checkpoint view comparisons, and replay diagnostics. Results show that supervision and evidence access provide complementary, task-dependent benefits. The combined system achieves 86.33% accuracy on StreamingBench and 69.77% macro accuracy on OVO-Bench, while processing fewer frames than an exhaustive three-view evaluation. Replay analysis further reveals that replacing previously correct answers is the main factor behind the remaining gap to the three-view oracle.
Critical strengths
-
The controlled comparisons are carefully designed. The generic temporal supervision baseline matches training instances, visual inputs, weights, ordering, optimization, and initialization. Access comparisons freeze the checkpoint. These controls help distinguish additional training from changes in evidence access.
-
The supervision improvement has statistical support. Across three paired seeds, the OVO-Bench improvement over generic supervision is consistently positive, averaging 1.72 percentage points. A source-video paired bootstrap yields a 95% confidence interval of [0.80, 2.60]. The complete system improves over the recent-four-frame baseline by 5.81 and 10.80 points on the two benchmarks.
-
The error analysis is concrete and appropriately scoped. The paper distinguishes proposal availability, acquisition, acceptance, and replacement using query-level counts. It explicitly conditions the dominance of replacement errors on coverage by the current three-view oracle, rather than labeling all-view failures as representation errors.
Critical weaknesses
-
Limited methodological novelty. Supervision for counting, ordering, and state changes, as well as on-demand access to historical evidence, has precedents. The additional methodological contributions of ViSS and TVS over related work are limited.
-
Insufficient detail on data construction and efficiency. Data-generation configurations and Generic–ViSS pairing rules remain underspecified. Efficiency comparisons mainly report model calls and processed frames, without measurements of actual latency or peak GPU memory.
-
No additional gain on StreamingBench. Compared with strictly matched generic temporal SFT, ViSS shows a mean change of −0.17 percentage points across three seeds, providing no evidence of an advantage on this benchmark.
-
Unclear necessity of the supervision mixture. Existing ablations do not sufficiently establish the value of the five supervision categories and their proportions. A controlled comparison with generic temporal QA plus insufficient-evidence supervision is missing, leaving the full mixture’s additional contribution to hallucination detection unclear.
Questions for authors
see above
4 comments
How much of the OVO supervision gain comes from hallucination detection
Dear Reviewer cDh5,
Your proposed Generic QA + Sufficiency comparison raises a related question that we can answer from the reported scores: how much of the overall ViSS advantage comes from hallucination detection (HLD) itself? Under OVO’s official scoring, HLD is one of three backward tasks, and backward tracing is one of three equally weighted groups. HLD therefore has weight 1/9 in the overall score.
| Comparison | Overall OVO change | HLD contribution to that change | All other tasks combined |
|---|---|---|---|
| ViSS vs. strictly matched Generic, mean over three paired seeds | +1.72 | +0.86 | +0.86 |
| Generic + Sufficiency vs. its Generic reference, supplemental seed 59 | +3.02 | +3.64 | −0.62 |
The first row uses the three-seed HLD improvement of 7.71 points: 7.71/9 ≈ 0.86. HLD thus accounts for about half of the matched ViSS gain; the other tasks together contribute the remaining half. In the supplemental comparison, the 32.80-point HLD improvement contributes 32.80/9 ≈ 3.64 points to the overall score, with the other tasks offsetting 0.62 points.
These are descriptive decompositions of the official score. Each row uses its own Generic reference, and the supplemental experiment uses a different data version. The result sharpens our claim in two ways: generic QA with sufficiency examples already supports strong HLD performance, and the strictly matched ViSS comparison shows a positive aggregate contribution from tasks beyond HLD. The incremental value of each supervision family remains open. We appreciate your suggestion, which helped us distinguish these conclusions.
Thank you for the responses and additional experiments. The analysis is valuable, but I still find the methodological novelty limited and the gains over strong baselines modest. I will keep my original rating unchanged.
Dear Reviewer cDh5,
Thank you for considering the additional experiments. Your suggestion to test Generic QA + Sufficiency helped us sharpen the paper’s conclusion: targeted sufficiency supervision already supports strong hallucination detection, while roughly half of the OVO gain in the strictly matched ViSS comparison comes from other tasks in aggregate.
We understand your reservations about methodological novelty and the size of the gains. We would ask you to weigh one further point in your overall recommendation: the study identifies a concrete failure that aggregate accuracy hides. Among the cases where one of the three views yielded a correct answer but TVS ultimately answered incorrectly, the policy had already retained the correct answer and then replaced it in 67% of cases on StreamingBench and 72% on OVO. This gives researchers a specific question to test when designing streaming systems: when additional history changes a prediction, how reliably does the model preserve an answer it already got right?
Would you be willing to reconsider your overall recommendation in light of that empirical contribution? We appreciate the care you have taken with the paper.
A kind reminder of the upcoming end of the discussion period
Dear Reviewer cDh5,
We would like to follow up on the supplementary experiments addressing your questions about supervision composition and hardware efficiency. Q4 reports three Generic+Sufficiency runs and explains their relationship to the original matched comparison. Q5 provides measured latency and peak GPU memory across the access policies. We also clarify the conclusions supported by these experiments and the remaining limits on mixture-level attribution.
We would appreciate your feedback on these additions and how they affect your assessment. Please let us know whether any further explanation would help resolve your remaining concerns.
Thank you for your careful review and consideration.
Best regards,
The Authors