acceptodds
Under review as a conference paper at ICLR 2027

Verification, Not Correctness: Label-Free Distillation of Grounded Video Reasoning

Abstract

Video question answering (VideoQA) models often answer correctly while pointing to the wrong video evidence. Agentic reasoners close much of this grounding gap by verifying intermediate claims against observed events, at a steep inference cost. We introduce \method, which distills a verifier-equipped agentic teacher into a student that emits an answer, typed temporal evidence, and a self-verification score—calibrated to predict the verifier's acceptance of its output—in a single autoregressive pass. A controller trained by preference optimization on the verifier's signal alone raises the teacher's verified yield; a tilted verification filter selects and weights traces without answer labels and distills outputs and belief-state representations; and a self-verification head amortizes the verifier itself, enabling a cost-controlled escalation cascade. On NExT-GQA the 4B student reaches \accgqa, within of the teacher's , at its inference cost, and improves on correctness-filtered distillation by \accgqa without answer labels in controller training or trace selection; more than half of this gain comes from a higher localization rate among correct answers. The advantage replicates on a second teacher–verifier pair and on ReXTime and CG-Bench. A supporting analysis expresses the filtered teacher's correct-but-ungrounded mass through measurable verifier primitives and transfers the bound to the student; we report it as a population-level diagnostic with stated limitations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Question Answering, Temporal Grounding, Knowledge Distillation, Verifier-Guided Learning, Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jkwDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the high inference cost of agent-based "teacher" models and the limited "grounding" capability of single-pass "student" models in the context of video question answering. It proposes the VERA framework: a teacher model employs a verifier, equipped with typed post-conditions, to iteratively examine reasoning trajectories. These trajectories are then filtered and weighted based on the verifier’s assessments. This data is used to distill a student model capable of producing an answer, typed evidence, and a self-verification score in a single forward pass. During deployment, inputs with low self-verification scores are routed (cascaded) to the teacher model.

Critical strengths

The problem is clearly defined: the "verification-based reasoning" of computationally expensive agent-based teachers is transformed into training signals, which are then distilled into a single-pass student model, resulting in an end-to-end, deployable pipeline.

The design of typed evidence and a reusable verifier is insightful; the verifier can be used not only to filter teacher trajectories but also to re-evaluate student outputs, providing supervision for self-verification.

Critical weaknesses

Theoretical frameworks often lean toward abstraction in their primary benchmarks, while their core assumptions frequently incorporate unmeasurable variables, rendering them ill-suited to provide valid certification for real-world deployment.

Acceptance by a verifier does not equate to the correctness of an answer or the empirical grounding of evidence; the persistence of "accepted yet unverified" (or unanchored) content undermines the integrity of the core claims.

Systems relying on "self-verifying headers", where the primary objective is simply to secure acceptance by a verifier, may inadvertently inherit the verifier’s own errors, thereby amplifying systemic biases throughout routing processes and cascaded operations.

Questions for authors

Please refer to the weaknesses.

Reviewer_AxaVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces VERA, a framework for distilling a verifier-equipped video-reasoning agent into a single-pass student that produces answers, temporal evidence, and a verifier-acceptance score. Its main contribution is using evidence verification rather than answer correctness to select and weight training traces, complemented by controller optimization, belief-state alignment, and a self-verification head for selective escalation to the teacher. Experiments across three grounded VideoQA benchmarks and two teacher-verifier pairs show improvements over correctness-filtered distillation. On NExT-GQA, the 4B student achieves 41.0 Acc@GQA versus the teacher’s 42.3 at approximately 5% of its inference cost. Ablations, controlled evidence interventions, and routing analyses further examine the contributions of verification and amortization.

Critical strengths
  • The paper addresses the practically relevant challenge of transferring grounded video reasoning from expensive agents to efficient models, and this idea is quite reasonable.
  • Controlled comparisons between verification-based and correctness-based filtering help isolate the contribution of the training-data selection signal.
  • Evaluation across multiple benchmarks, student scales, and two teacher–verifier pairs supports the approach’s broader applicability.
  • Evidence interventions with randomized and question-swap controls provide useful validation beyond standard accuracy metrics.
  • The paper clearly distinguishes verifier acceptance from genuine groundedness and reports limitations of its theoretical bounds.
  • Deployment analysis includes routing baselines and offline cost accounting, providing a practical perspective on efficiency gains.
Critical weaknesses
  • This paper is actually hard to follow. The writing should be polished.

  • The claimed improvement should be distinguished from reducing correct-but-ungrounded predictions. Although VERA improves joint grounded accuracy and localization conditional on answer correctness, the absolute correct-but-unlocalized mass remains approximately 40% across the main in-pipeline systems. Thus, the results demonstrate more grounded correct answers, but do not establish a substantial reduction in the failure mode that motivates the paper.

  • The self-verification head remains an imperfect proxy for the desired deployment property. It is trained to predict verifier acceptance rather than genuine answer correctness and grounding, and Table 17 reports an AUROC of only 0.60 for identifying unlocalized outputs among correct answers. The routing results are useful, but the head’s ability to detect precisely the correct-but-ungrounded failures remains limited.

  • The theoretical analysis provides limited practical explanatory or predictive value. The paper transparently acknowledges that its principal soundness and cascade bounds are vacuous at the reported operating points, while the representation–calibration analysis relies on unobservable quantities and additional assumptions. Consequently, the theoretical results do not substantiate the measured grounding improvements or provide actionable deployment guarantees.

  • The broader applicability of the full pipeline is only partially established. The second teacher–verifier experiment replicates filtering and amortization but not controller learning, and the additional grounded benchmarks are trained on their respective splits. These results support replication within grounded VideoQA, but leave the transferability of the complete training procedure and the self-verification calibration under distribution shift less clear.

Questions for authors

Please refer to weaknesses.

Reviewer_sJvCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a label-free distillation framework designed to transfer the grounded reasoning capabilities of computationally expensive Multimodal Large Language Model (MLLM) agents into lightweight student models for complex VideoQA tasks. By employing Direct Preference Optimization (DPO) guided by an external visual verifier rather than gold-standard answers, alongside InfoNCE-based representation alignment, the method effectively mitigates textual shortcuts and enforces authentic visual grounding. In addition, the framework integrates a calibrated self-verification head that enables an amortized cascade inference strategy, reducing test-time computational costs while preserving the multi-hop reasoning abilities of the heavy teacher model.

Critical strengths

S1. VERA goes beyond answer-only distillation by using an external verifier to curate grounded reasoning traces and transferring typed temporal evidence and belief-state representations to the student. This enables improvements in grounded VideoQA performance without requiring gold answer or span labels in the main training objectives or trace selection process.

S2. The calibrated self-verification head enables a cost-controlled cascade inference strategy that escalates uncertain cases to the teacher. This approach achieves near-teacher grounded VideoQA performance at substantially lower inference cost, demonstrating an effective trade-off between grounded accuracy and computational efficiency.

Critical weaknesses

W1. The central assumption of VERA is that verifier-guided trace selection transfers grounded reasoning. However, verifier acceptance remains an imperfect proxy for actual temporal grounding. For example, a substantial fraction of accepted and correct teacher traces still fail the localization criterion on NExT-GQA. This raises concerns about whether the proposed verification mechanism reliably captures the property that the method aims to distill.

W2. While VERA reduces per-query inference cost, its offline trace collection and distillation costs lead to a high break-even query volume, particularly compared with the verify-then-escalate baseline. The practical advantage of the proposed amortization strategy may therefore be limited in deployments that do not process sufficiently large query volumes.

W3. Although VERA avoids gold answer and span labels in its training objectives, the main reported configuration still uses annotated validation data for hyperparameter selection. While the paper reports a verifier-based selection alternative with comparable performance on NExT-GQA, it remains unclear whether this label-free selection strategy generalizes equally well across different benchmarks and teacher–verifier settings.

W4. The reported results across multiple benchmarks are encouraging, but the generalization evidence remains limited. In particular, the second teacher experiment does not replicate the controller-learning stage, and some cross-benchmark results are based on single runs. Moreover, although the learned controller improves verified-trace coverage on NExT-GQA, it remains unclear how effectively VERA handles domains where the teacher generates few or no verifier-accepted traces. Further evaluation across independently varied teacher and verifier components, particularly in more challenging or low-yield domains, would better establish the robustness and generality of the proposed framework.

Questions for authors

None.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.