acceptodds
Under review as a conference paper at ICLR 2027

MeSD: Multi-Evidence Self-Distillation for Video LLM

Abstract

While reinforcement learning with verifiable rewards provides reliable outcome supervision for VideoLLMs, sequence-level rewards offer limited token-level guidance. On-policy self-distillation addresses this limitation by conditioning a self-teacher on privileged information to provide dense token-level supervision. However, aggregating heterogeneous evidence within a single teacher context obscures cross-evidence agreement and conflict. A further challenge lies in determining whether teacher guidance should refine reward-based updates or provide corrective supervision for failed trajectories. To address these issues, we propose MeSD, a multi-evidence self-distillation framework for VideoLLMs. MeSD constructs three evidence-conditioned teachers with shared parameters, using the ground-truth answer as a common semantic context while separately incorporating temporal and spatial evidence. Given the same student-generated prefixes, MeSD evaluates evidence-specific preferences relative to the Answer Teacher and fuses teacher-common preferences with gated teacher-specific residuals. Furthermore, MeSD introduces Verification-Guided Optimization to classify trajectories as Success, Failure, or Indeterminate. For Success and Indeterminate trajectories, MeSD refines token-level advantage magnitudes while preserving reward-derived signs. For verified failure trajectories that contain the required evidence, MeSD applies failure-conditioned distillation, using reverse-KL correction toward the fused distribution. Experiments on multiple video benchmarks demonstrate consistent gains over reinforcement learning and self-distillation baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On policy distillation, VideoLLM, VLM

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_J6NHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MeSD, a multi-evidence self-distillation framework for VideoLLM post-training, addressing three problems: (i) sequence-level rewards provide limited token-level guidance; (ii) a single teacher context fails to capture cross-evidence agreement and conflict among heterogeneous evidence; and (iii) failed trajectories lack corrective supervision. To this end, the authors follow the on-policy self-distillation paradigm, using an EMA self-teacher conditioned on privileged annotation information. Existing dataset annotations (semantic answers, keyframe timestamps, and object bounding boxes) are organized into three evidence-conditioned teachers — Answer, Answer-Temporal, and Answer-Spatial. Using the Answer Teacher as a semantic anchor, the method measures the token-level preference shifts induced by temporal/spatial evidence, decomposes them into teacher-common preferences and teacher-specific residuals, and fuses them into a single target distribution via dual gating (an agreement gate and a magnitude gate). The motivation for evidence separation is presented clearly overall, and the framing of the pain point — that aggregating evidence in a single teacher context obscures evidence-specific contributions and conflicts — is well defined. The paper further proposes Verification-Guided Optimization: a task-specific verifier classifies student trajectories into Success, Failure, or Indeterminate; for successful and indeterminate trajectories, reward-derived signs are preserved during token-level advantage refinement; only for verified failures with the required evidence is the GRPO policy loss replaced with a reverse-KL objective, thereby providing corrective supervision even when group-relative advantages vanish. The paper presents a systematic experimental study across multiple benchmarks, including component ablations, comparisons of correction selection strategies, and multi-scale generalization experiments on Qwen3-VL 4B/8B/32B, concluding that MeSD consistently outperforms RLVR, RLSD, OPSD, VISD, and other baselines, while reporting a 65.8% reduction in training time relative to VISD.

Critical strengths
  • The ablations form a complete and coherent chain: w/o ES → w/o GPA → MeSD progressively isolates the independent contributions of "evidence separation" and "gated fusion"; w/o FCD and w/o VGO separately disentangle the value of the two optimization branches; and the correction selection strategies (Verified strict / All / Random) constitute a controlled comparison at a fixed correction-set size, directly testing the effectiveness of restricting correction to verified failures.

  • The paper formalizes the vague issue of "a single teacher context obscuring evidence contributions" into a well-defined decomposition of preference shifts (teacher-common preferences vs. teacher-specific residuals), and formalizes "limited corrective guidance for failed trajectories" as the vanishing-gradient problem of multiplicative refinement when group-relative advantages are zero, offering a principled account of how the reverse-KL objective bypasses this dead zone.

Critical weaknesses
  • The entire motivation rests on the claim that "sequence-level rewards offer limited token-level guidance," yet the paper lacks a direct comparison between sequence-level rewards alone (standard GRPO) and the addition of token-level supervision. If this premise does not hold, the entire complexity of the framework amounts to over-engineering. This is decisive for my overall assessment: even though the ablations are thorough, they only demonstrate that "individual components contribute within this design," not that "this design is the right abstraction for the problem."

  • MeSD involves three teacher distributions per position (roughly 3x teacher forward passes, with an implicit ensemble effect), whereas w/o ES performs only a single teacher forward pass. The two are not matched in terms of compute or distributional diversity. The paper provides no compute-matched baseline, does not test the key variant of "three contexts each containing all evidence" (i.e., separating the contexts without splitting the evidence), and does not evaluate stronger single-context prompting for conflict resolution (e.g., prompting the teacher to explicitly compare evidence consistency before producing its distribution).

Questions for authors
  • Please provide (or point to) a comparison under identical settings between standard GRPO (without any token-level refinement/distillation) and the full token-level supervision. If the gap is small, the necessity of the entire evidence-separation-and-fusion structure would be fundamentally challenged.

  • Please report an experiment with a multi-teacher variant where all three contexts contain all evidence and the resulting distributions are fused, serving as a compute-matched baseline for MeSD. Additionally, please clarify how the evidence fields were organized in the single-context baseline (ordering, label formatting), and whether a stronger single-context prompt asking the teacher to explicitly compare cross-evidence consistency before producing its distribution was attempted. This directly determines whether the core novelty claim stands.

  • Is there ablation support for routing Indeterminate trajectories to the credit refinement branch rather than the failure correction branch?

  • Are the numbers in the main tables and ablation tables from single runs? Please report the mean ± standard deviation over multiple random seeds for the key comparisons (MeSD vs. VISD, MeSD vs. w/o ES, Verified vs. All). In particular, are the small margins such as VideoMMMU Overall (56.1 vs. 55.8) within the noise range?

Suggestion for future work (does not affect the rating): Compare learnable evidence fusion mechanisms (e.g., learned gating or attention-based weighting) against the current handcrafted geometric/arithmetic decomposition with dual gating, and examine at larger model scales whether the marginal benefit of evidence separation diminishes as unified understanding capabilities strengthen. This would address the open question of whether the handcrafted structure is merely transitional scaffolding, but it is beyond the scope of the present paper.

Reviewer_6AuXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MeSD (Multi-Evidence Self-Distillation) to address a critical challenge in Reinforcement Learning with Verifiable Rewards (RLVR) for video reasoning tasks: the fact that sequence-level rewards for final answers often fail to provide sufficient token-level supervision for the model’s intermediate reasoning process. The authors argue that existing on-policy self-distillation methods struggle when handling heterogeneous evidence—such as temporal and spatial information—often conflating the roles of different evidence types or failing to resolve potential conflicts between them. Furthermore, simply adjusting the magnitude of token-level advantages for failed trajectories may not provide an adequate signal for error correction. To this end, MeSD incorporates two core modules:

  1. Multi-Evidence Self-Teacher: This module employs shared EMA (Exponential Moving Average) parameters to construct three conditional "teachers": an Answer Teacher, an Answer-Temporal Teacher, and an Answer-Spatial Teacher. Using the Answer Teacher as a semantic anchor, the framework compares token preferences before and after the inclusion of temporal or spatial evidence. It then integrates these teacher distributions through a combination of consensus preferences, teacher-specific residuals, and a gating mechanism.
  2. Verification-Guided Optimization (VGO): Based on a task-specific verifier, trajectories are categorized as "Success," "Failure," or "Indeterminate." For successful and indeterminate trajectories, the framework retains the sign of the reward-derived advantages while using teacher signals to refine the magnitude of these token-level advantages. For verified failure trajectories that nonetheless contain the necessary evidence, the framework replaces the standard GRPO policy loss with a reverse-KL distillation objective, aimed at aligning the model with the integrated teacher distribution. Experiments, initialized with the Open-o3-Video-SFT-7B model, evaluate MeSD across various benchmarks for video understanding, spatiotemporal reasoning, and temporal localization. The paper reports performance gains over baselines such as GRPO, RLSD, and VISD, with ablation studies analyzing the contributions of evidence separation, gated preference fusion, and distillation for failure cases. In summary, the paper proposes an approach that combines multi-evidence preference modeling with optimization objectives selected based on trajectory verification outcomes, ultimately improving both the answer accuracy and the spatiotemporal localization capabilities of VideoLLMs.
Critical strengths
  1. Well-motivated design for supervised learning in video spatiotemporal reasoning Rather than relying solely on the final answer reward for optimization, the paper models temporal and spatial evidence separately. It employs an "Answer Teacher" as a shared semantic anchor to characterize how different types of evidence influence token preferences. This design effectively addresses common challenges in video reasoning—such as arriving at a correct answer based on inaccurate evidence, or resolving potential conflicts between temporal and spatial evidence—making the motivation behind the approach quite clear.
  2. Comprehensive experimental evaluation with targeted component ablation The paper evaluates the method across several benchmarks, including V-STAR, WorldSense, VideoMMMU, LRR, TVGBench, and Charades-STA. Key reported results include:
  • On V-STAR, MeSD achieves an answer accuracy of 63.7%, outperforming VISD’s 61.9%.
  • Across a composite of multiple benchmarks (WorldSense, VideoMMMU, LRR, and TVGBench), MeSD achieves an average score of 50.3%, compared to 48.0% for VISD.
  • On TVGBench, MeSD attains an mIoU of 29.8%, surpassing GRPO’s 22.5%.
  • Ablation studies examine individual components—such as evidence separation, gated preference aggregation, failure-condition distillation, and verification-guided optimization—providing insight into the specific contribution of each module. These results offer solid preliminary evidence for the method's effectiveness. However, as some performance gains are marginal, further assessment—considering experimental variance and consistent training budgets—is necessary to determine the stability of these improvements.
Critical weaknesses
  1. The boundaries of innovation for the core algorithm remain somewhat ambiguous; the rationale for the gated fusion design requires further justification The core mechanisms of MeSD—including multi-condition teachers with shared parameters, preference residuals relative to an "Answer Teacher," the decomposition of consensus and residuals, and gated fusion based on directional consistency and relative magnitude—are logically sound. Nevertheless, the approach as a whole appears more akin to a combination and adaptation of existing techniques, such as self-distillation, teacher ensembling, and reward-guided optimization. While current ablation studies indicate that removing evidence separation or gated preference aggregation leads to a performance decline, the effectiveness of a module does not necessarily imply that its specific mathematical design offers a unique advantage. For instance, the paper needs to provide a more compelling justification for why the current gating formula is superior to simpler alternatives—such as geometric mean, arithmetic mean, fixed-weight fusion, or other common strategies for aggregating teacher distributions. Furthermore, the paper reports that MeSD does not outperform VISD on certain temporal localization metrics in V-STAR: it achieves a temporal IoU of 25.8% for "Chain1" (compared to 26.8% for VISD) and 25.0% for "Chain2" (compared to 27.2% for VISD). Consequently, the paper’s claims regarding overall performance gains should more clearly distinguish between answer accuracy, temporal localization, and spatial localization, so as not to mislead readers into assuming that improvements apply to all metrics.
  2. The effectiveness of "failure-condition distillation" requires more in-depth mechanistic validation. MeSD employs reverse-KL distillation on verified failure trajectories, using teacher distributions to replace the corresponding GRPO strategy loss. While this is a significant contribution, it raises a critical question: does the teacher distribution genuinely provide the correct direction for error correction, or does it merely alter the optimization objective by leveraging additional annotation information? The paper provides preliminary support through ablation studies—specifically by removing failure-condition distillation or by distilling from randomly selected trajectories—showing that the full method outperforms these variants on benchmarks like V-STAR and TVGBench. However, several points need further attention:
  • The verifier’s decision rules are task-dependent (e.g., using specific tIoU thresholds for temporal localization). How reliable are the classifications of "Success," "Failure," and "Indeterminate" for edge cases or tasks not well-covered by these verification rules?
  • Does the effectiveness of error correction for failure trajectories depend on the quality of the correct answers, temporal annotations, and spatial annotations available to the teacher? Does the method remain effective when these annotations are incomplete or noisy?
  • The target distribution for the reverse-KL objective is constructed over a limited set of candidate tokens, with the probabilities of all other tokens aggregated into a single category. To what extent does this approximation affect the efficacy of error correction and the quality of the distribution matching? These issues do not invalidate the existing experimental results, but they do affect the assessment of the method’s generalization capability and the reliability of its core mechanisms.
Questions for authors

How can it be demonstrated that the performance gains of MeSD stem specifically from its unique multi-evidence fusion and failure-correction mechanisms, rather than from additional privileged supervision or changes in optimization strategy? The paper’s core methodology introduces three interrelated components: multi-evidence teachers, gated preference fusion, and reverse-KL distillation applied to verified failure trajectories. Although the existing ablation studies remove individual modules, there are significant interactions between these components. The authors are requested to clarify the following:

  1. Under identical conditions—including initialization, training data, training steps, and computational budget—have the authors compared MeSD against alternative configurations, such as simple averaging of teacher distributions, fusion without gating, using only multi-evidence token-level advantage adjustments, or applying reverse-KL distillation exclusively to failure trajectories?
  2. Can the gains derived from distilling failure trajectories be attributed specifically to the verifier-based selection strategy—while keeping the teacher information constant—rather than to the reverse-KL objective itself?
  3. Could the authors provide the mean and standard deviation from multiple independent runs—particularly for benchmarks where the performance gap between VISD and MeSD is narrow—to confirm the statistical stability of the observed improvements?
Reviewer_mTtUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MeSD trains a video language model using shared-parameter self-teachers conditioned on the ground-truth answer and, when available, temporal or spatial annotations. The teachers score identical student-generated prefixes. Gated fusion combines their token distributions, and a verifier routes eligible failed trajectories to reverse-KL distillation; other trajectories receive reward-directed token advantage refinement. The paper evaluates this combination on video question answering and grounding benchmarks, reporting gains over its training baselines and ablations of evidence separation, fusion, and failure correction.

Critical strengths
  1. The practical gains are credible and reported with useful granularity. Table 1 separates answer, temporal, and spatial performance: against VISD, MeSD improves V-STAR answer accuracy from 61.9 to 63.7 and mLGM from 48.9 to 50.2, while its temporal IoU scores are lower, 25.8/25.0 versus 26.8/27.2 (Lin et al. 2026). Table 2 shows a larger TVGBench mIoU improvement, 24.8 to 29.8. This supports a useful video-training recipe while making its grounding limitations visible. Fig. 12's correct answer with incorrect temporal localization reinforces the motivation for evaluating evidence separately.

  2. In Table 4, removing failure-conditioned distillation lowers V-STAR mLGM from 50.2 to 41.0 and TVGBench mIoU from 29.8 to 23.0; evidence merging and simple averaging also reduce performance. Table 11 additionally matches reverse-KL selection counts by task and batch: random selection reaches 45.6/26.4, below strict failure selection at 50.2/29.8. These controls make the benefit more persuasive than leaderboard comparisons alone. The appendix's explicit verifier and candidate-distribution details also aid scrutiny. These results keep my recommendation close to the acceptance boundary.

Critical weaknesses
  1. The demonstrated training coverage does not yet substantiate the claimed interaction between evidence-conflict fusion and failure correction. Only 12,047 of the 37,231 samples have both temporal and spatial annotations (Table 5). With one auxiliary teacher, the arithmetic-geometric residual is zero, so the conflict gate has no disagreement to resolve. Table 7 excludes the joint spatiotemporal and spatial tasks from strict reverse-KL correction, while allowing particular multiple-choice and temporal failures. Under these stated rules, no trajectory appears to exercise both temporal-spatial conflict gating and failure correction. This is a conditional inference from the stated eligibility rules, not a measured branch count. Actual task-level coverage is needed to establish what the combined mechanism contributes. The multiple-choice rule is especially unclear: wrong options are eligible in Table 7, but Table 5 lists no temporal or spatial annotations for those samples. Fig. 2 also omits the return of ineligible failures to the reward-based branch, obscuring a central qualification.

  2. The ablations support the complete recipe more clearly than its proposed mechanism-specific increment. In Equations (2)–(5), with both auxiliary teachers present, the ungated term satisfies q_A exp(G) = sqrt(q_T q_S) before normalization; the answer teacher therefore affects the gates without adding an independent geometric vote. The averaging ablation does not isolate the answer anchor's contribution. AVSD's geometric consensus and gated arithmetic residual, and H²SD's success/failure routing, provide useful interpretive context (Nguyen et al. 2026; Cai et al. 2026). The decision-relevant gap is the paper's own attribution: the current controls do not separate anchoring, gating, eligibility, and correction-target quality. All teachers continue the failed student's prefix, so agreement alone does not establish a better next-token target. Prefix-continuation failures have been measured in text reasoning, but their relevance here requires video evidence (Zhu et al. 2026). The matched-count routing result is encouraging; teacher quality on eligible failures and more isolated controls could resolve these fixable uncertainties. Together with the coverage issue, they currently outweigh the gains and support a weak rejection.

References

Cai, Qiye, et al. 2026. “H²SD: Hybrid Hindsight Self-Distillation.” arXiv:2607.18955. https://arxiv.org/abs/2607.18955.

Lin, Hao, et al. 2026. “VISD: Enhancing Video Reasoning via Structured Self-Distillation.” arXiv:2605.06094. https://arxiv.org/abs/2605.06094.

Nguyen, Duy, et al. 2026. “AVSD: Adaptive-View Self-Distillation by Balancing Consensus and Teacher-Specific Privileged Signals.” arXiv:2605.20643. https://arxiv.org/abs/2605.20643.

Zhu, Siqi, et al. 2026. “The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes.” arXiv:2605.11182. https://arxiv.org/abs/2605.11182.

Questions for authors
  1. What fractions of each task have both auxiliary teacher views, nonzero teacher disagreement, and actual reverse-KL selection? How do multiple-choice samples satisfy the teacher-evidence availability requirement, and which failures return to reward-based optimization?

  2. Do controlled results with a fixed fused teacher and matched correction count isolate the contributions of the answer anchor, conflict gates, and stricter eligibility rules? Such evidence could establish the mechanism-specific increment currently left uncertain by Table 4.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response

We sincerely thank all three reviewers for their careful reading and constructive feedback. We appreciate their recognition of the motivation for separating temporal and spatial evidence to understand how each shapes token preferences (@J6NH, @6AuX), the gains on video question answering and grounding benchmarks (@mTtU, @6AuX), and the ablations examining evidence separation, fusion, and failure correction (@mTtU, @J6NH, @6AuX). Their feedback helped us clarify how fusion and correction interact during training (@mTtU), distinguish the benefit of evidence separation from the effects of prompting and teacher computation (@J6NH), and assess the choice of fusion rule and the quality of supervision on failed trajectories (@mTtU, @6AuX).

At the core of our work is the use of multiple sources of video evidence to guide token-level learning in VideoLLMs. A sequence-level reward offers little guidance on which token predictions should change, while combining all annotations in a single teacher context obscures where temporal and spatial evidence agree or conflict. MeSD uses the Answer Teacher as a common reference to measure how each source of evidence changes predictions on the same student-generated prefix, then fuses shared preferences with a gated contribution from evidence-specific preferences. Guided by verification, MeSD applies direct distribution-level correction to eligible failures while preserving the direction of reward-based updates on other valid trajectories. MeSD thus turns existing answer and grounding annotations into targeted supervision, improving performance across video understanding and spatiotemporal reasoning benchmarks.

We have responded to each reviewer’s comments and incorporated the key control experiments into the revised manuscript. Following the discussion, @J6NH and @6AuX both indicated that their main concerns had been addressed and raised their ratings from 2 to 3 (Weak Accept). The revisions cover four main areas.

  • Evidence separation and training cost. Table 6 compares MeSD with three controls using the same available evidence: a single all-evidence context, the same context with a stronger consistency instruction, and three contexts with different evidence orders. All configurations share the same initialization, training data, annotations, and training steps. We report teacher scoring passes and full training time alongside performance.

  • Answer reference and adaptive fusion. Section 3.2 clarifies how the Answer Teacher provides a common reference for samples with different combinations of annotations. Table 5 tests the reference and the gate separately. One control replaces the answer reference with the student distribution. The others compare adaptive gating with geometric, arithmetic, and fixed-weight fusion while keeping teacher evidence and trajectory selection fixed.

  • Failure selection and correction. We clarify that reverse-KL correction applies only to verified failures that meet the strict eligibility criteria and have the required teacher evidence. Other valid trajectories retain reward-based credit refinement. With correction counts matched by task and batch, strict selection improves V-STAR mLGM by 4.6 points and TVGBench mIoU by 3.4 points over random selection. Table 15 separates the contributions of failure correction and credit refinement. Appendix F.5 defines how to measure, for each task, the share of valid trajectories that have both auxiliary evidence types, exhibit teacher disagreement, and receive correction.

  • Teacher-target quality and stability across runs. Table 14 compares the probabilities that teachers and students assign to ground-truth answer tokens on identical failed prefixes. We also examine sensitivity to perturbations of temporal annotations and report three-seed results for selected comparisons in Table 16. We appreciate @J6NH’s suggestion to extend the multi-seed analysis to Verified versus All.

We have revised the main text and appendix in response to the reviewers’ feedback. Tables 5 and 6 now include the key control experiments, and the appendix provides supporting analyses. We have highlighted the revisions in blue. We sincerely thank the reviewers for helping us improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.