acceptodds
Withdrawn from ICLR 2027

EvoVideoRAG: Navigate, Verify, and Evolve for Token-Efficient Video Understanding via On-Policy Video Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Retrieval-Augmented Generation (RAG) grounds model generation in external evidence. With the development of multimodal models and agentic techniques, RAG has expanded from one-shot retrieval to continual evidence search, observation, and verification across heterogeneous information sources. Video is a particularly challenging modality: query-relevant information is temporally sparse and distributed across multiple scales, while each observation incurs substantial visual-token and computational costs. Existing video agents are commonly designed for individual tasks, making it difficult for a single policy to handle tasks with different input and output requirements, such as video corpus moment retrieval (VCMR) and video question answering (VideoQA). Meanwhile, successful and failed interaction trajectories are often discarded after a task, whereas continually accumulating external skills increases maintenance and invocation costs at inference time. To address these challenges, we propose EvoVideoRAG, a unified, task-adaptive, self-evolving, and cost-aware RAG framework for video-language understanding. Specifically, (1) EvoVideoRAG formulates heterogeneous tasks, including VCMR and VideoQA, as task-conditioned sequential evidence acquisition, enabling a shared agent to plan, retrieve, and verify video evidence and to adaptively decide whether to continue searching, adjust the temporal range, increase observation fidelity, or submit a prediction; (2) we introduce On-Policy Video Self-Distillation (OPVD), which turns new trajectories generated by the current policy into self-supervision for its successor, whose renewed interaction with the environment forms an intergenerational “interaction–distillation–reinteraction” self-evolution cycle without requiring an additional teacher or an ever-growing long-term skill library at inference time; and (3) we employ dynamic visual budgeting and cost constraints to allocate visual computation according to task difficulty and evidence sufficiency, reducing redundant frames and visual-token consumption while preserving task performance. Experiments on NExT-QA, Video-MME, Charades-FIG, and DiDeMo-FIG demonstrate the effectiveness of EvoVideoRAG in unifying video retrieval, temporal localization, and question answering through task-adaptive evidence acquisition within a unified agentic framework.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
retrieval-augmented generation, agentic video understanding, on-policy video self-distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RHnQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EvoVideoRAG, a unified agentic framework for video corpus moment retrieval and video question answering. It formulates video understanding as sequential evidence acquisition over a video/moment index. The policy can retrieve or switch candidates, expand or refine temporal intervals, increase observation fidelity, and submit the final result. The main training method is On-Policy Video Self-Distillation (OPVD). The current policy first generates trajectories, then selected states are paired with predefined legal alternative actions. Each action is evaluated by a continuation rollout using task reward minus visual-token cost. For QA, the reward uses the ground-truth answer. For localization, it uses the ground-truth video and temporal interval. If a better action is found, temporary guidance is constructed and used to obtain a teacher distribution from a frozen policy. The student policy is then trained by token-level forward KL without seeing the guidance. The method is evaluated on Charades-FIG, DiDeMo-FIG, NExT-QA, and Video-MME, and reports improvements over the SFT agent and GRPO baselines.

Critical strengths
  1. The paper studies an important problem. Adaptive evidence acquisition is useful for long-video understanding, where relevant evidence is sparse and uniform frame sampling can be inefficient.

  2. The reported results are strong. EvoVideoRAG improves over both the Agentic Base and GRPO on the evaluated localization and QA benchmarks.

  3. The paper considers visual computation cost in addition to task performance, and includes visual-token cost in the action utility.

Critical weaknesses
  1. The so-called “self-evolution” mainly introduces additional computation to explore multiple continuation rollouts, selects actions with higher task rewards, and then distills these preferred behaviors back into the policy. However, the supervision still strongly relies on ground-truth annotations: QA uses the ground-truth answer, while localization uses the ground-truth video and temporal interval to compute rollout rewards. From a high-level perspective, this is similar to reward-driven policy improvement as used in GRPO. The main difference is that OPVD adds an extra stage of rollout-based action selection followed by distillation.

  2. For localization, the ground-truth temporal window can already provide relatively direct action supervision. For example, based on the relation between the current interval and the GT interval, one can directly determine whether the model should expand the interval earlier, later, or keep the current range. Therefore, a direct GT-derived action supervision baseline is important. Without such a baseline, it is difficult to evaluate whether the expensive continuation rollouts and self-distillation are actually necessary.

  3. The questionable-state selection mechanism is not sufficiently validated. The model itself decides whether the current evidence is mismatched, incomplete, ambiguous, or already sufficient. However, the paper does not report the accuracy or stability of these judgments. It is also unclear what proportion of the selected questionable states actually contain a strictly better alternative action.

  4. The training cost is unclear and may be very high. For each selected state, the method evaluates multiple legal actions through continuation rollouts, and each rollout may involve multiple policy, retrieval, observation, and verification calls. However, the paper does not report key efficiency statistics such as total GPU hours, average number of candidate actions, average continuation length, total number of model calls, or total training-time visual tokens. These statistics are important for fairly evaluating the practical cost of the proposed method.

Questions for authors

see weakness

Reviewer_QVBjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EvoVideoRAG, an agentic RAG framework for video understanding that formulates video corpus moment retrieval (VCMR) and video question answering (VideoQA) as sequential evidence acquisition. They use an agent policy to plan retrieval and observation, verify whether the temporal evidence is sufficient and decide whether to continue searching, refine or submit the final prediction. Its main trainging component, OnPolicy Video Self-Distillation (OPVD), evaluates alternative actions using task reward and normalized visual-token cost, constructs temporary guidance from the verification result and preferred action, and distills the resulting self-teacher distribution back into the policy using forward KL with behavior preservation. The method is evaluated on NExT-QA, Video-MME for VideoQA and Charades-FIG, and DiDeMo-FIG for VCMR.

Critical strengths
  1. One of the contributions, OPVD, shows consistent gains over strong controlled baselines. The comparison against the SFT-initialized agentic base and GRPO is the strongest one in the ablation study. OPVD consistently improves both VCMR and VidoQA, sharing the same SFT initialization, trainging data, rewards, tool interfaces with GRPO.

  2. The paper provides a coherent framework for adaptive video evidence acquisition. Retrieval, temporal refinement, observation fidelity, evidence verification, and stopping and handled within a common sequential decision process.

Critical weaknesses
  1. The scope and novelty claimed in the paper appear overstated. The first contribution focuses on the heterogeneous-task unification and describes a “shared agent policy,” but the method and experiments instead refer to task-specific policy weights and task-specific Qwen3-VL-8B policies. Moreover, the current results show that the framework can accommodate VCMR and VideoQA, but do not demonstrate the benefit of a jointly learned cross-task policy. And prior works have already explored adaptive video observation, efficient temporal search, and multi-task video agents, so the distinctive contribution appears narrower than the paper’s framing suggests.
  2. The experiments validate full OPVD but do not isolate its novel mechanism or fully support the efficiency claim. OPVD simultaneously introduces alternative-action rollouts, reward and cost based action selection, temporary guidance, soft self-teacher distillation, and behavior preservation. The current ablation only compares the full method against SFT and GRPO, so it remains unclear which component is responsible for the improvement. In particular, since the preferred action as∗a_s^* is already identified before distillation, a direct hard-target/self-training baseline would be important to establish the value of soft self-distillation. There is also no ablation of the visual-cost term or dynamic versus fixed visual budgeting. Finally, efficiency is evaluated mainly through visual-token counts, while training overhead and inference latency are not reported.
Questions for authors
  1. What is the motivation for unifying VCMR and VideoQA? The two tasks already share a substantial evidence-acquisition pipeline, with retrieval naturally serving as an upstream component of video QA. The paper motivates unification through differences in search scope, granularity, termination conditions, and output formats, but it is unclear what additional benefit is obtained from treating them as a unified framework rather than related task-specific pipelines. Moreover, the paper refers to a “shared agent policy,” while the implementation appears to use task-specific policies. Could you clarify what is actually shared across tasks and provide evidence that the unified formulation itself is beneficial?

  2. Could you please strengthen the comparisons with established task-specific and agentic video baselines? The current evaluation includes several generic VLMs and a limited set of video agents, while some closely related work on adaptive and efficient video agents is not included. The VCMR experiments also omit standard baselines evaluated on Charades-FIG and DiDeMo-FIG in the original VERIFIED benchmark. Could you add representative comparisons under the same protocol, or explain why these baselines are not directly comparable?

  3. Why is soft self-distillation needed once the preferred action as∗a_s^* has already been identified? Could you compare OPVD against directly training on as∗a_s^* as a hard target? This would clarify whether the gain comes from alternative-action evaluation or from the self-distillation mechanism itself.

  4. What is the computational overhead of OPVD? Each selected state may require multiple continuation rollouts during training, while inference uses a multi-round agent loop. Reporting training cost, average agent calls, and latency would clarify the efficiency claim.

Reviewer_bnUPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoVideoRAG formulates video corpus moment retrieval and video question answering as sequential evidence acquisition, with a policy controlling retrieval, temporal refinement, observation fidelity, and termination. Its main training component, On-Policy Video Self-Distillation (OPVD), evaluates alternative actions using annotation-based quality–cost feedback, constructs temporary guidance for a frozen self-teacher, and distills the resulting action distributions while preserving successful, low-cost behaviors. Experiments cover four benchmarks.

The framework is practically motivated, but the current evidence does not sufficiently establish its methodological novelty, the fairness of the comparisons, or its claims of a unified and token-efficient policy.

Critical strengths
  1. Well-motivated evidence-acquisition formulation. Optimizing where to look, how closely to inspect, and when to stop is appropriate for video understanding. Distilling tool actions while keeping the QA answerer frozen helps separate improvements in evidence acquisition from changes to answer generation.

  2. Consistent improvements over the initial agent. OPVD improves over Agentic Base across all four datasets. It also outperforms the reported GRPO baseline, including gains of 2.61 and 4.89 percentage points in [email protected] on Charades-FIG and DiDeMo-FIG, respectively. These results provide encouraging evidence, although training-compute differences remain unresolved.

Critical weaknesses

1 The methodological novelty is insufficiently isolated, and the unified-policy claim is not well supported.** The method integrates adaptive video search, feedback-conditioned self-distillation and behavior preservation, with considerable overlap with prior work. Its putative unique design relies on video evidence gaps and action-level quality–cost trade-offs for distillation, yet ablations do not compare against closely related self-distillation baselines or disentangle temporary guidance, behavior preservation and cost regularization. As a result, performance gains may simply stem from extra action search and supervision. Furthermore, the paper advertises a shared unified policy across heterogeneous tasks, while the implementation adopts task-specific policies with intra-task weight sharing. The results only validate a reusable framework instead of a cross-task generalizable single policy, weakening a central contribution.

2 Benchmark comparisons lack proper controls, and efficiency claims against GRPO suffer from incomplete cost accounting.** Baselines use fixed uniform frame sampling while EvoVideoRAG leverages task-trained adaptive multi-round observation; matched backbone, training data and observation budget comparisons are absent. Video-MME is split into custom train/val/test partitions with a small test set, and baseline training access is unspecified. The marginal 69.26% vs 69.07% improvement over Gemini is negligible and not statistically robust. For efficiency, visual-token savings are inconsistent across benchmarks and ignore text generation, tool and indexing costs, and token counts are not comparable across model families. Training rollout budgets and GPU-hour costs versus GRPO are unreported, so the efficiency advantage under equal training compute remains unproven.

Questions for authors
  1. Policy sharing:Do all four benchmarks use the same trained policy and LoRA adapter, or separately trained models? If the latter, what evidence supports the claim of a single policy generalizing across heterogeneous tasks?

  2. Evaluation fairness: Were all Video-MME baselines evaluated on exactly the same 540 test questions, and did trainable baselines receive access to the same training partition? Do the gains persist with the same backbone, training data, and visual-token budget?

  3. Training-compute fairness:What are the total rollout counts and GPU hours for OPVD and GRPO, including all candidate-action continuations? Does OPVD retain its advantage when these costs are matched?

  4. Distinctive contribution: Does OPVD outperform directly training on the selected best actions and a comparable on-policy self-distillation baseline without video-specific guidance? What evidence isolates the benefits of guidance, preservation, and cost regularization?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.