- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes EvoVideoRAG, a unified agentic framework for video corpus moment retrieval and video question answering. It formulates video understanding as sequential evidence acquisition over a video/moment index. The policy can retrieve or switch candidates, expand or refine temporal intervals, increase observation fidelity, and submit the final result. The main training method is On-Policy Video Self-Distillation (OPVD). The current policy first generates trajectories, then selected states are paired with predefined legal alternative actions. Each action is evaluated by a continuation rollout using task reward minus visual-token cost. For QA, the reward uses the ground-truth answer. For localization, it uses the ground-truth video and temporal interval. If a better action is found, temporary guidance is constructed and used to obtain a teacher distribution from a frozen policy. The student policy is then trained by token-level forward KL without seeing the guidance. The method is evaluated on Charades-FIG, DiDeMo-FIG, NExT-QA, and Video-MME, and reports improvements over the SFT agent and GRPO baselines.
Critical strengths
-
The paper studies an important problem. Adaptive evidence acquisition is useful for long-video understanding, where relevant evidence is sparse and uniform frame sampling can be inefficient.
-
The reported results are strong. EvoVideoRAG improves over both the Agentic Base and GRPO on the evaluated localization and QA benchmarks.
-
The paper considers visual computation cost in addition to task performance, and includes visual-token cost in the action utility.
Critical weaknesses
-
The so-called “self-evolution” mainly introduces additional computation to explore multiple continuation rollouts, selects actions with higher task rewards, and then distills these preferred behaviors back into the policy. However, the supervision still strongly relies on ground-truth annotations: QA uses the ground-truth answer, while localization uses the ground-truth video and temporal interval to compute rollout rewards. From a high-level perspective, this is similar to reward-driven policy improvement as used in GRPO. The main difference is that OPVD adds an extra stage of rollout-based action selection followed by distillation.
-
For localization, the ground-truth temporal window can already provide relatively direct action supervision. For example, based on the relation between the current interval and the GT interval, one can directly determine whether the model should expand the interval earlier, later, or keep the current range. Therefore, a direct GT-derived action supervision baseline is important. Without such a baseline, it is difficult to evaluate whether the expensive continuation rollouts and self-distillation are actually necessary.
-
The questionable-state selection mechanism is not sufficiently validated. The model itself decides whether the current evidence is mismatched, incomplete, ambiguous, or already sufficient. However, the paper does not report the accuracy or stability of these judgments. It is also unclear what proportion of the selected questionable states actually contain a strictly better alternative action.
-
The training cost is unclear and may be very high. For each selected state, the method evaluates multiple legal actions through continuation rollouts, and each rollout may involve multiple policy, retrieval, observation, and verification calls. However, the paper does not report key efficiency statistics such as total GPU hours, average number of candidate actions, average continuation length, total number of model calls, or total training-time visual tokens. These statistics are important for fairly evaluating the practical cost of the proposed method.
Questions for authors
see weakness