acceptodds
Under review as a conference paper at ICLR 2027

VEPO: Learning to Credit Visual Evidence Seeking for Agentic Video Understanding

Abstract

Long-video understanding requires acquiring sparse, temporally distributed visual evidence from long and redundant videos. Agentic approaches address this challenge through iterative temporal search, allowing models to gather visual observations before producing an answer. We find that correct answers produced by agentic systems are not always supported by the acquired evidence. During training with standard GRPO, answer accuracy improves while temporal evidence recall and active search decline, a training dynamic we characterize as evidence-seeking collapse. The objective rewards final-answer correctness, assigning identical rewards to evidence-supported predictions and unsupported guesses without checking whether sufficient evidence was acquired. Recent methods incorporate tool-use incentives and temporal localization rewards to encourage search activity and access to relevant regions, but these signals do not directly assess whether the accumulated observations provide sufficient evidence to answer the question. We introduce Video Evidence Policy Optimization (VEPO), which uses a frozen Evidence Reward Model to evaluate the sufficiency of accumulated observations. The resulting evidence signal allows VEPO to prefer evidence-supported answers over unsupported guesses and to credit the search decisions that improve evidence sufficiency. Across four long-video benchmarks, VEPO consistently outperforms supervised fine-tuning and standard reinforcement learning, improving both answer accuracy and grounded evidence acquisition.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video understanding, multimodal agents, reinforcement learning, long video reasoning, tool use, agentic learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_spJnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates evidence-seeking collapse in agentic long-video understanding, where outcome-only RL may improve answer accuracy while reducing active evidence acquisition. To address this issue, this paper proposes VEPO, which uses a frozen Evidence Reward Model (ERM) to assess the sufficiency of accumulated search-retrieved visual observations. The resulting signal is used for trajectory-level rewards and action-local credit assignment, together with dynamic group filtering. Experiments across four long-video benchmarks demonstrate improvements in answer accuracy and evidence-supported correctness.

Critical strengths
  1. This paper investigates a meaningful problem in agentic video understanding: improving answer accuracy does not necessarily imply effective evidence acquisition. The analysis of training dynamics provides useful motivation for optimizing evidence sufficiency rather than merely rewarding final-answer correctness or tool usage.

  2. The proposed method is well motivated. The ERM evaluates accumulated visual evidence, while trajectory-level rewards and action-local credit assignment encourage the acquisition of useful evidence. This provides a reasonable approach to improving evidence-seeking behavior during RL training.

  3. The experiments are relatively comprehensive and demonstrate meaningful improvements over SFT and standard GRPO across four benchmarks. The paper also includes component ablations, cross-backbone evaluation, inference efficiency analysis, and independent evidence assessments using another model and human annotators, providing substantial empirical support for the proposed approach.

Critical weaknesses

The methodological advantages over existing evidence-aware approaches are not sufficiently established. Prior methods such as TimeSearch-R[1] have already explored rewarding the completeness of retrieved video evidence. While VEPO introduces a separately trained ERM and action-local credit assignment, the main controlled comparisons focus on outcome-only GRPO and generic tool-use rewards. The component ablations demonstrate their usefulness, but it remains unclear how much additional benefit VEPO provides over existing evidence-aware reward designs under comparable training conditions.

[1] TimeSearch-R: Adaptive Temporal Search for Long-Form Video Understanding via Self-Verification Reinforcement Learning, ICLR 2026

Questions for authors
  1. How would the evidence-support evaluation change if the initial overview frames were included? In particular, what proportion of correct predictions classified as unsupported could already be justified by the overview? Would the evidence-seeking collapse analysis and the improvements in supported correctness remain consistent?

  2. The main experiments are conducted on the relatively outdated Qwen2.5-VL-7B backbone. Although the appendix includes results on Qwen3-VL-4B, could the authors further validate VEPO on a more recent and stronger model, such as Qwen3.5-9B, to assess whether its improvements over standard GRPO still hold?

5 comments
AuthorsOpenReview ↗

Author Response (W1), Part 1: Comparison with Evidence-Aware Rewards

We thank Reviewer spJn for the positive assessment and for finding the problem meaningful and the method well motivated.

W1: Controlled Comparison against Existing Evidence-Aware Reward Designs

TimeSearch-R:

  • Motivation and core distinction. The core difference is how two capabilities receive credit: acquiring semantically sufficient visual evidence and using that evidence to answer correctly. TimeSearch-R asks the policy to re-answer the question from the retrieved frames and rewards the search when the re-answer is correct. Search credit therefore still comes from answer correctness, so incomplete evidence can receive credit when the model guesses correctly. VEPO evaluates the semantic sufficiency of the acquired observations with the ERM and scores final-answer correctness separately. Replacing TimeSearch-R's self-verifier with an external model or self-consistency would not remove this coupling because the search would still be judged through another answer.

  • Native evidence acquisition. TimeSearch-R uses a text-conditioned SigLIP retriever built on a precomputed video-wide frame cache. The policy can submit a broad temporal range and leave query-conditioned semantic frame selection to the external retriever. Its CSV reward evaluates whether the returned frames support a correct re-answer, without directly crediting the policy for locating useful visual evidence. VEPO uses no external semantic frame selector. Its tool returns uniformly sampled frames from the temporal interval selected by the policy, so the policy must learn where to search, with semantic evidence credit directly supervising this capability.

  • Action-local evidence credit. TimeSearch-R produces a single CSV reward after the complete search trajectory, without identifying which search supplied the missing information. VEPO evaluates the accumulated evidence after each search and assigns additional credit when an action increases semantic evidence sufficiency. It therefore teaches the policy which searches fill the current evidence gap, while repeated or unhelpful searches receive no additional evidence credit.

Experimental validation:

  • Training behavior. In our controlled TimeSearch-R-style reproduction (Tab. R1a), from the first to the last 25 RL training steps, answer accuracy increases (64.6% → 70.4%), while temporal evidence recall (38.6% → 32.4%) and active search (3.60→2.343.60\rightarrow2.34) both decline, and supported correctness remains nearly unchanged (40.4% → 41.1%). Rewarding search through re-answer correctness still leads to evidence-seeking collapse.

Tab. R1: Controlled TimeSearch-R-Style Reward Analysis

(a) Measured Training Dynamics

RL StepsAcc.C=Gs=1C=G_s=1E[Gt]\mathbb{E}[G_t]Avg. Calls
0–2464.640.438.63.60
25–4968.437.228.92.67
50–7471.939.925.92.32
75–9971.638.227.72.22
100–12473.141.832.42.30
125–14971.441.930.92.83
150–17472.941.539.42.42
175–19970.441.132.42.34
AuthorsOpenReview ↗

Author Response (W1), Part 2: Comparison with Evidence-Aware Rewards

W1 (continued): Controlled Comparison against Existing Evidence-Aware Reward Designs

  • Controlled reward comparison. To strictly isolate the reward design, we hold the Qwen2.5-VL-7B SFT model, RL data, rollout budget, and native tool interface fixed, and replace VEPO's reward with the TimeSearch-R-style re-answer reward (Tab. R1b, c). The two settings differ only in whether search is supervised through re-answer correctness or through VEPO's separate semantic-evidence and answer signals.

(b) Final Accuracy under the Same Training Setup

RewardMLVUVideoMMELongVideoBenchLVBench
TimeSearch-R-style67.362.761.345.8
VEPO68.064.463.047.5

(c) LVBench Evidence Acquisition under the Same Training Setup

RewardAcc.C=Gs=1C=G_s=1E[Gt]\mathbb{E}[G_t]Avg. Calls
TimeSearch-R-style45.821.931.92.24
VEPO47.536.339.13.18
  • Tool-use and retrieval controls. On LVBench, 65.5% of TimeSearch-R's calls request the entire video, confirming that semantic frame selection is largely performed by the external SigLIP retriever rather than the policy's temporal search. We therefore restore the original SigLIP retriever and separately cap VEPO at one or two search calls (Tab. R1d).

(d) LVBench under Controlled Search and Retrieval Settings

MethodRetrieverAvg. CallsAcc.C=Gs=1C=G_s=1
TimeSearch-R (w/o SigLIP)None1.5638.417.3
TimeSearch-R + SigLIPSigLIP1.4141.423.8
VEPO (1 Call)None1.0045.324.2
VEPO (2 Calls)None2.0046.631.8
VEPONone3.1847.536.3
VEPO + SigLIPSigLIP3.2447.937.1

With the policy, data, tools, and interaction budget fixed, VEPO achieves higher accuracy across all four benchmarks and boosts evidence-supported correctness on LVBench from 21.9% to 36.3%, while preserving stronger temporal recall and active search. Moreover, TimeSearch-R remains below VEPO even with its SigLIP retriever restored and with VEPO restricted to a single native search (45.3% vs. 41.4%), while adding SigLIP to VEPO yields only a marginal 0.4-point gain.

Takeaway: TimeSearch-R derives search credit from re-answer correctness and delegates semantic frame selection to an external retriever. VEPO learns semantic visual evidence acquisition within the policy, credits the individual searches that improve it, and achieves stronger evidence-supported performance under controlled training and retrieval settings.

AuthorsOpenReview ↗

Author Response (Q1): Initial Overview in Evidence Evaluation

Q1: Including the Initial Overview in Evidence Evaluation

Across long-video benchmarks, the 64-frame overview covers only about 1% of candidate frames on LVBench and 2% on CG-Bench, providing coarse global context without replacing active search for localized evidence. Including both the overview and retrieved observations in the evidence evaluation under both our ERM and the independent Gemini 2.5 Flash judge (Tab. R2a) raises supported correctness across all methods while preserving VEPO's clear lead (36.3% → 40.2% under ERM and 31.8% → 35.5% under Gemini).

Tab. R2: LVBench Evidence Support with the Initial Overview

(a) C=Gs=1C=G_s=1 by Evidence Input

MethodAcc.Retrieved (ERM)Overview + Retrieved (ERM)Retrieved (Gemini)Overview + Retrieved (Gemini)
Conan39.70.017.40.015.6
LongVT29.411.816.310.514.6
VideoTemp-o336.813.620.811.818.4
Video-MTR40.414.122.012.819.8
TimeSearch-R + SigLIP41.423.828.520.625.1
SFT43.821.530.218.926.8
Outcome-only GRPO42.319.227.916.824.7
VEPO47.536.340.231.835.5

To directly quantify the proportion requested by the reviewer, we examine correct predictions judged unsupported under Retrieved and evaluate how often the initial overview alone contains sufficient evidence under both our ERM and Gemini 2.5 Flash (Tab. R2b).

(b) Overview Support among Retrieved-Unsupported Correct Predictions

MethodSupported by Overview Alone (ERM, %)Supported by Overview Alone (Gemini, %)
VEPO21.319.1
TimeSearch-R + SigLIP22.219.7
Outcome-only GRPO24.120.8
LongVT24.421.2
SFT27.524.1
TimeSearch-R (w/o SigLIP)28.024.5
Video-MTR28.925.0
VideoTemp-o329.325.6
Conan43.739.3

Among VEPO's retrieved-unsupported correct predictions, 21.3% (ERM) and 19.1% (Gemini) are supported by the overview alone, the lowest proportions across all compared methods. This confirms that excluding the overview in the main diagnostic does not preferentially favor VEPO.

We then use GPT-5 mini to separate questions whose evidence is already visible in the overview (Overview-Sufficient, 27.7%) from those requiring additional search (Search-Required, 72.3%). In Tab. R3, existing agentic baselines rely mainly on the Overview-Sufficient subset (48.5%–65.5%) and drop sharply on the Search-Required subset (0.0%–17.4%; outcome-only GRPO likewise degrades relative to SFT on both subsets, 64.3% vs. 66.0% and 14.0% vs. 16.5%), whereas VEPO achieves the highest C=Gs=1C=G_s=1 throughout (67.8% and 29.6%).

Tab. R3: LVBench C=Gs=1C=G_s=1 by Overview Sufficiency

MethodAcc.Overview-Sufficient (27.7%)Search-Required (72.3%)Overall
Conan39.762.70.017.4
LongVT29.448.54.016.3
VideoTemp-o336.848.510.220.8
Video-MTR40.465.55.422.0
TimeSearch-R + SigLIP41.457.617.428.5
SFT43.866.016.530.2
Outcome-only GRPO42.364.314.027.9
VEPO47.567.829.640.2

Takeaway: Most long-video queries require visual evidence beyond the initial overview, and VEPO's retrieved-unsupported correct answers rely least on the overview across all compared methods. VEPO remains strongest under both evaluation protocols and both judges because it actively acquires the visual evidence missing from the overview.

Reviewer_eyCLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a failure mode in agentic long-video understanding, training with final-answer rewards can improve answer accuracy while reducing temporal search and evidence acquisition. Motivated by this phenomenon, the authors propose Video Evidence Policy Optimization (VEPO), which uses an ERM to assess whether the observations collected across search steps jointly provide sufficient information to answer a question. Experiments on four long-video benchmarks show improvements over supervised fine-tuning and standard GRPO.

Critical strengths
  1. The paper highlights a useful distinction between answering correctly and collecting enough evidence to support the answer.
  2. The method addresses both which trajectories to reward and which search actions contributed useful evidence.
  3. Results are consistent across four benchmarks, and the additional model and human evaluations strengthen the evidence for the reported gains.
Critical weaknesses
  1. Declining search activity may reflect better use of the initial overview rather than evidence-seeking collapse. Since evidence judges exclude overview frames, the current evaluation cannot distinguish efficient answering from unsupported guessing.
  2. The main evaluation of supported correctness uses the same ERM that provides training rewards, raising concerns that the policy may exploit its biases.
Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Author Response (W1): Overview Reliance and Evidence-Seeking Collapse

We thank Reviewer eyCL for the positive review and for highlighting the distinction between answering correctly and collecting sufficient evidence.

W1: Overview Reliance and Evidence-Seeking Collapse

Including the overview in evidence evaluation. The 64-frame overview covers only about 1% of candidate frames on LVBench and 2% on CG-Bench, providing coarse global context while leaving the localized visual details required for answering unobserved. To distinguish efficient answering from unsupported guessing, we evaluate C=Gs=1C=G_s=1 over the retrieved observations alone and over the overview together with the retrieved observations under both our ERM and the independent Gemini 2.5 Flash judge (Tab. R1). Including the overview raises supported correctness across all methods while preserving VEPO's clear lead under both evidence inputs and both judges (36.3% → 40.2% under ERM and 31.8% → 35.5% under Gemini). Outcome-only GRPO remains far below its answer accuracy even with the overview counted (27.9% and 24.7% vs. 42.3%), so a large share of its correct answers is unsupported by everything the policy observed.

Tab. R1: LVBench C=Gs=1C=G_s=1 by Evidence Input

MethodAcc.Retrieved (ERM)Overview + Retrieved (ERM)Retrieved (Gemini)Overview + Retrieved (Gemini)
Conan39.70.017.40.015.6
LongVT29.411.816.310.514.6
VideoTemp-o336.813.620.811.818.4
Video-MTR40.414.122.012.819.8
TimeSearch-R + SigLIP41.423.828.520.625.1
SFT43.821.530.218.926.8
Outcome-only GRPO42.319.227.916.824.7
VEPO47.536.340.231.835.5

Separating efficient overview use from evidence-seeking collapse. We use GPT-5 mini to judge whether the overview alone contains sufficient evidence for each LVBench question, separating Overview-Sufficient questions (27.7%) from Search-Required questions (72.3%) in Tab. R2. Outcome-only GRPO does not improve over its SFT initialization on the Overview-Sufficient subset (64.3% vs. 66.0%) and falls further behind on the Search-Required subset (14.0% vs. 16.5%), so its declining search brings no better use of the overview while losing evidence on search-required questions. Existing agentic baselines show the same pattern, performing reasonably on the Overview-Sufficient subset (48.5%–65.5%) but dropping sharply on the Search-Required subset (0.0%–17.4%). VEPO achieves the highest C=Gs=1C=G_s=1 on both subsets (67.8% and 29.6%), with its largest advantage on questions that require additional search.

Tab. R2: LVBench C=Gs=1C=G_s=1 by Overview Sufficiency

MethodAcc.Overview-Sufficient (27.7%)Search-Required (72.3%)Overall
Conan39.762.70.017.4
LongVT29.448.54.016.3
VideoTemp-o336.848.510.220.8
Video-MTR40.465.55.422.0
TimeSearch-R + SigLIP41.457.617.428.5
SFT43.866.016.530.2
Outcome-only GRPO42.364.314.027.9
VEPO47.567.829.640.2

Takeaway: Declining search under outcome-only GRPO does not reflect better use of the overview, since it brings no gain on overview-sufficient questions and loses evidence on search-required ones. With the overview included, the evaluation separates efficient answering from unsupported guessing, and VEPO remains strongest on both question types.

AuthorsOpenReview ↗

Author Response (W2): Independent Evaluation beyond the Training ERM

W2: Independent Evaluation beyond the Training ERM

Judge-free temporal evidence. Temporally grounded correctness (C=Gt=1C=G_t=1 in Table 2) is computed against human-annotated evidence intervals without any learned judge, and VEPO raises it from 9.7% for the strongest baseline to 21.5%.

Independent semantic judges. Appendix C.4 re-evaluates C=Gs=1C=G_s=1 with Gemini 2.5 Flash and five blinded human experts, neither of which participates in training. Each judge receives only the question and the retrieved observations, without the method identity, ERM score, agent reasoning, or predicted answer. We further evaluate the same trajectories with DeepSeek-V4.1-Flash in Tab. R3. A policy that exploited ERM-specific biases would lose its advantage under judges that never interact with training, yet VEPO still ranks first by a wide margin and more than doubles the strongest baseline under all three (31.8% vs. 12.8%, 33.2% vs. 13.5%, and 27.0% vs. 11.4%).

Tab. R3: LVBench C=Gs=1C=G_s=1 across Evaluators

MethodERMGeminiHumanDeepSeek
Conan0.00.00.00.0
LongVT11.810.511.25.3
VideoTemp-o313.611.812.14.1
Video-MTR14.112.813.511.4
TimeSearch-R (w/o SigLIP)17.311.512.08.8
VEPO36.331.833.227.0

Takeaway: VEPO's gain in supported correctness holds under annotation-based temporal grounding, two independent model judges, and blinded human experts, confirming that it is not an artifact of the training-time ERM.

Thanks for the additional experiments. The rebuttal addresses my concerns, and I will keep my score unchanged.

Thank you for your time and constructive feedback. We are glad that the additional experiments and clarifications have addressed your concerns. We will incorporate them into the revised manuscript to make the presentation and supporting evidence clearer.

Reviewer_e2D2OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates evidence-seeking collapse in agentic long-video understanding, where outcome-only reinforcement learning may improve training accuracy while reducing active visual search. The authors propose VEPO, which uses a frozen Evidence Reward Model (ERM) to assess the sufficiency of retrieved visual observations and introduces trajectory-level rewards and action-local credit assignment. Experiments across four benchmarks demonstrate improvements over SFT and conventional GRPO.

Critical strengths
  1. The paper addresses an important problem in agentic video understanding, highlighting the distinction between answer correctness and visual evidence acquisition.
  2. The experiments cover multiple benchmarks, component ablations, training dynamics, and additional evaluations with independent model and human judges.
Critical weaknesses

Major Weaknesses

1. Limited novelty and insufficient distinction from prior work. VideoSEAL (ICML 2026) has investigated evidence misalignment and the effects of outcome-only optimization, while TimeSearch-R (ICLR 2026) introduced completeness self-verification to assess the adequacy of retrieved evidence. VEPO differs through its independently trained ERM and action-local credit assignment, but the advantages of these designs over existing formulations are not sufficiently established. In particular, controlled comparisons against completeness-based rewards are missing.

2. Insufficient validation of evidence-grounded reasoning. The ERM excludes the initial global overview available to the policy. Consequently, correct answers supported by the overview may be classified as lacking retrieved evidence, despite having visual support. Moreover, sufficient retrieved evidence does not establish that the model actually relies on it. Although independent evaluations are provided, they follow the same restricted evidence definition and cannot fully substantiate the broader claims about reducing unsupported reasoning.

3. Inadequate justification of action-local credit assignment. Eq. (4) assigns additional credit using positive increments in ERM sufficiency scores. However, these increments depend on retrieval order and may not faithfully capture the contributions of individual observations. When multiple windows are jointly necessary, earlier supporting observations may receive limited additional credit compared with the final window completing the evidence set. The existing ablations demonstrate performance gains but do not directly validate the quality of action-level attribution.

4. Insufficiently controlled experimental comparisons. VEPO performs substantially more search calls than several competing methods (3.18 versus 1.56 for TimeSearch-R on LVBench), making it difficult to separate the benefits of the reward design from increased retrieval. Furthermore, Table 1 indicates that TimeSearch-R is evaluated with a retrieval component removed, although the original method uses SigLIP-based frame selection. The precise modification and its impact should be clarified. Comparisons under matched retrieval budgets and training configurations would better isolate the proposed method's contribution.

Minor Weaknesses

  1. Figures 3 and 4 contain relatively small annotations and legends. Improving their readability would strengthen the presentation.
  2. The different frame budgets used during policy training, ERM scoring, and evaluation could be summarized more clearly to facilitate reproducibility.
Questions for authors
  1. How does VEPO improve upon the completeness self-verification mechanism in TimeSearch-R under controlled training and retrieval settings?
  2. How would evidence-support results change when including the initial overview? Can evidence-removal experiments establish whether predictions actually depend on retrieved observations?
  3. How reliably do ERM score increments reflect individual search contributions, particularly when complementary evidence is retrieved in different orders?
8 comments
AuthorsOpenReview ↗

Author Response (W1, Q1), Part 1: Core Insights and Related Methods

We thank Reviewer e2D2 for the detailed and constructive review and address each point below.

W1 & Q1: Core Insights and Distinction from TimeSearch-R and VideoSEAL

VEPO's core insight is to assign separate credit to two distinct capabilities: acquiring visual observations that are semantically sufficient to resolve the question, and correctly interpreting the acquired evidence to produce the final answer. Self-verification methods evaluate search quality through answer correctness on the retrieved frames, which still credits incomplete observations whenever the model guesses the right answer. Our frozen ERM directly evaluates whether the accumulated visual observations jointly contain sufficient information to resolve the query, decoupling evidence assessment from terminal answer prediction.

TimeSearch-R:

  • Motivation and core distinction. TimeSearch-R asks the policy to re-answer the question from the retrieved frames and rewards the search trajectory when this re-answer is correct. Search supervision in TimeSearch-R is still derived from answer correctness, allowing trajectories with insufficient visual evidence to receive full search credit whenever the model guesses correctly from parametric priors or coarse context. VEPO evaluates the semantic sufficiency of accumulated observations through the ERM and scores final-answer correctness as a separate objective. Replacing TimeSearch-R's self-verifier with an external answering model or self-consistency leaves this coupling unresolved, as search actions would still be judged through answer prediction.

  • Native evidence acquisition. TimeSearch-R relies on a text-conditioned SigLIP retriever built over a precomputed video-wide frame cache. Under this interface, the policy can submit a broad temporal range and delegate query-conditioned semantic frame selection to the external retriever, while its completeness self-verification (CSV) reward evaluates whether those externally selected frames yield a correct re-answer. VEPO operates without an external semantic frame selector. The environment returns uniformly sampled frames directly from the temporal interval specified by the policy, requiring the policy to learn native temporal localization under direct semantic evidence supervision.

  • Action-local evidence credit. TimeSearch-R assigns a single trajectory-level CSV reward at the end of the rollout, leaving individual search actions indistinguishable within a multi-turn sequence. VEPO evaluates the accumulated observations after each search turn and assigns action-local credit whenever a search action improves joint evidence sufficiency. This step-wise signal directly credits the specific search decisions that resolve the current evidence gap while withholding additional credit from redundant or off-target calls.

VideoSEAL: VideoSEAL addresses architectural coupling by assigning evidence seeking to a planner and stop-and-answer authority to a separate inspector. During training, its reward combines answer correctness with temporal interval overlap, whereas semantic groundedness is used only as a post-hoc evaluation metric. VEPO directly addresses how semantic evidence acquisition is credited during policy optimization. The ERM evaluates the joint semantic sufficiency of accumulated observations as an explicit training signal, and action-local credit attributes sufficiency gains to individual search actions. VideoSEAL decouples agent roles architecturally, whereas VEPO decouples the capabilities being credited during reinforcement learning. These two designs are complementary, and VEPO can provide direct semantic evidence supervision for the planner within a decoupled architecture.

AuthorsOpenReview ↗

Author Response (W1, Q1), Part 2: Core Insights and Related Methods

W1 & Q1 (continued): Core Insights and Distinction from TimeSearch-R and VideoSEAL

Experimental validation against TimeSearch-R:

  • Controlled reward comparison and training dynamics. To isolate the effect of the reward formulation, we hold the Qwen2.5-VL-7B SFT model, RL training data, rollout budget, and native tool interface fixed, and replace VEPO's objective with the TimeSearch-R-style re-answer reward (Tab. R1). Comparing the two runs directly tests whether supervising search through re-answer correctness can sustain active evidence seeking. As shown in Tab. R1(a), optimizing the re-answer reward increases training answer accuracy from the first to the last 25 RL training steps (64.6% to 70.4%), while temporal evidence recall (38.6% to 32.4%) and active search calls (3.603.60 to 2.342.34) decline, and supported correctness remains nearly unchanged (40.4% to 41.1%). Judging search through re-answer correctness still permits unsupported guesses to receive credit, causing the policy to improve its terminal answer predictor while degrading its temporal search behavior.

Tab. R1: Controlled TimeSearch-R-Style Reward Analysis

(a) Measured Training Dynamics

RL StepsAcc.C=Gs=1C=G_s=1E[Gt]\mathbb{E}[G_t]Avg. Calls
0–2464.640.438.63.60
25–4968.437.228.92.67
50–7471.939.925.92.32
75–9971.638.227.72.22
100–12473.141.832.42.30
125–14971.441.930.92.83
150–17472.941.539.42.42
175–19970.441.132.42.34

(b) Final Accuracy under the Same Training Setup

RewardMLVUVideoMMELongVideoBenchLVBench
TimeSearch-R-style67.362.761.345.8
VEPO68.064.463.047.5

(c) LVBench Evidence Acquisition under the Same Training Setup

RewardAcc.C=Gs=1C=G_s=1E[Gt]\mathbb{E}[G_t]Avg. Calls
TimeSearch-R-style45.821.931.92.24
VEPO47.536.339.13.18
  • Benchmark and evidence outcomes. Under this identical training and tool setup, VEPO consistently achieves higher answer accuracy across all four benchmarks (Tab. R1b) and raises evidence-supported correctness (C=Gs=1C=G_s=1) on LVBench from 21.9% to 36.3% (Tab. R1c), while sustaining higher temporal evidence recall and active search. Furthermore, inspecting TimeSearch-R's native tool behavior on LVBench reveals that 65.5% of its search calls request the entire video, confirming that its original pipeline relies heavily on the external SigLIP retriever to locate relevant frames (analyzed under controlled call budgets in Tab. R7). These results demonstrate that explicitly crediting semantic evidence sufficiency prevents evidence-seeking collapse and enables the policy itself to learn effective temporal search.

Takeaway: Acquiring sufficient visual evidence and answering correctly from that evidence are two distinct capabilities. TimeSearch-R derives search credit from re-answer correctness and delegates semantic frame selection to an external retriever. VEPO learns native visual evidence acquisition within the policy and credits evidence sufficiency separately from final answer prediction.

AuthorsOpenReview ↗

Author Response (W2, Q2): Evidence-Grounded Reasoning

W2 & Q2: Evidence-Grounded Reasoning

Including the initial overview. Across long-video benchmarks, the uniformly sampled 64-frame overview covers only about 1% of candidate frames on LVBench and 2% on CG-Bench, providing coarse global context while leaving fine-grained visual details unresolved. To examine whether baseline predictions are already supported by this initial context, we evaluate semantic sufficiency GsG_s over both the retrieved windows alone (Retrieved) and the combined set of overview and retrieved frames (Overview + Retrieved) under both our ERM and the independent Gemini 2.5 Flash judge in Tab. R2. Including the overview increases supported correctness across all evaluated methods, yet VEPO maintains a clear lead under both definitions and both judges, reaching 40.2% (ERM) and 35.5% (Gemini) supported correctness on LVBench.

Tab. R2: LVBench C=Gs=1C=G_s=1 by Evidence Input

MethodAcc.Retrieved (ERM)Overview + Retrieved (ERM)Retrieved (Gemini)Overview + Retrieved (Gemini)
Conan39.70.017.40.015.6
LongVT29.411.816.310.514.6
VideoTemp-o336.813.620.811.818.4
Video-MTR40.414.122.012.819.8
TimeSearch-R + SigLIP41.423.828.520.625.1
SFT43.821.530.218.926.8
Outcome-only GRPO42.319.227.916.824.7
VEPO47.536.340.231.835.5

We further use GPT-5 mini to partition LVBench queries into those where the initial overview is already sufficient to answer the question (Overview-Sufficient, 27.7%) and those requiring additional temporal inspection (Search-Required, 72.3%) in Tab. R3. This split directly tests whether low search frequency in existing baselines reflects efficient early stopping on easy queries or premature termination on unresolved queries. Existing agentic baselines perform reasonably on the Overview-Sufficient subset (48.5%–65.5%) but drop sharply on the Search-Required subset (0.0%–17.4%), and outcome-only GRPO likewise degrades relative to SFT across both subsets (64.3% vs. 66.0% and 14.0% vs. 16.5%). VEPO achieves the highest supported correctness across both subsets (67.8% and 29.6%), with its largest lead on queries requiring active search.

Tab. R3: LVBench C=Gs=1C=G_s=1 by Overview Sufficiency

MethodAcc.Overview-Sufficient (27.7%)Search-Required (72.3%)Overall
Conan39.762.70.017.4
LongVT29.448.54.016.3
VideoTemp-o336.848.510.220.8
Video-MTR40.465.55.422.0
TimeSearch-R + SigLIP41.457.617.428.5
SFT43.866.016.530.2
Outcome-only GRPO42.364.314.027.9
VEPO47.567.829.640.2

Causal reliance on retrieved observations. To verify that the policy actively uses the retrieved observations to produce its final answer, we conduct a counterfactual intervention on LVBench by either removing all search-retrieved frames (w/o Retrieved) or replacing them with randomly sampled off-target windows (Random Windows), while keeping the initial overview and textual interaction trace unchanged (Tab. R4). Disrupting the retrieved visual evidence reduces VEPO's accuracy by 7.0 points, which is more than double the 2.8-point drop observed for outcome-only GRPO. While the outcome-only policy relies primarily on the initial overview and parametric priors, VEPO's accuracy gains causally depend on the visual evidence acquired during search.

Tab. R4: Counterfactual Reliance on Retrieved Observations on LVBench

MethodAcc.w/o RetrievedRandom Windows
Outcome-only GRPO42.339.539.5
VEPO47.540.540.7
Reviewer_GFErOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VEPO studies a training problem in video agents: answer accuracy can increase while temporal search and evidence recall decrease. The method trains a visual evidence reward model to judge whether retrieved observations jointly contain enough information to answer a question.

During policy training, correct answers with sufficient retrieved evidence receive a higher reward than correct answers without it. VEPO also filters rollout groups and adds positive changes in evidence scores to the advantages of the corresponding search-action tokens. Experiments report improvements across four video benchmarks, with additional results on a second policy backbone and independent evidence assessments.

Critical strengths

1. The accuracy gains are consistent across tasks and policy backbones.

With Qwen2.5-VL-7B, VEPO improves over the shared SFT starting point on all four benchmarks, including 58.3% to 63.0% on LongVideoBench and 43.8% to 47.5% on LVBench. The additional Qwen3-VL-4B results support the method beyond one model. The comparisons with outcome-only RL and tool rewards help establish the value of the full training approach.

2. The paper checks evidence quality beyond the training reward model.

Appendix C.4 includes a separate Gemini judge and five blinded human experts. Both support the reported improvement in retrieved-evidence quality. The auxiliary ERM experiments also test whether its selected windows help a separate answering model. These checks reduce the concern that the policy only learns to satisfy its own reward model.

Critical weaknesses

1. The evidence metric excludes visual information available to the policy.

All agentic methods receive 64 overview frames in the main evaluation, but the evidence judges assess only frames returned by later searches. A correct answer supported by the overview can therefore be labeled unsupported. Appendix C.3 explicitly acknowledges this limitation. For example, Conan receives zero supported correctness because it makes no search calls, even though it has seen the overview. The independent judges use the same restricted input, so their agreement does not resolve this issue. During training, the ERM also sees fewer frames per retrieved window than the policy. These choices make it difficult to distinguish unsupported guessing from answering efficiently using visual information already available. This weakens the central interpretation of “evidence-seeking collapse.”

2. The specific value of action-local credit is not fully isolated.

Removing the local bonus reduces accuracy, but this also removes an extra positive signal applied to search actions in correct trajectories. The experiment therefore does not establish whether the gain comes from assigning credit to the right action or from generally reinforcing search in successful trajectories. Equation (4) also keeps score increases while discarding decreases, so changes caused by reward-model noise can add credit without a net improvement in final evidence quality. A control that preserves the total bonus but distributes it equally across search actions would test the claimed credit-assignment benefit more directly.

Questions for authors

1. Could you reassess evidence support using the complete visual input seen by the policy, including the overview?

Please separate cases where the overview is already sufficient from cases that require additional search, and report whether the main comparisons still hold.

  1. The ERM training labels are built from temporal annotations, while the paper argues that temporal coverage does not necessarily imply semantic sufficiency. How often do the sampled frames from a positive context actually contain enough visible detail to answer the question? A manual check of multi-window training examples would help assess this source of label noise.
4 comments
AuthorsOpenReview ↗

Author Response (W1, Q1): Initial Overview in Evidence Evaluation

We thank Reviewer GFEr for the careful review and the concrete suggestions, all of which we report below.

W1 & Q1: Initial Overview in Evidence Evaluation

1. Including the overview in evidence evaluation. We define GsG_s over retrieved observations to measure the policy's ability to actively acquire sufficient evidence through search. The 64-frame overview is a manually specified input provided before interaction, rather than evidence acquired through the policy's search decisions. To further assess whether the final answer is supported by all visual information available to the policy, we also report C=Gs=1C=G_s=1 using the overview and retrieved observations jointly, evaluated by both our ERM and the independent Gemini 2.5 Flash judge, whose retrieved-only results are reported in Appendix C.4.

Tab. R1: LVBench C=Gs=1C=G_s=1 by Evidence Input

MethodAcc.Retrieved (ERM)Overview + Retrieved (ERM)Retrieved (Gemini)Overview + Retrieved (Gemini)
Conan39.70.017.40.015.6
LongVT29.411.816.310.514.6
VideoTemp-o336.813.620.811.818.4
Video-MTR40.414.122.012.819.8
TimeSearch-R + SigLIP41.423.828.520.625.1
SFT43.821.530.218.926.8
Outcome-only GRPO42.319.227.916.824.7
VEPO47.536.340.231.835.5

Including the overview improves supported correctness across all methods, confirming that the initial context supports some predictions. However, a substantial outcome–evidence gap remains under outcome-only GRPO, showing that its unsupported correct answers cannot be explained solely by excluding the overview from evaluation. VEPO remains strongest under both evidence definitions and substantially narrows this gap, demonstrating that its advantage comes from acquiring additional supporting evidence through active search.

2. Distinguishing efficient overview use from evidence-seeking collapse. Reduced search indicates efficient stopping only when the initial overview already contains sufficient evidence. However, the 64 uniformly sampled overview frames cover only about 1% of candidate frames on LVBench and 2% on CG-Bench, leaving many questions dependent on targeted search. We use GPT-5 mini to separate LVBench into overview-sufficient questions (27.7%) and search-required questions (72.3%), and evaluate supported correctness using all observed frames within each subset.

Tab. R2: LVBench C=Gs=1C=G_s=1 by Overview Sufficiency

MethodAcc.Overview-Sufficient (27.7%)Search-Required (72.3%)Overall
Conan39.762.70.017.4
LongVT29.448.54.016.3
VideoTemp-o336.848.510.220.8
Video-MTR40.465.55.422.0
TimeSearch-R + SigLIP41.457.617.428.5
SFT43.866.016.530.2
Outcome-only GRPO42.364.314.027.9
VEPO47.567.829.640.2

Outcome-only GRPO fails to improve even when the overview is already sufficient and degrades further when additional evidence must be acquired through search. Its declining search therefore reflects premature termination rather than more effective use of the overview. VEPO performs best on both subsets, with its largest advantage on search-required questions, showing that it learns to continue searching when the initial context is insufficient.

Takeaway: The retrieved-only metric measures evidence actively acquired through search, while the overview-inclusive metric measures support from the policy's complete visual input. VEPO leads under both definitions. More importantly, outcome-only GRPO gains nothing from reduced search when the overview is sufficient and loses evidence support when additional search is required, confirming evidence-seeking collapse rather than efficient overview use.

AuthorsOpenReview ↗

Author Response (W1–W2): ERM Frame Sampling and Action-Local Credit

W1: ERM Frame Sampling

We sample four frames from each retrieved window to reduce ERM inference cost. To verify that this efficiency choice does not determine the evidence-support results, we rescore the same LVBench rollouts using all eight frames observed by the policy.

Tab. R3: LVBench C=Gs=1C=G_s=1 across ERM Frame Budgets

Method4 Frames/Window8 Frames/Window
Outcome-only GRPO19.221.1
VEPO36.337.9

Using all eight frames produces only modest changes in C=Gs=1C=G_s=1 and preserves VEPO's lead over outcome-only GRPO. The four-frame sampling therefore improves ERM efficiency without changing the main comparative conclusion.

Takeaway: Efficient ERM frame sampling reduces evaluation cost while preserving VEPO's advantage in evidence-supported correctness.

W2: Action-Local Credit Assignment

1. Action-specific credit versus generic search reinforcement. We isolate the value of action-specific attribution by holding the total local bonus of each trajectory fixed and changing only how it is distributed across search actions. The equal-split variant assigns the same share of the total bonus to every search call, while VEPO assigns more credit to actions that increase evidence sufficiency.

Tab. R4: Action-Local versus Equal-Split Credit (Accuracy, %)

Credit AssignmentMLVUVideoMMELongVideoBenchLVBench
VEPO w/o Local Evidence Credit (Table 3)65.461.961.745.1
Equal Split66.863.062.146.1
Action-Local (VEPO, Eq. 4)68.064.463.047.5

Equal-split credit improves over removing local credit entirely, showing that reinforcing search actions in successful trajectories is beneficial. Action-local credit consistently improves further, demonstrating that VEPO benefits not only from additional positive reinforcement but also from assigning it to the searches that increase evidence sufficiency.

2. Robustness of positive evidence increments. We examine whether positive ERM score increments identify evidence-bearing search actions or merely reflect arbitrary score fluctuations. Holding the retrieved windows of VEPO rollouts on LVBench fixed, we reverse or randomly permute their order and measure both where the local credit is assigned and how broadly it covers the annotated evidence. The first metric reports the fraction of total credit assigned to evidence windows, while the second reports the fraction of evidence windows receiving positive credit.

Tab. R5: Alignment on LVBench between Local Credit and Annotated Evidence under Retrieval-Order Changes

Retrieval OrderCredit Assigned to Evidence Windows (%)Evidence Windows Receiving Positive Credit (%)
Original67.053.4
Reversed48.124.5
Random57.836.1

Local credit aligns most strongly with annotated evidence under the policy's original search order. Changing the order reduces this alignment because the marginal value of an observation depends on the evidence already accumulated. Evidence-bearing windows nevertheless continue to receive substantial credit under reversed and random orders, indicating that positive score increments track visual evidence rather than fixed positions in the trajectory. The trajectory-level reward complements this local signal by evaluating the sufficiency of the complete evidence set.

Takeaway: Generic reinforcement of all search actions is beneficial, but assigning credit according to evidence gains performs better because it concentrates policy updates on the searches that acquire supporting visual evidence.

AuthorsOpenReview ↗

Author Response (Q2): Visual Sufficiency of ERM Training Contexts

Q2: Visual Sufficiency of Sampled ERM Training Contexts

1. Offline annotated intervals versus online tIoU rewards. As noted in Section 1, online tIoU is an unreliable RL reward for two reasons: an agent's search window can be overly broad or only partially overlap one event while missing details, and conversely, tIoU falsely penalizes valid searches that gather sufficient visual clues outside the single annotated span. In offline ERM training, however, annotated intervals serve as a high-precision subset of all visually sufficient evidence: each positive context samples directly within the tight ground-truth intervals and includes all supporting windows required by the question, whereas matched negative contexts keep the same distractors but omit required supporting windows. Because the ERM observes only the question and sampled frames without the annotated intervals, it learns to judge visual evidence sufficiency from image content and recovers valid evidence-seeking trajectories beyond the annotated intervals during RL.

2. Manual audit and robustness to residual sampling noise. To directly measure how often four sampled frames from annotated intervals suffice to answer the question, five independent annotators manually audit a randomly sampled set of 500 positive ERM training contexts from CG-Bench (250 single-window and 250 multi-window examples).

Tab. R6: Manual Audit of CG-Bench ERM Training Contexts

SubsetSufficientPartialInsufficientAgreement (κ\kappa)
Single Window (N=250N=250)70.8%11.6%17.6%0.718
Multi Window (N=250N=250)80.4%14.0%5.6%0.746
Overall (N=500N=500)75.6%12.8%11.6%0.732

In Tab. R6, 75.6% of positive contexts overall and 80.4% of multi-window contexts contain fully sufficient visible detail (88.4% and 94.4% including partial support, leaving only 5.6% insufficient in multi-window examples), with substantial inter-annotator agreement (κ=0.732\kappa=0.732). Because ERM training contrasts matched positive and negative pairs and GRPO relies on relative ranking within each rollout group, learning remains robust to the small remaining sampling noise, as confirmed by Table 8, where the trained ERM improves both evidence discrimination and downstream answering accuracy across all four benchmarks.

3. Jointly optimizing two distinct capabilities: searching for relevant frames and answering from them. More broadly, training this dedicated visual ERM supports VEPO's core insight that locating sufficient visual evidence and answering correctly from observed frames are two distinct capabilities. Existing outcome-driven methods only optimize answering accuracy given whatever limited frames have been observed, allowing the policy to guess from sparse context without learning to search. By decoupling these two signals, VEPO jointly optimizes the ability to search for relevant, sufficient visual frames (via ERM-guided action-local credit in Eq. 4) and the ability to answer correctly once relevant frames are acquired (via the evidence-supported trajectory preference in Eq. 3).

Takeaway: Tightly annotated intervals provide a clean visual subset for ERM training, enabling the ERM to recover valid visual searches beyond annotated spans and VEPO to jointly optimize evidence acquisition and answer correctness.

With one day left in the discussion period, we would appreciate it if you could revisit our rebuttal. Our rebuttal addresses each of the points you raised, and we would be happy to clarify anything further. Thank you for your time.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their comments. We have added the controlled comparisons and evidence analyses to the paper. Changes are marked in red.

  • Evidence use. Section 4.1 evaluates whether the observations support a correct answer when the initial overview is included. We also test how accuracy changes when retrieved frames are removed or replaced with random windows. Appendices C.4–C.5 report the independent evidence judgments and the frame interventions.
  • Reward and credit assignment. With the same training setup and tool interface, VEPO achieves higher accuracy than the TimeSearch-R-style re-answer reward on all four benchmarks (Table 13). Appendix C.6 also compares retrieval settings with SigLIP and with a cap on search calls. In Section 4.3, Uniform Local Credit divides each trajectory’s total local evidence bonus equally among its search calls. VEPO’s action-specific credit improves accuracy over this control on all four benchmarks (Table 3).
  • Direct RL on Qwen3.5-9B. Section 4.2 reports results without cold-start SFT. On LVBench, outcome-only GRPO makes no search calls on average and reaches 47.8% accuracy. VEPO averages 3.00 calls and reaches 54.4%. Appendix C.8 reports the training curves and benchmark results (Figure 10 and Table 17).
AuthorsOpenReview ↗

Main Contribution and Summary of the Discussion

We thank the reviewers and the Area Chair for their time. We summarize VEPO’s main contribution and the experiments we added in response to the reviewers.

Finding the right frames and using them to answer correctly are two distinct abilities. Outcome-only reinforcement learning gives an agent full credit whenever its answer is correct, even if it missed the key scene. We observe that agents trained this way can learn to search less and miss more relevant events. VEPO gives the two abilities separate learning signals. A frozen evidence reward model evaluates whether the frames gathered so far contain the clues needed to answer the question. VEPO credits searches that increase this evidence score. We score the final answer separately and use both signals to train the same policy. On LVBench, VEPO improves answer accuracy from 42.3% to 47.5% over outcome-only GRPO and raises the share of questions answered correctly with supporting retrieved frames from 19.2% to 36.3%.

Which searches receive credit matters (GFEr). We held each trajectory’s total search bonus fixed and changed only how it was distributed. Rewarding individual calls according to the evidence they added outperformed splitting the bonus equally across calls on all four benchmarks (Table 3). The benefit is especially clear when the initial overview leaves the question unresolved. On those LVBench questions, VEPO answered 29.6% correctly with visual support, compared with 14.0% for outcome-only GRPO, even when the evaluator could see the overview as well as the retrieved frames.

Finding the right frames and interpreting them correctly are distinct skills (e2D2). An agent can find the key scene and misinterpret it, or miss the scene and still guess correctly. TimeSearch-R rewards a search when the model answers correctly on a second attempt, so a correct guess can earn full credit even when the retrieved frames miss the key event. VideoSEAL separates the planner and inspector and uses temporal overlap as a training signal. In VEPO, the policy chooses where to look in the video and receives uniformly sampled frames from its chosen interval. A frozen evidence reward model scores whether the frames gathered across all searches jointly contain enough information to answer the question. VEPO scores final-answer correctness separately. The trajectory reward favors correct answers backed by a sufficient set of observations, while each search earns additional credit only to the extent that it increases evidence sufficiency. For example, a search can earn credit for finding the first of two events needed to answer a question. With the same training setup and tool interface, VEPO achieved higher answer accuracy on all four benchmarks than a TimeSearch-R-style reward (Table 13). On LVBench, supported correctness also rose from 21.9% to 36.3%. The frame intervention further showed that the agent used the retrieved observations in forming its answers (Appendix C.5).

Independent evaluations confirm the improvement in evidence-grounded answering (eyCL). Five blinded human experts evaluated whether the retrieved frames supported the answer. On LVBench, VEPO answered 33.2% of questions correctly with human-judged visual support, compared with 12.1% for the strongest baseline in that comparison (Appendix C.4). A separate evaluation using human-annotated time intervals reached the same conclusion: VEPO improved temporally grounded correctness from 9.7% to 21.5% over the strongest baseline (Table 2).

VEPO also sustains evidence seeking on a stronger backbone (spJn). We trained Qwen3.5-9B directly with the same RL data and tool budget, without cold-start supervised training. On LVBench, outcome-only GRPO averaged zero search calls and reached 47.8% accuracy. VEPO averaged three calls and reached 54.4% (Table 17). This reproduces the central finding on a second backbone: rewarding the evidence an agent gathers can preserve active search while improving its answers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.