acceptodds
Desk rejected at ICLR 2027

WORTH: Decision-Aligned Credit Assignment for Evidence Acquisition in Long Video Agents

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Agentic video understanding has emerged as a promising paradigm for long-video reasoning, wherein an agent first perceives a sparsely sampled overview and then selectively retrieves evidence from task-relevant temporal segments. However, existing agents are typically optimized via outcome-based reinforcement learning with a single trajectory-level reward, which neither evaluates whether a given acquisition justifies its cost nor attributes the final outcome to individual observations. We term this deficiency the acquisition–attribution mismatch, and our systematic analysis shows that it drives agents to acquire evidence indiscriminately while most of the acquired observations contribute nothing to the final answer. To address this, we propose WORTH (Whether Observation Retrieval Truly Helps), a decision-aligned reinforcement learning framework that decouples acquisition from attribution via two dedicated credit signals. An acquisition credit estimates the ex-ante value of retrieval as the cost-adjusted gain of acquiring over answering immediately, computed through paired continuations from the same state. An observation credit estimates the ex-post contribution of each retrieved observation as its exact Shapley value, obtained by re-answering from every subset of acquired observations. Both credits are derived from the policy itself and replace the trajectory-level advantage exclusively on the tokens corresponding to their respective decisions. Experiments demonstrate that WORTH achieves state-of-the-art performance among most video agents across multiple long-video understanding and temporal grounding benchmarks, improving accuracy by 2.2% over outcome-based RL baselines while consuming 67% fewer tokens. Code, data, and models will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Long-Video Understanding, Video Agents, Agentic Reinforcement Learning, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_faFTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

WORTH separates whether to acquire evidence from which observations contribute to the answer. After an evidence-verified cold start, it assigns cost-adjusted acquisition credit to the first branch decision and Shapley-based credits to the first plan's requests. Other generated tokens retain the GRPO advantage. Video QA and temporal grounding experiments report improved accuracy with lower inference token use.

Critical strengths
  1. Controlled accuracy-efficiency gains. Table 4 controls initialization, data, and interface. WORTH reduces the acquisition rate from 92% to 26% and normalized token use from 3.9 to 1.3 while improving accuracy. The alternative-baseline ablation also supports comparing acquisition against answering immediately.
  2. Credit for redundant evidence. Shapley attribution can share credit between interchangeable observations that leave-one-out attribution would miss (Eq. 6). Assigning that credit to the corresponding request tokens connects supervision to the acquisition decision.
  3. Evaluation beyond answer accuracy. Tables 2-3 cover video QA and temporal grounding, with grounding also included in Table 4. This tests whether the cheaper acquisition policy retains localization ability, supporting the overall system beyond QA gains.
Critical weaknesses
  1. Accuracy gains are less pronounced than efficiency gains. In Table 4, normalized token use falls from 3.9 to 1.3, while accuracy improves by 0.4-4.0 percentage points across five QA columns. Observation credit alone adds 0.3-1.5 points. Whether higher inference expenditure could yield further accuracy gains remains unexplored.
  2. Limited decision-level coverage. New credits apply only to the initial branch and first plan, although episodes allow three plans (Sec. 3.2). Initial acquisition credit includes downstream returns, but later requests receive no separate counterfactual credit. Their frequency, cost, and accuracy contribution are not quantified for the evaluated policy.
  3. The need for exhaustive attribution is not fully established. The diagnostic analysis motivates accounting for redundant observations, but Table 4 compares observation credit with no observation credit rather than cheaper approximations. It remains unclear whether exhaustive subset enumeration is necessary to obtain the reported accuracy gains.
Questions for authors
  1. Can WORTH trade its inference savings for higher accuracy? An accuracy-cost curve including Policy and Reader tokens would help. If so, are these results obtained by changing inference settings for the same model, or by retraining with different cost penalties?
  2. For the evaluated policy, how often are second and third plans used, how much token cost do they add, and what evidence indicates that they improve answer accuracy?
  3. With acquisition credit unchanged, how does exhaustive Shapley attribution compare with a cheaper approximation, such as sampled Shapley, in accuracy and training cost?
  4. Table D.3 reports 6.4 minutes per step for WORTH versus 4.8 for GRPO. How much of the added wall-clock time comes from forced continuations, including Reader calls, and from Shapley subset readouts? Please account for overlap from parallel execution.
Reviewer_tmE6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies RL-trained agents for long-video QA that can either answer from a sparse 64-frame overview or request temporal segments for closer inspection. Using forced readouts on four public agents, it argues that trajectory-level outcome rewards cannot tell whether an acquisition was necessary, and that most acquired observations do not change the answer. The proposed method, WORTH, adds two credits on top of GRPO: an acquisition credit that compares "acquire" with "answer now" from the initial state, with an evidence cost deducted, and an observation credit given by the exact Shapley value of each observation in the first plan. Starting from a large evidence-verified SFT cold start on Qwen3-VL-8B, WORTH improves accuracy over GRPO by about 1.8 points on average, while reducing the acquisition rate from 92% to 26% and the evidence cost to one-third.

Critical strengths
  • The diagnostic protocol (answer without acquisition / with requests only / with observations) is simple and gives a clean decomposition of the effect of an acquisition. Uncertainty is handled properly, with paired readouts and bootstrapping over videos.
  • The method is clearly connected to the diagnosis. The analysis of why GRPO gives almost no signal on the acquire decision when all rollouts acquire is correct and useful.
  • The RL ablations start from the same cold start and data, so the effect of the training signal is isolated.
  • The paper is clearly written.
Critical weaknesses

Main concern: WORTH teaches the agent to answer from insufficient evidence

It seems to me that WORTH encourages the agent to acquire less and answer directly, and that the way it does so rewards answering from insufficient evidence. My concern is that this kind of learning teaches the model to guess, or even hallucinate, an answer when the evidence it has seen does not support one. I explain the reasons below.

  1. The objective rewards correct answers, not supported answers. The acquisition credit (Eq. 4) is

Δ=E[R−λC∣acquire]−E[R∣answer now].\Delta = \mathbb{E}[R - \lambda C \mid \text{acquire}] - \mathbb{E}[R \mid \text{answer now}].

If the direct answer is correct, Δ≤−λ E[C]≤0\Delta \le -\lambda\,\mathbb{E}[C] \le 0, so answering directly always wins. The credit does not ask why the direct answer is correct: visual evidence in the overview, language priors, option elimination, and a lucky guess on a 4–5-way multiple-choice question all receive the same credit. Moreover, per Eq. 7, the answer tokens keep the plain outcome advantage AbaseA^{\text{base}}, so nothing in the objective requires the final answer to be supported by evidence. A correct guess and a grounded answer are rewarded identically, but the grounded answer additionally pays the evidence cost λC\lambda C.

  1. The paper's own diagnosis suggests that many "correct without acquisition" answers are not grounded. Finding 2 and App. B.4 show that the answers of these agents are largely insensitive to evidence: returned evidence has no consistent net effect, and revisions made after seeing evidence are no more accurate than revisions made with the evidence withheld. In other words, a correct answer without acquisition often does not mean the overview contained the evidence. It means the model's answer was correct regardless of evidence. WORTH turns exactly this situation into a learning target: when the answer is correct without evidence, stop looking.

  2. RL pushes direct answering well beyond what was verified as sufficient. In the evidence-verified cold-start corpus, only 33% of trajectories answer directly (about 39% of QA trajectories when the grounding ones are excluded), and only when the planner judges every need to be visible in the overview (App. C.2, Fig. E.8). After SFT, the policy answers directly on 43% of questions; after WORTH, on 74% (Table 4). The training and evaluation distributions differ, but the gap is large, and the paper does not show that these additional direct answers are supported by the overview.

  3. The removed acquisition is often the only close look at the video. Fig. 1(a) shows that Video-o3 and LongVT make about one call per rollout, VideoTemp-o3 fewer than one, and only ParaVT 4–5. "Indiscriminate acquisition" is therefore mostly a single retrieval. With 64 overview frames, a one-hour video is seen about once per minute, so this retrieval is the agent's only grounding step. Consistently, the gains over GRPO come mainly from MLVU (+4.0) and Video-MME (+1.9), where the overview often suffices, while on LVBench WORTH gets 41.6 vs. 41.2 (GRPO) and 41.5 (Video-o3).

  4. Groundedness is never measured. Accuracy and cost cannot tell whether the 74% direct answers rely on evidence or on guessing.

Questions for authors
  1. Among WORTH's correct direct answers, how many are supported by evidence in the overview frames? How does this compare with GRPO and the SFT model?
  2. Why does WORTH answer directly on 74% of questions when only 33% of the verified cold-start trajectories were judged answerable from the overview?
  3. What are the acquisition rate and accuracy per video-duration bucket, especially for videos longer than 30 minutes? Which models generated the cold-start data, and what does SFT-only achieve in Tables 2–3?
  4. Relation to VideoSEAL. The insight of this paper runs somewhat opposite to VideoSEAL ( ICML 2026). VideoSEAL treats correct-but-ungrounded answers as the main failure of outcome-only RL for long-video agents and requires verified evidence before answering. WORTH's acquisition credit instead rewards answering directly whenever the outcome is correct, regardless of whether the answer is supported by evidence. Could the authors discuss this tension directly in the main text: should a long-video agent be optimized to be correct, or to be correct with evidence? Since part of WORTH's gain may come from answers driven by priors rather than evidence, could the authors also compare the two approaches in a setting that is fully out-of-distribution for both, i.e., videos and questions on which neither method was trained and where language priors are uninformative (e.g., counterfactual or synthetic content, or questions whose evidence lies outside the overview frames)? This would show which design generalizes when guessing no longer pays off.
Reviewer_feXMOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the necessity and effectiveness of evidence acquisition in long-video agents. Through a diagnostic analysis of four existing agents, the authors identify three problems: (1) agents acquire evidence indiscriminately; (2) acquisition yields little net gain; and (3) most observations do not contribute to the final answer. The authors attribute these phenomena to the conflation of the ex-ante value of acquisition with the ex-post contribution of individual observations, termed the acquisition–attribution mismatch.

To address this issue, the authors propose a decision-aligned reinforcement learning framework that separates acquisition from attribution through two dedicated credit signals. Acquisition credit estimates whether acquiring evidence is worthwhile, while observation credit estimates the contribution of each acquired observation. Both credits replace the trajectory-level advantage exclusively on the tokens corresponding to their respective decisions. Experiments based on Qwen3-VL-8B demonstrate improved performance across multiple video understanding and temporal grounding benchmarks, together with reduced evidence acquisition costs.

Critical strengths

In Table 1, the paper uses various metrics to diagnose multiple video agents, raises the problems, and analyzes their causes, which strongly establishes the motivation of the paper.

The authors compare with multiple models on five Long Video Understanding Benchmarks and three Temporal Grounding Benchmarks, making the evaluation comprehensive.

In the ablation study in Table 4, WORTH achieves modest performance improvements over the outcome-based RL baseline while reducing token consumption by approximately 67%. This is a valuable improvement in efficiency.

Critical weaknesses

Table 1 uses metrics such as Acq., U_acq, Helped, and Harmed to diagnose the problems. However, the subsequent evaluation primarily reports final accuracy, acquisition rate, and token costs. These aggregate improvements do not sufficiently establish that WORTH resolves the three initially diagnosed problems. I would encourage the authors to repeat the same diagnostic analysis on the trained WORTH model.

The authors discuss the limitations of leave-one-out in Section 2.4. However, computing Shapley values requires considerable additional computation. I hope the authors can compare the training results of the two methods to better demonstrate the necessity of using Shapley values.

In the computation of Observation Credit, the authors do not consider cases such as Figure F.13, where one episode requires multiple rounds of planning. They only compute Shapley values for multiple parallel retrieval requests in the first plan, while setting subsequent plans to outcome advantage. I hope the authors can further explain the reason.

Questions for authors

Please find the comments from weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.