acceptodds
Withdrawn from ICLR 2027

GETR: Guided Exploration Trajectories for Long-Video Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-video reasoning requires finding and combining evidence that may appear throughout an hour-long video. An answer-only reward does not tell the policy whether it inspected the supporting evidence, because the same answer may come from partial or irrelevant observations, or from a guess. Giving the model a tool to look closer does not help by itself, because an untrained model does not know where to look: Qwen3-VL-8B with a crop tool is less accurate than with uniform frames. We propose Guided Exploration Trajectories for Reasoning (GETR), which rewards a long-video policy for looking at the correct evidence. The policy starts from hints, candidate segments retrieved offline from text summaries of the video, and is rewarded for where its first crop falls and for whether its final crop covers the labeled evidence. An IoGT coverage reward measures whether its final crop contains the labeled evidence, and we report how often any crop reaches that evidence. We train the model with GRPO on 1,668 question–answer–evidence triplets, without SFT or labeled tool calls, whereas LongVT and EVA fine-tune on 263K and 21K supervised samples, respectively, including distilled tool-call trajectories. Across four long-video benchmarks (Video-MME-Long, LVBench, HERBench-Lite, and CG-Bench), GETR improves the Qwen3-VL-8B base model by 3.6 points on average, and by 4.4 on which the same model given the crop tool without training instead loses 5.8 points. The gains are largest where the evidence is short relative to the video (CG-Bench +6.3, LVBench +5.4).

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
VideoQA, Text Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_J93LOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

GETR trains Qwen3-VL-8B with GRPO alone (no SFT, no tool traces) on 1,668 question–answer–evidence triplets from the LongVT-Parquet RL split. The agent receives a 256/512-frame uniform pass and two tools:

  • get_hint: returns up to four 30-s windows retrieved offline by matching the question against per-chunk text summaries written by Qwen3-VL-8B.
  • crop_video: returns 8 frames from any window.

Beyond answer and format rewards, the reward includes a thresholded IoGT coverage term on the final crop and an IoU term (α = 0.2) on the first crop. Across Video-MME-Long (with subtitles), LVBench, HERBench-Lite, and CG-Bench, GETR improves the uniform-frame Qwen3-VL-8B by 3.6 points on average (4.4 on the three non-subtitle benchmarks). The paper also reports how often a crop actually reaches the annotated evidence.

Critical strengths
  1. Honest evidence-reach analysis. Table 2, Table 5, Fig. 4b, and the error partition in App. A.4 explicitly show how often crops reach the labeled evidence. Few papers in this area report this, and it is valuable.
  2. Useful negative result. Giving an untrained strong VLM a crop tool lowers accuracy (Table 1, rows "crop" and "crop + hint").
  3. Low supervision budget. The paper uses RL only, on 1.7k examples, and reports CIs for the duration/clue analyses (§4.4).
  4. The argument for IoGT over IoU for an inspecting agent is reasonable.

Critical weaknesses

W1. The key control is missing. The central claim is that rewarding where the policy looked teaches it where to look. The necessary comparison is GRPO with only R_acc + R_fmt on the same backbone, tools, hints, data, frame budgets, and steps. Without it, the gains cannot be attributed to R_iogt and R_loc. Table 4 is a pilot on a different backbone (LongVT-RL), on CG-Bench only, without intervals, and it is not a substitute. An RL baseline without tools (uniform frames only, same data) is also needed to separate "RL on this data" from "tool use".

W2. The gains may reflect protocol compliance rather than better search.

  • The untrained hint+crop model leaves 72.8% of LVBench trajectories without a closed <answer> (§4.1).
  • Fig. 8's before/after trajectories issue identical tool calls; the only difference is that the trained model commits to an answer.
  • No crop reached the evidence for 80% of GETR's correct CG-Bench answers (§4.2).
  • Hit@IoGT rises only from 7.1% to 16.8% on CG-Bench and from 2.1% to 12.5% on LVBench, and only 5.7 of the 11.3-point gain over the untrained crop policy comes from questions where a crop reached the evidence (Fig. 4b).

Against uniform frames, the +3.6 could also come from the extra crop frames. Please add (i) a uniform baseline with the same total number of frames (512 + 8 × average number of crops), and (ii) accuracy restricted to trajectories that close an answer, for all tool rows.

W3. Internal inconsistencies about what was trained. (a) §3.1–3.2 define R_loc as the IoU of the first crop with the labeled evidence, and Fig. 5's caption says GETR uses the evidence-anchored R_loc. Yet §4.3 states "Scored against the hint windows, R_loc is met almost from the first step (mean 0.99)". It presents the evidence-anchored version as a variant "trained with four hints instead of two" whose "evaluation is confounded by a training-mask error and is reported in Appendix A.3". App. A.3 contains no such evaluation. (b) §3.1 argues that hints must be returned unordered and without scores (90% vs. 8% of groups with reward variance). Yet App. A.10 ("starting from the highest-confidence candidate"), A.12 ("returns ranked windows"), Fig. 9 ("ranked by relevance"), and Table 6 ("hint windows and confidences") describe ranked, scored hints. (c) K_max = 4 in the method, but the examples show two hints. Please state exactly which reward, hint count, and hint format the reported checkpoint used, and report the evaluation of the other variant.

W4. Masking incomplete rollouts. Rollouts that do not end in exactly one closed <answer> receive neither reward nor gradient (Eq. 8). The policy is therefore never penalized for failing to answer, and the group statistics are computed over survivors only. Evaluation then adds a forced answer turn. Please report the masked fraction over training and an ablation that assigns these rollouts a zero reward instead.

W5. Train/test mismatch and limited transfer.

  • Training uses 256 frames at 25,088 px; evaluation uses 512 frames at 50,176 px.
  • Training videos have a median length of 3.4 min, with evidence spanning a median 7.5% of the video. On LVBench and CG-Bench the evidence spans only 0.5–0.6% (App. A.3).

The paper's own numbers show that coverage transfers poorly to long videos (final-crop recall of 11.6% on LVBench and 16.7% on CG-Bench), which undercuts the claim that GETR "learns where to look" on hour-long videos.

W6. Evaluation details.

  • All multiple-choice accuracies are scored by a Qwen2.5-72B judge (App. A.7), which also extracts options from unstructured text for non-compliant baselines. Rule-based option extraction should be primary, with the judge reported separately.
  • Video-MME-Long is evaluated with subtitles, where text alone reaches 60.1.
  • Results are single runs.
  • In Table 3, the subset intervals (about ±4) make most hint conditions statistically indistinguishable, as the authors acknowledge.
Questions for authors
  1. Which R_loc (hint-anchored or evidence-anchored), how many hints, and ranked or unranked hints were used for the checkpoint in Table 1? Where is the evaluation referenced as "reported in Appendix A.3"?
  2. What does answer-only GRPO with the same setup achieve?
  3. What fraction of training rollouts were masked by Eq. 8, and how did it evolve?
  4. How do accuracies change when MCQ answers are extracted by rule instead of by the LLM judge?
Reviewer_Lh6QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GETR trains a long-video reasoning policy using GRPO on 1,668 question–answer–evidence examples, without supervised fine-tuning on tool trajectories. Offline video summaries provide candidate temporal windows, and the policy uses a cropping tool to gather further observations. The stated reward combines answer correctness, format validity, first-crop localization, and final-crop evidence coverage measured by IoGT. The paper reports a 3.6-point average improvement over uniform-frame Qwen3-VL-8B across four benchmarks and analyzes how frequently crop intervals overlap annotated evidence. However, inconsistencies concerning the evaluated reward and hint configurations, together with limitations of interval-based coverage, weaken the central interpretation.

Critical strengths

Training without tool-trajectory SFT on only 1,668 examples is attractive. The controlled comparisons against the same Qwen3-VL-8B backbone report improvements across all four benchmarks, including 5.4 points on LVBench and 6.3 points on CG-Bench. This provides promising evidence that targeted RL supervision can improve long-video interaction without a large trajectory-distillation stage.

The analysis explicitly acknowledges that most correct GETR answers on CG-Bench are not accompanied by a crop satisfying the coverage criterion, and that initial frames may already contain relevant evidence. This is an informative distinction that prevents answer accuracy from being treated as sufficient evidence of successful active search.

Critical weaknesses
  1. Equation 6 defines the first-crop reward against ground-truth evidence, and Section 3 describes up to four unordered, unscored hints. However, Section 4.3 discusses a hint-targeted reward and introduces the evidence-targeted, four-hint configuration as a variant whose evaluation is confounded by a training-mask error. Appendix A.10 additionally refers to selecting the highest-confidence candidate, while the audited examples include ranked two-hint outputs. The referenced Appendix A.3 does not resolve the mask-error issue. These inconsistencies make it difficult to determine which checkpoint, reward, and hint interface generated the main results, or whether the intended method has been cleanly validated.

  2. Each crop returns only eight sampled frames, whereas the terminal reward and coverage metrics operate on the continuous crop interval. A sufficiently broad interval can fully contain a short annotated event while all eight sampled frames miss it. Since terminal IoGT does not penalize excess duration, broad crops can obtain maximal coverage reward without guaranteeing informative observations. Consequently, the reported coverage gains do not establish that the policy actually observed or used the decisive evidence. The paper needs frame-level evidence checks and a controlled analysis separating grounding improvements from format recovery and ordinary RL gains.

Questions for authors
  1. Which exact checkpoint, first-crop reward, hint count, and hint ordering produced Tables 1–3? Please reconcile Sections 3 and 4.3 with Appendices A.3 and A.10 and explain the training-mask error.

  2. How often do crops satisfying IoGT actually return frames containing the annotated evidence? Please report crop-duration distributions and frame-level evidence inclusion, particularly for short events.

  3. After correcting the implementation issues, what are the results for answer-only RL, RL with hints, and the full reward on the same backbone, with matched prompts, generation budgets, and final-answer recovery rules?

Reviewer_YJJpOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

To address the limitations in long-video reasoning, specifically the inability to determine whether a model has located supporting evidence when relying solely on "answer-based" rewards, and the fact that untrained models fail to use cropping tools effectively, this paper proposes a method called GETR. Starting with candidate segments retrieved from offline text summaries, GETR trains a long-video policy using GRPO. It provides rewards based on whether the initial crop location and the final crop successfully encompass the annotated evidence, all without the need for SFT or explicit tool-use annotations. The authors evaluate GETR across four long-video benchmarks, reporting average performance gains over the Qwen3-VL-8B base model, evidence coverage metrics, and comparisons against a baseline using an untrained cropping tool.

Critical strengths

The reward design focuses on "whether the correct evidence was observed" rather than just the final answer, which helps distinguish between effective evidence localization and mere guessing or the observation of irrelevant content.

The training process does not rely on SFT or traces of tool-use annotations; it utilizes only 1,668 question-answer-evidence triplets, demonstrating high data efficiency.

The method achieves an average improvement of 3.6 points across four long-video benchmarks, with more pronounced gains in tasks where the evidence constitutes a relatively small portion of the video. Furthermore, comparisons show that providing an untrained model with a cropping tool can actually degrade performance.

Critical weaknesses

The coverage reward checks only one piece of evidence at a time. This makes it hard to handle cases with many pieces of evidence spread out across the video.

The method mainly helps the model find “where to look.” It does not make the model reason better after it has found the evidence.

The method depends on how well prompts match visual summaries. It may miss spoken content. The search also helps less when the answer is already in evenly sampled frames or existing transcripts.

The actual amount of evidence covered is still low. This is especially true when the evidence is only a very small part of a long video.

High accuracy can hide problems in finding evidence. Many correct answers are reached without the model actually cropping the marked evidence.

Questions for authors

Please refer to the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.