- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Counterfactual Information Gain Policy Optimization (CIGO) to improve credit assignment for tool-using video agents, focusing on when and where to retrieve additional frames. CIGO combines counterfactual STOP/LOOK rollouts, a likelihood-based information reward, and a replay buffer for training multi-step retrieval decisions. Starting from Qwen3-VL-8B-Thinking, the method uses 4.8K RL samples without SFT and reports gains of 0.5–4.8 points across long-video QA and temporal grounding benchmarks. Additional analyses examine tool-call behavior, retrieval quality, and performance on blind versus non-blind questions.
Critical strengths
- Well-motivated problem. CIGO addresses the limitations of outcome-only rewards by evaluating the state-dependent value of retrieval decisions.
- Simple and efficient training. Counterfactual rollouts provide informative learning signals even when correctness rewards are identical. Training requires only 4.8K samples without SFT or teacher-generated trajectories.
- Useful behavioral analysis. The paper examines tool-call distributions, retrieval quality, and blind/non-blind performance, providing insights beyond aggregate accuracy.
- Good reproducibility. Training configurations, prompt templates, and an anonymized code repository are provided.
Critical weaknesses
- Small gains and unfair comparisons. Improvements over the base model are only 0.5–1.9 points on four of five QA benchmarks, without statistical significance analysis. Most competing agents use weaker base models and different retrieval budgets, making direct comparisons difficult.
- Weak ablation baselines and limited evidence of adaptive retrieval. GRPO variants perform substantially worse than the untrained base model, suggesting CIGO may primarily learn to avoid harmful retrieval. Moreover, tool-call allocation shows no adaptation on LVBench (1.0× necessary/unnecessary ratio), weakening the central claim.
- Unclear reward design and implementation. The self-referential information reward may conflate retrieved evidence with generated reasoning, potentially encouraging reward hacking. The STOP/LOOK training mechanism, discount factor γ, and likelihood computation for temporal grounding are insufficiently explained.
- Limited novelty and missing efficiency analysis. The main components closely resemble prior work (CaRT, IGPO, CIGPO), without controlled comparisons. Despite emphasizing retrieval efficiency, the paper reports no token or latency measurements. Minor presentation issues, including an empty appendix and inconsistent notation, remain.
Questions for authors
- Could you report results for the zero-shot base model with tool access, along with CIGO's evaluation-time tool-call distribution and accuracy conditioned on the number of calls? Why do the GRPO baselines perform substantially worse than the no-tool base model?
- Could you provide confidence intervals or results across multiple seeds, and compare CIGO with IGPO- or CaRT-style objectives using the same base model and training data?
- How are STOP/LOOK continuations sampled or enforced during training? How is γ applied, and how is the information reward computed for temporal grounding? Have you investigated potential reward hacking through generated reasoning?
- Why does adaptive retrieval show no improvement on LVBench? Could you also report per-question token consumption and latency to substantiate the claimed efficiency benefits?
5 comments
"The self-referential information reward may conflate retrieved evidence with generated reasoning, potentially encouraging reward hacking."
In the proposed method (post-rationale information potential), the rationale is unlikely to lead to reward hacking or inflation due to 2 designs: (1) Φ is scored right before the final answer, and all continuations are sampled from the same fixed prefix. Therefore the model must make decisions that lead to more information gain before the answer, either it is directly answering or calling the tool. (2) we normalize within the valid options, so Φ measures the belief shift among the options rather than the probability of emitting an answer at that point.
We implement the CIGO variant that assigns information reward post-retrieval before further reasoning, and observe that gain still persists if we score Φ post-retrieval. Our full analysis is in GR Q6. We argue that measuring evidence-only information gain is another effective instantiation for our proposed training framework, and we have added this post-retrieval variant to the paper (App. D.2, Table 10).
| Φ scored | Video-MME | MLVU | LVBench | LongVideoBench | Video-MMMU | Charades-STA | Tool calls at evaluation (%): 0 / 1 / 2 / 3 |
|---|---|---|---|---|---|---|---|
| Base model | 67.4 | 72.0 | 45.1 | 62.3 | 67.5 | 49.9 | 100 / – / – / – |
| Post-rationale (default) | 68.1 | 72.7 | 47.9 | 63.8 | 68.4 | 57.5 | 65.6 / 13.6 / 17.3 / 3.6 |
| Post-retrieval | 68.9 | 73.1 | 48.7 | 64.4 | 69.6 | 59.1 | 12.5 / 85.1 / 1.2 / 1.3 |
"The STOP/LOOK training mechanism, discount factor γ, and likelihood computation for temporal grounding are insufficiently explained."
Please refer to our full response in GR Q8 (branch sampling), GR Q7 (γ) and GR Q5 (Φ) for detailed clarifications. We have also clarified all three in Sec. 3 of the revised paper. In short:
- Branch sampling. From each prefix, the policy samples 8 continuations, each of which either answers directly or calls the tool once and then answers. Branches are sampled freely; forcing both branches lowers performance on all six benchmarks.
- Discount γ. γ multiplies only the correctness reward of LOOK continuations. CIGO uses γ = 1.0; with γ < 1, we observe that the policy converge to a no-call behavior.
- Φ for temporal grounding. We score only the timestamp digits, renormalize each digit over 0–9, and average over digit positions.
"The main components closely resemble prior work (CaRT, IGPO, CIGPO), without controlled comparisons."
IGPO. We reproduce IGPO on the same base model, data and budget (Table 2). CIGO outperforms it on all six benchmarks (+0.6 to +2.8). Like the other standard-rollout methods, IGPO calls the tool exactly once on 92.6% of questions.
| Method | Video-MME | MLVU | LVBench | LongVideoBench | Video-MMMU | Charades-STA | Exactly 1 call (%) |
|---|---|---|---|---|---|---|---|
| Base, no tools | 67.4 | 72.0 | 45.1 | 62.3 | 67.5 | 49.9 | – |
| IGPO | 65.6 | 71.5 | 45.1 | 62.6 | 67.8 | 56.7 | 92.6 |
| CIGO | 68.1 | 72.7 | 47.9 | 63.8 | 68.4 | 57.5 | 13.6 |
CaRT. We thank the reviewer for request the additional of this meaningful baseline. Unfortunately, due to time and resource limitations, we could not reproduce CaRT's counterfactual rollout sampling for supervised fine-tuning. We provide a table to specify how our proposed CIGO distinguish from existing related methods:
| Method | Rollout start | What is compared | Training signal |
|---|---|---|---|
| IGPO | Question | Consecutive turns of one trajectory | Outcome + per-turn information gain |
| CIGPO | Question | Consecutive evidence sets (text QA) | Outcome + information gain |
| CaRT | Prefixes selected offline where the decision changes the outcome | Edited prefixes (counterfactual states) | SFT on stop/continue labels from an external model; RL variant rewards agreement with the labels |
| CIGO | Question, or a policy-sampled prefix from a replay buffer | STOP vs. LOOK from the same prefix (counterfactual actions) | Correctness + counterfactual information gain; no SFT |
We added this comparison to App. C.1 (Table 7) and the IGPO reproduction to App. C.2 (Table 8).
"The paper reports no token or latency measurements."
We provide a detailed response in GR Q3 and the full analysis on all six benchmarks in Sec. 4.5 (Fig. 4) of the revised paper. Below we summarize the key takeaways and numbers on three benchmarks.
Takeaway. CIGO lies on the accuracy–token frontier on all three benchmarks and is the most accurate agent on Video-MME and Video-MMMU at a similar token count. Its latency is higher, mainly inherited from its Thinking base model, which already takes 23.2 s per question on Video-MME without tools.
| Agent | Video-MME: acc. / tokens / s | Video-MMMU: acc. / tokens / s | Charades-STA: mIoU / tokens / s |
|---|---|---|---|
| Base (no tools), reduced | 65.0 / 7.2k / 20.7 | 62.5 / 10.3k / 78.8 | 45.7 / 2.3k / 7.6 |
| Base (no tools), default | 67.4 / 19.8k / 23.2 | 67.5 / 22.6k / 87.5 | 49.9 / 4.7k / 9.3 |
| Video-o3-7B | 66.5 / 18.0k / 9.0 | 51.7 / 19.2k / 10.6 | 60.7 / 6.0k / 2.8 |
| VideoTemp-o3-7B | 64.5 / 22.7k / 23.2 | 53.2 / 34.4k / 37.6 | 57.8 / 8.3k / 4.7 |
| CIGO, reduced | 65.1 / 8.8k / 29.8 | 62.4 / 10.7k / 66.6 | 54.9 / 3.3k / 12.1 |
| CIGO, default | 68.1 / 21.5k / 35.8 | 68.4 / 23.0k / 75.5 | 57.5 / 5.6k / 13.5 |
Default: the setting of Sec. 4.1; reduced: 256 frames at 224². Tokens and latency are per question; latency on one L40S GPU, batch size 1.
"Minor presentation issues, including an empty appendix and inconsistent notation."
We completed the appendix, including method and training details, comparison with related methods, additional ablations, tool-call and inference-cost analyses, and have fixed previous the notation inconsistencies in Sec. 3.
Thank you again for your insightful comments, which have helped improve the paper. We hope these results address your concerns, and we are happy to answer any further questions.
Dear Reviewer 51g3,
Thank you for your thorough review and your insightful feedback. We answer each point below; each answer gives the key results and refers to the General Response (GR) where it adds more detail, and all new results are in the revised PDF.
Over 3 seeds, the gain exceeds one standard deviation on LVBench, LongVideoBench and Charades-STA. For a controlled comparison, Table 2 of the paper trains every variant from the same base model on the same data and call budget. We also add the base model with and without tools (table below). We also reproduce IGPO on the same base model, data and budget (results under the novelty question below).
Bold: the gain over Base exceeds one SD. Tool access alone does not improve the base model on any of the 6 benchmarks.
Table 1 now reports the seed statistics, and Table 2 adds the base model with tools.
As shown in Sec. 4.4, CIGO calls more when the information is insufficient than when it is sufficient (1.33× on LVBench and 4.41× on ReXTime), whereas existing agents call a similar number of times regardless. We note that CIGO not only avoids unnecessary tool calls, but also improves tool-call quality: its retrieved windows are the most accurate and the most helpful to a reference model (Sec. 4.4, Table 4).
We also re-evaluated all our models and reproduced baselines under a unified evaluation protocol, with frame resizing disabled in the evaluation framework following Qwen3-VL’s suggested practice. Under this protocol, GRPO is within 1.6 points of the base model on long-video QA and above it on Video-MMMU and Charades-STA:
The original “unnecessary” label (whether the ground truth evidence improves the reference model's success rate) mixes 2 groups: (a) questions the overview already answers, and (b) questions that even adding the ground-truth frames cannot reliably solve. In the latter case, it is still reasonable to call the tool, since the existing information is insufficient for answering the question. Split by whether the overview suffices, CIGO calls 1.33× more on insufficient questions on LVBench and 4.41× on ReXTime; it is the only agent whose 95% CI excludes 1 on LVBench.
Ratio: mean calls on insufficient questions / mean calls on sufficient questions. †: the 95% bootstrap CI includes 1.
We updated Table 3 (Sec. 4.4) with this split; GR Q4 gives the definitions of the three groups and the mean number of calls in each.