- Rating
- 3(+1)
- Confidence
- 3(+1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether video agents actually use the visual evidence returned by tools. It defines a commitment point based on a decline in the proportion of high-entropy reasoning tokens, distinguishing between agents that commit before receiving tool results and agents that commit after uninformative results. The proposed ERA framework combines a commitment-location reward with a counterfactual contrastive reward to encourage further retrieval when evidence is insufficient. Experiments compare different SFT and RL recipes across video benchmarks under both ordinary tool returns and interventions involving substituted returns and assertive messages.
Critical strengths
The paper addresses an important problem with a useful diagnostic perspective. Accuracy and tool-call frequency alone do not establish whether an agent responds to acquired evidence. Connecting the timing of commitment with counterfactual tool returns exposes distinct failure modes that conventional evaluations can miss. This is a meaningful contribution to understanding tool use in video agents.
The improvements under intervention are supported by both behavioral measures and answer accuracy. For example, on VideoNet-Gen, accuracy under the pressure protocol increases from 0.31 for SFT-Mix to 0.50 with ERA. Results on VideoZoomer also provide evidence of transfer beyond a single initialization. The separation of SFT and RL further illustrates that learning the form of re-seeking does not necessarily produce evidence-dependent behavior.
Critical weaknesses
The training objective and a central evaluation metric share the same proxy definition, creating a risk of circular validation. Entropy changes are used both to construct the reward and to classify behaviors such as overreliance. The policy could therefore learn the desired entropy pattern without a corresponding improvement in evidence use. Counterfactual answer accuracy provides some independent support, but further validation is needed to establish that the commitment point reliably measures evidence use across models and training conditions.
The strongest gains are concentrated in a specific artificial intervention protocol. With real tool returns, ERA improves accuracy over the strongest same-backbone training baseline by approximately one percentage point. Larger gains primarily arise when returns are substituted and accompanied by assertive messages. Together with model-dependent certification of tool-required questions and relatively small evaluation subsets, this limits the evidence for broader improvements in video reasoning and naturally occurring tool failures.
Questions for authors
Do you have validation independent of the entropy-based definition, such as human annotations of evidence use, changes in answer distributions following evidence substitution, or agreement between detected commitment points and actual changes in the agent’s answer?
Could you report separate results for substituted returns without assertive messages and assertive messages without substituted returns? Do the gains persist under naturally occurring retrieval failures or on unfiltered benchmark splits? These results would materially affect my assessment of the method’s generality.
3 comments
One part of Q1 remains open for me. Could you report, on the same set of trajectories, how often the answer changes after substituting the tool return, broken down by the original entropy-based classification? In particular, are trajectories classified as evidence-resolved more sensitive to evidence substitution than those classified as overlook?
Dear Reviewer Uxyp,
Thank you for the follow-up. We ran the comparison you describe on one set of trajectories per model, with one substitution rule and the original entropy-based labels.
- Trajectories. Qwen3.5-9B (three inference runs), SFT-Mix + ERA, and ERA-RL (no SFT) (training seeds 0, 1, 2) on VideoNet-Gen (100 questions), with 8 frames per seek, real returns, and plain messages. Each trajectory keeps its original label from Eq. 2, whether its answer was correct or not.
- Substitution. We keep each trajectory fixed up to its last tool return and continue it twice with greedy decoding, once with the original return and once with a segment outside . The answer changes if the two continuations end in different options.
- Must answer. In both continuations, the message with the return asks for an answer in this turn, so the agent cannot recover by seeking again.
- Free. No instruction is added. The agent may seek again and then receives the segments it requests.
Table R2. Substituting the last tool return, by original classification. Changed: pairs whose two continuations end in different options / valid pairs, in which both continuations give a final option. Seek again: share of substituted continuations that call the tool again. Overrely requires a return without the evidence, which is rare with real returns, so it is not compared. Trajectories with no tool call or no detected commitment point are also left out.
──────────────────────────────────────────────────────────────────────────────────────
Must answer Free
───────────────── ──────────────────────────────
Model Classification Changed Seek again Changed
──────────────────────────────────────────────────────────────────────────────────────
Qwen3.5-9B Overlook 9/161 (5.6%) 0.0% 10/162 (6.2%)
Evidence-resolved 46/113 (40.7%) 0.0% 47/115 (40.9%)
SFT-Mix + ERA Overlook 5/60 (8.3%) 11.3% 4/61 (6.6%)
Evidence-resolved 104/197 (52.8%) 47.7% 61/203 (30.0%)
ERA-RL (no SFT) Overlook 8/73 (11.0%) 19.2% 7/69 (10.1%)
Evidence-resolved 70/154 (45.5%) 55.1% 39/141 (27.7%)
──────────────────────────────────────────────────────────────────────────────────────
- Answers. When SFT-Mix + ERA must answer, the answer changes for 52.8% of its evidence-resolved trajectories and 8.3% of its overlook trajectories, a gap of 44.5 points.
- Before and after ERA training. Qwen3.5-9B is never trained with an entropy reward, and its gap is already 35.1 points (40.7% vs 5.6%). The gap remains at 34.5 points after ERA-RL and 44.5 after SFT-Mix + ERA, so training on the entropy reward does not close it.
- Actions. When free, Qwen3.5-9B never seeks again, and its change rates stay the same (40.9% and 6.2%). The evidence-resolved trajectories of SFT-Mix + ERA seek again after 47.7% of substituted returns, against 11.3% for its overlook trajectories, and their change rate falls from 52.8% to 30.0%.
In all three models, trajectories classified as evidence-resolved are therefore more sensitive to evidence substitution than those classified as overlook. We will add this analysis to Section 4.3.
We hope this resolves the open part of Q1, and we would appreciate any further feedback. Thank you for your time and consideration.
Best regards,
The Authors
We apologize for the late reply: a reviewer's question on overlap led to the frame-budget experiments below, which took longer than expected. We hope you will take another look at our work.
Q1. Independent validation and clarification of the commitment point
Thank you for raising this concern. Our description should clarify the detection procedure and its use in behavioral classification; the classification is then checked by behavior that uses no entropy. A trajectory with one tool call follows:
<think>...</think>blocks, identify the top 20% by entropy across this combined sequence, and calculate their proportion within reasoning spans. Local drops can occur around tool calls; a typical post-return commitment shows a more pronounced drop in the following block, with the rate remaining low afterwards. Equation 1 formalizes this sustained change and detects τ⋆.We will clarify the interaction format, detection procedure, and turn-level classification in the main text.
Q2. Separate interventions, natural retrieval failures, and unfiltered questions
The gain persists under each intervention alone, under natural retrieval failures, and on unfiltered questions.
Table R1. VideoNet-Gen (100 certified questions) and official VideoNet (4,000 unfiltered questions), in %. Mean ± std over training seeds 0, 1, 2 (Qwen3.5-9B, Base in the text: three inference runs). ERA is SFT-Mix + ERA; ERA-RL applies the ERA reward to Base without SFT. RS: share that seeks again. Acc. P / Acc. N: first return substituted, with assertive / plain messages; Acc. P (real): real returns with assertive messages.
The gain is therefore not tied to the combined pressure protocol, and it is largest when a single return is insufficient.