- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes ECR, a training-free framework that uses structured evidence judgments and a fixed policy to retain, revise, or arbitrate a long-video agent’s answer. Experiments show improved accuracy and fewer harmful revisions.
Critical strengths
-
Relevant problem. The paper addresses harmful revisions in self-correction and makes a sensible distinction between supporting a new answer and refuting the original one.
-
Useful empirical results. ECR substantially reduces harmful revisions while maintaining accuracy comparable to unconditional challenger adoption.
-
Informative evaluation. The paper reports fixes and breaks separately and includes replay, ablation, and approximately compute-matched comparisons.
Critical weaknesses
My expertise is outside long-video understanding, so I may be missing some field-specific context. My main concerns are about the methodological contribution and the experimental evidence supporting it.
-
Limited methodological novelty. The method appears implementable as a reusable agent skill combining prompting and fixed decision rules. It is unclear what methodological contribution it offers beyond such a workflow.
-
Insufficient compute-matched baselines. The main gains include additional search and verification. Comparisons with equally resourced AVP search/reflection or generic self-correction are needed to establish ECR’s specific benefit.
-
Limited backbone validation. Most experiments rely on Qwen3-VL-Plus. The 48-question GPT-5.5 experiment is insufficient to establish effectiveness across stronger contemporary multimodal models.
Questions for authors
As weaknesses
4 comments
W2: Comparison with extra search and reflection for AVP
As you note, the full pipeline adds search and verification on top of AVP. To establish what ECR itself contributes, we gave AVP extra search or generic reflection and compared them with ECR on the same initial AVP answers. Neither extra search nor generic reflection improves on AVP, while ECR raises accuracy by 8.59 points at nearly the same added cost (point (1)). The specific benefit of ECR thus lies in how the extra computation is used (point (2)).
(1) AVP with extra search or generic reflection compared with AVP with ECR. Extra search and generic reflection do not reproduce the gain of ECR. We used a newly sampled set of 256 questions, independent of ECR-SCOPE-256, and applied each method to the same fixed initial AVP answers, under the same cap of 15,000 added tokens per question.
| Method (256 questions) | Accuracy | Relative to AVP | Net correction | Added calls / tokens per question |
|---|---|---|---|---|
| AVP | 134/256 (52.34%) | |||
| AVP + extra search | 133/256 (51.95%) | −0.39 points | −1 | 2.64 / 12,390 |
| AVP + generic reflection | 121/256 (47.27%) | −5.08 points | −13 | 2.63 / 12,420 |
| AVP + ECR | 156/256 (60.94%) | +8.59 points | +22 | 2.62 / 12,450 |
Extra search changes accuracy by −0.39 points and generic reflection by −5.08 points, with net corrections (fixes minus breaks) of −1 and −13. ECR gains 8.59 points with a net correction of +22, which is 8.98 points above extra search. Paired tests give p = 0.00047 for ECR against AVP and p = 0.00082 for ECR against extra search. The cost in the last column is what each method adds on top of the frozen AVP run. It is nearly the same for the three methods, about 2.6 model calls and 12.4k tokens per question, so the gain of ECR does not come from a larger budget.
(2) What this shows about the specific benefit of ECR. The gain depends on how the extra computation is used. Searching further and answering again without distinction does not improve the initial answers in this comparison. Placing explicit evidence constraints on the revision of an existing answer keeps a better balance between fixes and breaks.
Revision: Section 4.4 summarizes this comparison, and Appendix I.3 gives the full results with the added cost (Table 24).
W3: Validation with stronger backbones
Thank you for this comment on backbone coverage. The GPT-5.5 experiment in Appendix E.3 was a small check on 48 questions. To test ECR on stronger models more thoroughly, we extended it to 209 questions and then added 100 more. On the 309 questions ECR improves GPT-5.5 by 3.88 points with p = 0.0227 (point (1)), and we also replaced the arbitration model with GPT-5.6 Luna (point (2)).
(1) GPT-5.5 as the backbone on 309 questions. On 309 questions, ECR raises the accuracy of GPT-5.5 from 81.23% to 85.11% (+3.88 points), with 18 fixes and 6 breaks (exact McNemar p = 0.0227). On the first 209 questions the gain is +3.83 points with 12 fixes and 4 breaks (p = 0.0768). To strengthen the test, we added 100 questions that do not overlap these 209 and kept the GPT-5.5 backbone, the ECR rules and the evaluation protocol unchanged. The added questions give +4.00 points with 6 fixes and 2 breaks (p = 0.289), in line with the first 209. Because they were added after the first 209 had been evaluated, the table reports the two parts next to the pooled result.
| GPT-5.5 | First 209 questions | Added 100 questions | All 309 questions |
|---|---|---|---|
| Base correct | 170 | 81 | 251 |
| ECR correct | 178 | 85 | 263 |
| Base accuracy | 81.34% | 81.00% | 81.23% |
| ECR accuracy | 85.17% | 85.00% | 85.11% |
| Gain | +3.83 points | +4.00 points | +3.88 points |
| Fixes | 12 | 6 | 18 |
| Breaks | 4 | 2 | 6 |
| Net correction | +8 | +4 | +12 |
| Exact McNemar p | 0.0768 | 0.289 | 0.0227 |
On the 309 questions, the correction precision is 75.0% and 97.61% of the initially correct answers are kept. Fixes outnumber breaks three to one in both parts, so the balance of fixes and breaks remains favorable on a stronger backbone.
(2) A different model in the arbitration stage. On 151 questions, replacing Qwen3-VL-Plus with GPT-5.6 Luna as the blind verifier raises accuracy from 61.59% to 64.24%. The two results support the applicability of ECR across models, at the level of the backbone and at the level of the arbitration model.
Revision: Appendix E.3 now reports the extension to 309 questions in Table 20, Appendix I.6 reports the verifier replacement, and Section 4.5 refers to both.
Your comments led us to compare ECR with extra search and generic reflection on the same initial answers, and to extend the test on a stronger backbone. The questions you referred to are answered in our replies to W1 to W3. We hope these results clarify what ECR contributes, and we would be glad to discuss any point further.
Thank you for your response, I have updated the rating.
Thank you very much for reading our response and for updating your rating. Your support is very encouraging to us. Your review also helped us improve the paper. The comparisons with extra search and generic reflection, and the larger GPT-5.5 experiment, were both added in response to your comments, and they give clearer evidence of what ECR itself contributes. We will keep these results in the final version.
Thank you again for your time and your careful review!
Thank you for your review and comments! We respond to each point below, with new results on extra search, reflection and a stronger backbone. The changes in the revised paper are marked in blue with a bar in the margin, and each change carries a margin tag naming the corresponding comment. The tags for your comments are purple (for example, P8LR·W1), and those for the other reviewers use other colors.
W1: Methodological contribution beyond a workflow of prompts and fixed rules
Thank you for raising this question. ECR is implemented with prompts and a fixed policy, as you observe, and its contribution lies in how the revision decision is formulated. A symmetric selector asks which candidate is more credible. ECR asks whether the new evidence gives sufficient grounds to overturn the current anchor, which makes revision a directional decision on evidence permission. Point (1) describes what this formulation adds, and point (2) gives the evidence that it matters.
(1) What ECR offers beyond prompting and fixed decision rules. It offers a criterion that determines what the prompts judge and what the rules permit, in three respects. First, support for the challenger (SP) and refutation of the anchor (RA) are judged separately, so that local support for the challenger is not mistaken for grounds to overturn the existing answer. Second, DP checks whether the evidence is sufficient in time, logic and coverage. For example, an event not being observed in a local segment does not mean that it never occurs in the whole video. Third, the certificate records the evidential state and its reason, and the fixed policy maps them to RETAIN, SWITCH or ESCALATE. Every action can thus be traced to a specific evidence relation, which a single preference score would not allow.
(2) Empirical evidence that this criterion makes a difference. The criterion changes the outcome when the form of the workflow is the same. A simple conservative gate and the symmetric selector are also built from prompts and fixed rules. At the same revision rate on 491 questions, ECR obtains 57 fixes and 13 breaks against 52 and 17 for the gate, and is 1.83 points more accurate (exact McNemar p = 0.0352). On the extended matched control of 655 cases, ECR reaches 54.81% against 51.91% for the symmetric selector (exact McNemar p = 0.0271). The core contribution of ECR is therefore an explicit, traceable and comparable criterion for revision decisions, which goes beyond adding one more general-purpose agent skill.
Revision: Contribution (3) in Section 1 now includes the rate-matched control, and Section 4.4 reports both comparisons.