- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Summary This paper proposes GOPAgen which brings codec GOPs and motion vectors into agentic long video question answering. The system first checks global-frame sufficiency then uses query-conditioned two-stage temporal selection to locate relevant intervals. A codec-aware Motion Agent and a Caption Agent build typed memory pages and a GOP-Tree organizes ranks and compresses the evidence for final reasoning. It reports competitive results on MotionBench EgoSchema LongVideoBench and MLVU with lower online latency and includes fairly complete ablations on motion vectors tree structure optical flow window selection latency and failure modes. Contributions It uses codec motion vectors rather than optical flow as a complementary motion cue and converts them into retrievable text evidence through a Motion Agent. It builds an agentic pipeline with two-stage temporal selection and global-sufficiency gating that stays competitive at lower online latency. It provides No MV Reference-frame only and Optical Flow ablations that support the role of motion vectors. Validation The direction is valuable but the evidence is not yet sufficient. The compared methods are outdated and the main tables center on GPT-4o and Gemini-1.5/2.0 which cannot support a state-of-the-art claim. The figures are rough and do not match the two-stage temporal selection in the text. Candidate counts do not align with the Appendix B.3 description the GOP-Tree efficiency attribution does not hold and misses from sparse global sampling cannot be recovered. Most leads are within 2 pp and no significance is reported. Overall it is a borderline paper that a substantive rebuttal could change.
Critical strengths
- The entry point is reasonable. The paper does not equate codec motion vectors with optical flow but treats them as encoder prediction cues that need RGB reference frames for semantic interpretation.
- The usefulness of motion vectors is backed by ablations. No MV drops 4.5 pp on MotionBench relative to Full GOPAgen and Reference-frame only is also lower.
- Experimental coverage is fairly complete including backbone swaps component removal optical-flow and codec comparisons parameter sensitivity retrieval diagnostics and failure classification.
- The limitations are stated with restraint acknowledging that fewer visual tokens do not scale proportionally with FLOPs or total cost and that the method relies on several models and heuristic search.
Critical weaknesses
- The compared methods are clearly outdated. The main tables still center on GPT-4o and Gemini-1.5/2.0 which were no longer frontier at submission time and cannot support a state-of-the-art claim. A comparison against strong baselines already published by the deadline is needed.
- The figures are rough and do not match the text. The concept and pipeline diagrams are unclear and Figure 2 shows Coarse Fine and Super-Fine levels while the text and Appendix B.3 implement only "two rounds with a fixed search depth". The figures should be reconciled with the text and should show the global-sufficiency gate branch.
- The core motivation is not closed and the prompt is self-contradictory. The motivation states that sparse sampling can "miss a short-lived event" yet the Motion Agent only processes already-selected intervals and global inspection uses at most 32 frames for a 60-minute video so a short action missed at the global stage fails the whole chain. Evidence recall should be reported by action duration. In addition the text says the Motion Agent outputs "a fine-grained motion description, not a separate answer" while the prompt asks to adopt its "reference answer" which the appendix admits does not exist as a field.
Questions for authors
- The comparison set lags the field. The newest listed methods are from 2025 with no 2026 work, and the closed-source baselines are especially dated (GPT-4o, Gemini-1.5/2.0), which is too weak to support the state-of-the-art claim. Please add comparisons against strong models already published by the deadline, e.g. Qwen 3.8 27B (open) and GPT-5.6 (closed).
- Please define the generated/scored/retained candidate counts at each stage and give one full retrieval trace, explaining why the 4.8 and 6.2 in Tables 9/10 are far below the ~24 implied by B.3. Also, under the same selection and gating, the Flat-memory variant is actually faster (63.7 vs 65.0s), so where does the GOP-Tree's independent efficiency gain come from?
- When a key short action is absent from the 32 global frames, how does the system select the right interval? Can you report coarse/fine/evidence recall by action duration? Also, the text says the Motion Agent only outputs a motion description, yet the prompt asks the system to adopt its "reference answer," which does not exist. How is this reconciled?
- How do Figure 2's three levels (Coarse/Fine/Super-Fine) map onto the "two rounds with a fixed search depth" in the text? Is Super-Fine just the Motion Agent's internal GOP granularity rather than a separate selection stage?
- As frontier models rapidly improve, both closed (e.g., 6astra) and open (e.g., Qwen3.8-27B) models offer ever stronger native long-context video understanding. Is the video reasoning task this paper targets still a real problem? More importantly, relative to simply plugging such strong models into a general-purpose base harness (e.g., Codex), where does your method's value and irreplaceability lie? Please provide a direct comparison against a "base harness + frontier model" setup.