- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The empirical evidence is broad. Leading results on six benchmarks, gains with two unrelated open backbones, and ablations of parallel spawning, task objectives, inspection modes, and multimodal aggregation all point in the same direction. The same-backbone efficiency comparison with VideoSeek is particularly useful, although it needs stronger protocol reporting.
Critical strengths
The empirical evidence is broad. Leading results on six benchmarks, gains with two unrelated open backbones, and ablations of parallel spawning, task objectives, inspection modes, and multimodal aggregation all point in the same direction. The same-backbone efficiency comparison with VideoSeek is particularly useful, although it needs stronger protocol reporting.
Critical weaknesses
The main comparative evidence still does not fully isolate the framework’s contribution. Videopus uses GPT-5 as both Orchestrator and Executor and can invoke multiple models per query, but most headline tables compare its results with heterogeneous methods using different models, sampling protocols, and numbers of inference calls. A direct GPT-5 end-to-end baseline is absent on most benchmarks, and it is unclear whether VideoSeek and every other baseline were rerun with identical subtitles, visual-unit budgets, prompts, and answer parsing. Because the paper claims consistent superiority over state-of-the-art methods, these controls are essential.
The efficiency and robustness claims also remain under-specified. Token and model-call accounting does not explain whether all parallel Executor calls, image tokens, retries, and failed sessions are included.
Questions for authors
-
Please provide no-tool GPT-5 results on all six benchmarks under the same global sampling, subtitle policy, prompts, and answer parser. Also specify exactly which benchmarks used subtitles or ASR transcripts.
-
Were VideoSeek and every rerun baseline given the same base model, visual-unit budget, executor concurrency, inference-round limit, and retry policy? Please report matched-budget results rather than only best reported scores.
-
Please disaggregate average tokens and model calls into Orchestrator, Executor, image, input, output, and retry costs.
4 comments
Response to Reviewer 9diz [2/2]
Specifically, we evaluate VideoSeek and Videopus on LVBench-Abl with the same 100K-token budget target and 84K-token pre-call threshold. Token accounting includes text and image inputs and outputs from all model calls, including the Orchestrator and Executors.
| LVBench-Abl | Backbone | Token Budget | ACC↑ | Actual Tokens (K)↓ |
|---|---|---|---|---|
| VideoSeek | GPT-5 | 100K target / 84K pre-call check | 62.3 | 67.7 |
| Videopus | GPT-5 | 100K target / 84K pre-call check | 74.2 | 65.2 |
Both methods follow the same budget policy, while actual token consumption may differ due to early stopping and evidence acquisition efficiency. We account for actual image tokens and all model calls rather than treating equal visual-unit counts as equivalent computational budgets.
Under the unified GPT-5 setting, Videopus outperforms ParaVT, VideoSeek, and VideoARM by 13.4, 9.9, and 5.2 pp, respectively. Under matched token budgets, it maintains an 11.9 pp advantage over VideoSeek while using 2.5K fewer tokens on average. These results show that Videopus's gains persist under both backbone-controlled and budget-matched comparisons, supporting the effectiveness of its evidence acquisition and aggregation mechanisms.
W2&Q3. Breaking down all calls, tokens, and retries by role.
Section 4.4 reports average tokens and model calls on full LVBench, while Appendix F analyzes repeated context-processing costs. We further provide a request-level breakdown of text input, image input, output tokens, and model calls, covering all parallel Executors in Videopus and the corresponding components in VideoSeek.
| LVBench-Ful | Component | Text Input (K) | Image Input (K) | Output (K) | Total Tokens (K) | Model Calls / QA ↓ |
|---|---|---|---|---|---|---|
| VideoSeek | Main Agent | 65.2 | 0.0 | 13.5 | 78.6 | 5.5 |
| Action Parser | 7.7 | 0.0 | 2.7 | 10.3 | 5.5 | |
| Answer Tool | 11.5 | 0.0 | 0.2 | 11.7 | 1.0 | |
| Overview&Inspection Tools | 1.6 | 40.0 | 9.6 | 51.2 | 4.8 | |
| Total | 86.0 | 40.0 | 25.9 | 152.0 | 16.9 | |
| Videopus | Orchestrator | 15.0 | 18.2 | 4.3 | 37.6 | 2.8 |
| Executors | 5.5 | 21.8 | 19.3 | 46.6 | 3.9 | |
| Total | 20.5 | 40.1 | 23.6 | 84.2 | 6.8 |
All values are per-question averages. Minor discrepancies in component sums arise from rounding. Videopus reduces model calls by 59.8% and total tokens by 44.6%, including all 46.6K tokens consumed by parallel Executors. While image input remains nearly identical (40.0K vs. 40.1K), text input decreases from 86.0K to 20.5K (76.2%). These results show that the token savings primarily stem from fewer global reasoning rounds and less repeated context processing, rather than reduced visual input.
Regarding retries, the Videopus Harness handles non-fatal intermediate errors and reports them to the Orchestrator without restarting execution. Only execution-blocking failures, such as empty Orchestrator outputs, missing required Evidence fields in Executor reports, or tool execution failures, trigger local task restarts or full trajectory reruns. No such retries occurred in the full LVBench logs, resulting in zero additional model calls or token costs.
Additional Revisions and Experiments
Beyond the concerns addressed above, we have further clarified benchmark protocols in Appendix B, baseline configurations and result sources in Appendix C, and distinctions from related video agents in Appendix D. We have also expanded Appendix E with additional controlled comparisons, full LVBench ablations, inspection-policy analyses, hyperparameter sensitivity studies, and cross-backbone generalization. Appendix I further presents systematic failure and recovery analyses with additional inference trajectories.
We hope these revisions provide further evidence for assessing the effectiveness, efficiency, and generalization of Videopus. We appreciate your thoughtful feedback and consideration.
Thank you for the detailed rebuttal and additional experiments. Please make sure these results and protocol clarifications are clearly integrated into the main text as well as the appendix. I have raised my score to 3.
We sincerely appreciate your careful reassessment and for raising your score to 3. We are encouraged by your recognition of our additional experiments and clarifications. We will ensure that the key results and protocol details are clearly integrated into the main text, with further details provided in the appendix. Thank you again for your constructive feedback, which has helped us improve the manuscript.
Response to Reviewer 9diz [1/2]
Thank you for your valuable feedback, which has helped us further improve our work. We have revised the manuscript accordingly and added the requested material to the appendix. -Abl denotes our ablation subset; -Ful or no suffix denotes the full dataset.
To isolate the contribution of Videopus from the backbone model, we evaluate GPT-5 (No Tools) on all six benchmarks. Both settings use the same GPT-5 version, initial global visual sampling, question prompts, and answer parsing rules. No-tool GPT-5 answers directly from the initial visual observations, whereas Videopus performs targeted inspections and iterative evidence aggregation.
Regarding input policy, none of our experiments use ASR. Only LongVideoBench uses dataset-provided subtitles following its evaluation protocol. LVBench, EgoSchema, MLVU, Video-MME, and Video-Holmes use neither subtitles nor additional video captions. The new no-tool controls follow the same policy, including identical subtitle inputs for LongVideoBench.
Videopus consistently outperforms no-tool GPT-5 across all six benchmarks by 1.8-12.4 percentage points. On LVBench, Summarization and Temporal Grounding improve by 20.7 and 20.9 points, respectively. These results demonstrate the value of targeted evidence acquisition through Parallel Task Spawn and source-grounded visual aggregation through Multimodal Evidence Aggregation, which help recover information missed by global sampling and support cross-interval reasoning.
Like previous works, our main comparisons use published results to preserve each method's original model and inference settings, avoiding potential performance degradation from reimplementation. For example, VideoARM evaluated GPT-5 but achieved better performance with OpenAI o3 + GPT-4o. Thus, the main tables report performance under each method's original configuration rather than controlled comparisons. We have clarified the backbone models, additional input modalities, and result sources in Appendix C of the revised manuscript.
The paper already compares Videopus with VideoSeek using the same GPT-5 backbone, showing a 7.2 pp accuracy improvement and 44.6% fewer tokens on full LVBench. To further control for backbone differences, we rerun ParaVT, VideoSeek, and VideoARM with GPT-5. Their inference frameworks are adapted to GPT-5 where necessary, with ASR disabled for VideoARM to match our visual-only setting.
These reruns use the same GPT-5 version, evaluation samples, input modalities, answer parsing, and retry policy, while preserving each method's agent instructions, tool sampling, and task planning. Since visual-unit definitions, call structures, concurrency, and round limits differ across frameworks, we retain their original mechanisms and additionally compare VideoSeek and Videopus under matched total-token budgets.