- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents VimPro, a proxy-guided reasoning framework for long-video question answering. It constructs episodic memory at multiple temporal scales offline and maintains question-specific evidence memory and working memory during inference. Its core component, MemLens, is based on Qwen3-VL-4B and trained through cold-start supervised fine-tuning followed by answer-gated GRPO reinforcement learning to select answer-supporting evidence from coarsely retrieved candidates. The answer agent reasons over the current memory and invokes textual retrieval or localized visual inspection tools as needed to gather additional evidence. The authors report strong accuracy on Video-MME, LongVideoBench, and LVBench. Ablations on memory components, MemLens, multi-turn reasoning, and training stages support the effectiveness of the overall approach. They also compare runtime and token consumption against two memory-based agents.
Critical strengths
1.The paper addresses the gap between semantic relevance and usefulness for answering by assigning fine-grained evidence selection to a lightweight, specifically trained model. This division of responsibilities helps reduce irrelevant or misleading context passed to the answer agent. With the candidate set and answer agent held fixed, adding MemLens improves accuracy on Video-MME Long from 76.6% to 81.0%, providing direct evidence for the value of this component. This is my main reason for supporting acceptance. 2.VimPro achieves strong results across three long-video QA benchmarks, and ablations on its components and training stages support the overall design. Comparisons with M3-Agent and WorldMM show potential reductions in runtime and token consumption. Combining lightweight evidence selection with a strong answer model offers a useful approach to building scalable long-video QA systems.
Critical weaknesses
1.VimPro relies on offline episodic memory and coarse retrieval to obtain evidence. Although visual inspection can recover details omitted from textual summaries, it requires the relevant time span to have already been located. If missing information prevents coarse retrieval from finding the supporting segment, subsequent reasoning may still lack the necessary evidence. The system’s performance is therefore constrained by the quality of the offline memory and retrieval coverage. 2.The paper presents answer gating and fine-grained rewards as important elements of MemLens training. However, the ablations mainly compare Base, SFT, RL, and SFT+RL, without comparing the full reward scheme against versions that remove answer gating or use simpler rewards. These results support the effectiveness of two-stage training, but leave unclear how much additional benefit the proposed reward design provides beyond reinforcement learning more generally. This makes the core training contribution difficult to assess. 3.The main accuracy results use GPT-5, whereas the efficiency experiments in Table 2 use a locally deployed Qwen3-32B without reporting each method’s accuracy under that configuration. It is therefore unclear whether the reported efficiency gains come with a loss in accuracy. The results do not fully support the claim that VimPro achieves both high accuracy and low computational cost.
Questions for authors
1.Visual inspection can recover missing details, but still relies on textual memory to identify the relevant time spans. The limitations section acknowledges that failures in coarse retrieval can make a question unanswerable, but does not quantify their impact. Could the authors provide an error analysis showing how many incorrect answers result from omissions in the offline memory or failures to retrieve supporting segments, to assess how much this bottleneck limits overall performance? 2.Table 6 demonstrates the benefits of SFT and RL, but does not isolate the contribution of answer gating. Could the authors compare the full reward scheme with a version without answer gating, using the same training data and budget, to establish whether this design provides an additional performance gain? 3.Could the authors report the accuracy of VimPro and the baselines under the configuration used in Table 2, and compare accuracy, runtime, and token consumption using the same answer model configuration, to determine whether the efficiency gains come at the expense of accuracy?
4 comments
Q3: The main accuracy results use GPT-5, whereas the efficiency experiments in Table 2 use a locally deployed Qwen3-32B without reporting each method’s accuracy under that configuration. It is therefore unclear whether the reported efficiency gains come with a loss in accuracy. The results do not fully support the claim that VimPro achieves both high accuracy and low computational cost.
A3: Thank you for raising this concern. We agree that the original efficiency study, without accuracy measured under the same setting, was insufficient to establish the accuracy–cost trade-off.
First, the original efficiency experiment used Qwen3-32B because M3-Agent relies on a fine-tuned checkpoint of this model. Using the same Qwen3-32B for the other methods was intended to reduce model-related differences when comparing computational costs.
Second, as reviewer nvPC noted, Qwen3-32B is a text-only model. To jointly evaluate accuracy and efficiency with multimodal answer models, we conducted additional comparisons with WorldMM using Qwen3.5-27B and GPT-5.
To address the accuracy–cost concern directly, we have expanded the evaluation from 10 videos to 30 randomly sampled videos from Video-MME Long, averaging 41.5 minutes and covering 90 questions.
| Method | Answer Model | Accuracy (%) ↑ | End-to-End Time (min) ↓ | Token Consumption (M) ↓ |
|---|---|---|---|---|
| WorldMM | Qwen3.5-27B | 64.4 | 1748 | 58.2 |
| VimPro | Qwen3.5-27B | 77.8 | 734 | 17.1 |
| WorldMM | GPT-5 | 68.9 | 1547 | 59.5 |
| VimPro | GPT-5 | 82.2 | 614 | 17.6 |
Accuracy is measured over all 90 questions. Time and token consumption are aggregated over the complete evaluation, including video processing, memory construction, evidence retrieval, and final answering.
As shown in the table, VimPro consistently outperforms WorldMM in both accuracy and efficiency. Under both Qwen3.5-27B and GPT-5, VimPro achieves an accuracy improvement of approximately 13.3 percentage points. Meanwhile, it reduces end-to-end runtime by 58.0% and 60.3%, and token consumption by 70.6% and 70.4%, respectively. These consistent improvements demonstrate a favorable accuracy–efficiency trade-off for VimPro under both answer-model configurations.
Q4: Visual inspection can recover missing details, but still relies on textual memory to identify the relevant time spans. The limitations section acknowledges that failures in coarse retrieval can make a question unanswerable, but does not quantify their impact. Could the authors provide an error analysis showing how many incorrect answers result from omissions in the offline memory or failures to retrieve supporting segments, to assess how much this bottleneck limits overall performance?
A4: Thank you for this suggestion. We analyzed all 171 incorrect answers on Video-MME Long using teacher-assigned evidence utility scores, treating items with u ≥ 50 as supporting evidence.
We distinguish three cases: supporting evidence is absent from the candidate pool, present but not passed to the answer model, or passed to the answer model but followed by an incorrect answer.
| Error category | Percentage |
|---|---|
| Memory/retrieval miss | 66.7% |
| Evidence-selection miss | 9.9% |
| Answering failure | 23.4% |
The largest category involves missing supporting evidence in the candidate pool, indicating that memory quality and retrieval coverage remain a major bottleneck. However, this analysis cannot distinguish information omitted during memory construction from supporting memories that exist but were not retrieved. The remaining errors involve evidence selection or incorrect answering despite receiving supporting evidence.
These categories are based on teacher-estimated utility rather than definitive causal attribution. We will include this analysis in the revised paper and clarify that MemLens improves selection among retrieved candidates but cannot guarantee recovery of evidence absent from the candidate pool.
Q5: Table 6 demonstrates the benefits of SFT and RL, but does not isolate the contribution of answer gating. Could the authors compare the full reward scheme with a version without answer gating, using the same training data and budget, to establish whether this design provides an additional performance gain?
A5: Thank you for the suggestion. We have added an answer-gate ablation using the same training data and budget, as detailed in A2.
Q6: Could the authors report the accuracy of VimPro and the baselines under the configuration used in Table 2, and compare accuracy, runtime, and token consumption using the same answer model configuration, to determine whether the efficiency gains come at the expense of accuracy?
A6: Thank you for the suggestion. We have added joint accuracy, runtime, and token-consumption comparisons under the same answer-model configuration, as detailed in A3.
Thank you for your response. After considering your points, I have decided to keep my original score.
We genuinely appreciate your recognition of our contributions and your positive feedback. Thank you for the time you invested in reviewing our paper and for your insightful questions. In our response, we will address each question individually, quoting them and providing our answers accordingly.
A1: Thank you for highlighting this limitation. We agree that VimPro depends on the quality of its offline episodic memory and the coverage of coarse retrieval. MemLens can select useful evidence from the retrieved candidates, but cannot recover supporting segments that never enter the candidate set.
VimPro mitigates this bottleneck through multi-scale episodic memory and iterative retrieval. Memories at different temporal scales provide complementary retrieval cues, while the answer agent can issue supplementary queries based on its evolving reasoning state. Once a relevant span is located, visual inspection can recover details omitted or inaccurately represented in the textual memory. However, these mechanisms do not guarantee recovery when the supporting span remains unlocated.
This limitation is acknowledged in Appendix F. We will clarify it in the main text and distinguish evidence-selection improvements from retrieval-coverage limitations, avoiding any implication that visual inspection can compensate for all failures of offline memory or coarse retrieval.
A2: Thank you for raising this point. We ablate the composite GRPO reward by removing one component at a time and retraining MemLens, keeping all other training settings unchanged. All variants are evaluated under identical inference settings. We keep Rformat fixed as a structural-validity constraint and focus our ablation on the components directly related to evidence selection and downstream answerability. Specifically, we examine: (i) w/o Revi, removing the signed evidence-ranking reward; (ii) w/o Rans, removing the auxiliary-answer reward; (iii) w/o Rcalib, removing confidence calibration; and (iv) w/o answer gate, disabling answer-gated reward modulation by setting g=1 while leaving the other reward components unchanged.
Table: Ablation on the GRPO reward components. “✓” indicates that the component is used; each row except the last removes one component from the full objective.
Each component contributes to the overall performance. The evidence-ranking reward Revi and auxiliary-answer reward Rans yield the largest improvements (6.7 and 3.5 points), as they encourage MemLens to prioritize answer-supporting evidence and distinguish among candidate options. Meanwhile, the answer gate (2.8 points) promotes evidence selection aligned with downstream answerability, while Rcalib (1.4 points) encourages reliable confidence estimation. By integrating these complementary signals, the full objective achieves the highest accuracy of 81.0% on Video-MME Long.