- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DynFrame makes both the temporal window and the sampling rate part of a video model’s generated retrieval command. Starting from uniformly sampled frames, the model requests one additional clip, receives its visual embeddings, and continues reasoning. Training combines supervised retrieval-and-reasoning trajectories with SD-GRPO, which adds a separately normalized retrieval advantage before frame injection while retaining answer-derived feedback throughout. Evaluation covers six benchmarks for temporal grounding, grounded VideoQA, and long-video understanding.
Critical strengths
-
Learning the temporal window and sampling density together is a useful extension of video retrieval. The model can request different observation granularities through a compact interface within its reasoning trajectory. The dynamic-density ablation in Table 2d improves over both tested fixed rates, providing evidence that this design helps answer questions requiring different amounts of temporal detail.
-
The temporal grounding results are convincing relative to the backbone. On ActivityNet-MR, DynFrame-8B obtains 51.5 mIoU compared with 36.2 for Qwen3-VL-8B (Table 1). The evaluation covers several task families, and the sparse-observation experiment shows that the framework can retain performance when the initial video pass contains less evidence. These results support the practical usefulness of retrieving additional frames during reasoning.
Critical weaknesses
-
I am not yet convinced that the RL gains establish the benefit of segment-specific credit assignment, which is a central contribution. Table 2b shows that SD-GRPO improves over vanilla GRPO, but the baseline’s exact training reward is not stated clearly enough to establish that it receives the same retrieval supervision. Figure 4 evaluates both methods with the same diagnostic reward definitions; that does not establish that those rewards were used identically during optimization. The missing control is to retain the same separately normalized sampling and answer advantages, apply their sum throughout the generated response, and then change only the segment assignment. Until this is tested, the improvement could come from explicit retrieval feedback rather than the proposed credit-assignment mechanism.
-
The benefit of learned sampling density is not established at comparable visual cost. Table 2d supports higher accuracy than the tested fixed rates, but changing FPS also changes the number of retrieved frames. Average frame counts, visual-token counts, and end-to-end latency are not reported, so I cannot assess whether the policy allocates evidence better for a given budget. This matters because reducing repeated retrieval and context growth is part of the motivation, while the implementation rebuilds the KV cache by prefilling the full extended context after retrieval. A comparison against fixed density with a comparable average visual budget would help establish the value of learning density rather than simply choosing a different amount of visual input.
Questions for authors
-
What exact reward does vanilla GRPO optimize in Table 2b? Does its training reward include the same span and FPS feedback as SD-GRPO?
-
Does SD-GRPO still improve when the sum of the same separately normalized sampling and answer advantages is applied throughout the generated response, with the other training settings held fixed?
-
What are the average retrieved frame counts, visual-token counts, and inference latency for dynamic versus fixed density? Does the accuracy advantage persist when their average visual budgets are comparable, and how often does the retrieval frame cap activate?
I am open to increasing my score if the authors can provide convincing responses to the above questions and sufficiently address my concerns.
2 comments
I thank the authors for their response and have decided to maintain my original score.
## Response to Reviewer Pkzb
Thank you for specifying the controls needed to strengthen the attribution and for your willingness to consider raising your score.
W1/Q1/Q2 — Reward definition and credit assignment. Vanilla GRPO in Table 2(b) uses the answer reward only, without span/FPS feedback. We agree that this baseline alone does not isolate credit assignment. The normalization-matched control retains the same separately normalized sampling and answer-plus-format advantages and applies their sum to every generated token.
We conducted an additional GRPO experiment using Ranswer+Rsample with a shared advantage. As shown in the updated Figure 4, its final answer reward is lower than that of GRPO trained with Ranswer alone. Its sampling reward improves overall and approaches that of SD-GRPO, whereas its answer reward initially rises and then declines. This suggests that applying sampling feedback to post-injection reasoning may hinder answer learning, even as sampling reward improves.
We further compare these variants on NExT-GQA, Charades-STA, MLVU and LVBench. SD-GRPO consistently outperforms both controls across all four benchmarks, whereas the shared-advantage variant does not improve upon answer-only GRPO. Together with the normalization-matched control, these results support the importance of segment-specific credit assignment beyond adding sampling feedback alone in our setting.
W2/Q3 — Visual budget and inference cost. In our training implementation, the coarse observation and retrieved clip are each capped at 128 frames, giving at most 256 sampled frames per trajectory. The visual-token count is computed from the sampled frame count and resized resolution according to Qwen3-VL's spatial and temporal merging scheme. Our updated cap statistics show that only 0.1% / 0.9% / 1.6% of retrievals exceed the cap on NExT-GQA, Video-MME and MLVU, respectively. So cap-induced changes to sampling density affect only a small minority of retrievals.
To assess inference latency under different FPS settings, we first analyze DynFrame's predicted retrieval rates across all evaluated benchmarks, obtaining a mean FPS of 2.6. Based on this distribution, we compare the Qwen3-VL backbone and DynFrame with fixed 2 and 3 FPS. As shown in the table, DynFrame has lower reported inference latency than the backbone on NExT-GQA and LVBench, indicating that DynFrame can retrieve clearer visual evidence, which enables more focused and efficient reasoning. Compared with fixed 2 and 3 FPS, dynamic FPS keeps inference latency on the same order of magnitude while improving performance on all four benchmarks (Table 2(d)).
Moreover, we compare DynFrame with other retrieval-based models on the long-video understanding benchmarks LVBench and Video-MME, running each baseline with the inference settings in its official scripts. As shown in the table below (Table 7 in Appendix B), DynFrame-8B has the lowest inference latency on both benchmarks and processes 5.3–6.3× fewer tokens than the other methods, including 6.3× fewer than VideoChat-R1.5. Visual revisits and token counts are averaged over the two benchmarks.