- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
ResAdapt predicts query-conditioned per-frame resolutions before video encoding. A lightweight Allocator is trained with CAPO using task rewards and cost shaping, while the standard variant freezes the downstream model. Experiments show reduced token usage and long-video latency, with zero-shot transfer to other backbones.
Critical strengths
- Pre-encoding resolution allocation reduces both visual encoding and downstream token costs while preserving sampled time points.
- At 128 frames, ResAdapt achieves a six-benchmark mean of 56.28.
Critical weaknesses
-
Training cost limits the practical efficiency assessment. Training uses 32 H100 GPUs and 16 allocation–response rollouts per input over 93.4K examples, but total training time/GPU-hours and amortization against inference savings are not reported. Moreover, inference savings are regime-dependent: Table 10 shows slowdowns for short clips at conservative retention.
-
The central design choices lack decisive controls.
- The direct-cost baseline uses only a reported penalty coefficient of 0.5 (Appendix E.3.3). Its collapse may reflect excessive cost pressure rather than the necessity of CAPO.
- The headline 128-frame result at 22.9% retention and the 16× sampling experiment lack a same-frame, budget-matched uniform-resizing baseline. This leaves gains from denser temporal coverage insufficiently separated from learned resolution allocation.
- CAPO penalizes mean scale rather than realized token cost, which also depends on scale variance. The benefit of this proxy over actual retention or mean squared scale is not established.
Questions for authors
- What are total training GPU-hours, including tuning, and how many inference requests are needed to amortize this cost?
- Does CAPO outperform direct-cost training after sweeping the penalty coefficient to reach a comparable training-time budget?
- How does uniform resizing of the same 128 frames compare at matched realized retention, particularly for the headline result and the 8-frame-equivalent experiment? Please report accuracy and end-to-end cost.
- Does CAPO retain its advantage when cost shaping uses realized token retention or mean squared scale instead of mean scale?
Dear Reviewer fZxb,
Thank you again for your constructive feedback. As the discussion period draws to a close, we would greatly appreciate another look at the training-cost estimate, the new 128-frame uniform-resizing controls, and our explanations of CAPO’s training behavior and cost proxy.
If these revisions strengthen your assessment of the method and its practical value, we would be sincerely grateful if you would consider increasing your rating by one point. We welcome any remaining questions and appreciate your careful consideration.
Best regards, The authors