- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes a method for long video understanding in a way that does not require the video at query-time. They do so by a test-time optimization process. For each video, they generate a large set of synthetic QA and next-token prediction samples, which are used to select a subset of informative layers of the video QA model, based on how much they increase performance when using the visual information versus a text-only question. Then, these layers are fine-tuned with LoRA in the query-only setting to distill their behavior when they are exposed to visual information. This way, the LoRA parameters represent the "memory" of the video at hand. The method is validated on the Qwen3 backbone on a few standard long video QA datasets.
Critical strengths
- The paper and proposed method are well motivated.
- The idea of distilling the representation change induced by visual context into the query-only setting is interesting.
Critical weaknesses
A key weakness is the performance and associated computational cost. The initial computation cost is very high, yet the model does not outperform the base model on average on any of the three datasets. If the goal is efficiency, the method should be compared against other methods for more efficient long video QA, rather than only NKR. Still, since the cost for optimization on one video is 70 minutes, as listed, one needs over a hundred questions for a single video for the method to make sense. Moreover, several of the presentation choices in the paper are somewhat misleading. For instance, the main paper lists Video-MME as a main experiment several times, yet the results are buried in the appendix. The computational cost of training for a single video is also in the appendix, despite the cost of 70 minutes per video for the smallest model being a crucial limitation. Similarly, baselines are not bolded when they outperform the proposed method, etc.
Questions for authors
- What are the detailed results on Video-MME (i.e., how do other baselines perform, and how is the performance on short/medium/long videos)?
- If the main goal of the method is lowering computational cost, how does the method compare to other efficient QA methods, such as adaptive token methods or even zero-shot ones like video panels? How many questions does one need to ask before the current method becomes worth it for different video lengths, and is this realistic?
- What is the reason for the budget-averaged memory selection, i.e., why is it important that lower lambdas already produce better performance? What is the difference in performance to simply using lambda=1? Some other ablations are also missing, for instance, the value of alpha.
- Does the method work for other models than Qwen3?
- Why are only the local QA samples used for layer selection?