- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes HIPPOSTREAM, a fixed-size bank of token-shaped states for streaming video question answering. Each one-second chunk produces one frame and one reconstruction-gradient update to the bank, while first-order meta-training learns its initialization and write/read interfaces with Qwen3-VL frozen. Evaluations on OVO-Bench and VSI-SUPER-Recall show gains on coarse backward-summary tasks, alongside losses on detailed episodic recall and several current-frame tasks.
Critical strengths
-
Updating a small token bank by gradients provides a distinct fixed-state alternative to caches and retrieval stores whose storage grows with the stream. The persistent state remains , and the cost of each WRITE does not depend on how much earlier video has already been processed.
-
Table 14 reports a 6.76-point improvement on Action-State Inference over the no-memory baseline. This is concrete evidence that four persistent tokens can retain useful coarse information about earlier events.
Critical weaknesses
-
The WRITE objective does not directly establish that the four-token bank preserves evidence needed by future questions. Equations (1)–(2) repeatedly move the bank toward a pooled or first-position representation of each chunk, and first-order MAML prevents the answer loss from directly training . The target ablations compare compression heuristics, but do not show whether answer-critical details survive repeated overwriting.
-
The empirical gains are concentrated on coarse summary memory rather than detailed episodic recall. Table 14 reports on ASI but on EPM, while every HLD answer is Unable to answer and the corrected-prompt gain in Table 13 is only . The gradient bank also falls from 56.7 at 10 minutes to 31.7, 28.3, and 15.0 at 30, 60, and 120 minutes in Table 16, so the broader claim of reliable access to old events is not yet supported.
-
The properly meta-trained memory does not provide a meaningful aggregate gain in Table 9. The no-memory baseline scores 58.52, compared with 58.28, 58.24, and 58.53 for the first-position, mean-pooled, and attention-pooled variants. These results support a redistribution from real-time and detailed-recall tasks toward coarse backward-summary tasks, while the headline shape-skip rows use random slow components and therefore cannot establish that offline meta-learning is responsible.
-
Constant state size does not imply low inference cost. Every one-second chunk requires a frozen-LLM forward and backward pass, and Appendix G reports 5–8 hours for a full memory-on evaluation and up to approximately 30 minutes for one Forward query. Without matched no-memory latency, streaming throughput, peak GPU memory, and a forward-only recurrent or bounded-memory baseline, it is difficult to assess whether the storage saving justifies the additional compute.
Questions for authors
-
It would be helpful to evaluate whether the bank preserves answer-critical details using multiple questions per video, including questions about low-salience past events. Comparing the current reconstruction-trained writer with one that receives answer-loss gradients would further clarify whether the WRITE objective retains more than dominant chunk content.
-
Please report a full-1640 comparison under one corrected prompt and scorer between random and properly meta-trained slow components, with separate ASI, EPM, HLD, real-time, and overall results. This would show whether MAML adds value beyond the fast gradient update itself.
-
A matched systems comparison would make the fixed-memory trade-off clearer. Reporting latency per chunk, sustainable streaming FPS, query latency, throughput, and peak GPU memory for no memory, HIPPOSTREAM, a forward-only recurrent memory, and a bounded retrieval baseline would be particularly informative.