- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HTSC-AT for efficient long-video understanding with lightweight MLLMs. The framework has three stages: adaptive visual tokenization based on motion-semantic change, event-aware temporal compression, and sparse-dense attention over the compressed event representation. The paper evaluates the proposed method on several long-video benchmarks and shows promising results.
Critical strengths
-
The paper considers token reduction at several levels of the long video pipeline. The overall design is coherent: adaptive visual tokenization, event-level compression, and structured sparse attention are meant to address different sources of redundancy.
-
The experiments are extensive. The paper evaluates on multiple long-video benchmarks and includes separate analyses for token allocation, temporal compression, and sparse attention, together with runtime and memory measurements
Critical weaknesses
-
In Sec. 2.2.2, the paper determines how many tokens to keep for each frame, but does not explain how these tokens are actually selected or constructed. In Sec. 2.3.1, the ActionFormer branch is described as operating on 30-token windows over ( contains a variable number of spatial tokens per frame). It is therefore unclear whether ActionFormer predicts boundaries at the spatial-token level or on some frame-level representation. The semantic discontinuity branch also computes similarity between adjacent tokens in $T^{(1)}$. Two different regions within the same frame could produce a large mutation score despite there being no temporal boundary. The paper should clarify the representation used by both branches.
-
Several important design claims are not supported by the ablations. There is no CLIP-only and optical-flow-only comparison (Eq. 2) for the claimed dual-stream design, no isolation of the two TEBD boundary branches, and no ablation of the proposed TPE. The sparse-connectivity comparison is also not degree-matched: the random baseline samples three events per query event, whereas the proposed design uses six anchors. Why not compare against six randomly sampled events?
Questions for authors
- Since efficiency is a key claim, I would also like to see TTFT and end-to-end latency compared with existing long-video MLLMs under the same hardware and input budget (e.g., a cap of 1024 frames), rather than only against the uniform-token baseline. The three-stage compression pipeline seems likely to add some preprocessing overhead.
- After is computed in Eq. (3), how are the actual $N_t$ spatial tokens obtained from the original CLIP token grid?
- How are overlapping ActionFormer predictions combined for window size 30 and stride 10?
- Can the authors provide the missing component-level ablations to demonstrate the necessarity of each part?
- Can the authors add a degree-matched random baseline, e.g. random six anchors with the same adjacent-event connections?
- Minor formatting issue: "Results." on line 368 is isolated on a separate line.