- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper works towards efficient video understanding by adapting image-pretrained Vision Transformers with SSMs to avoid global spatiotemporal attention while seemingly maintaining the training memory requirements by proposing Parallel Kinematic Selective State Space Scanners (PKS⁴), a framework that applies spatial parallelization over the inter-frame correlation obtained from the Kinematic Prior Encoder (KPE). The proposed methods seem to show competitive results against selected baselines while claiming to improve training efficiency by reducing the compute requirement by up to 10.7x compared to the VideoMamba-M800e configuration.
Critical strengths
-
The proposed framework is relatively simple and practical for introducing temporal modelling into an image-pretrained model without excessive modification while maintaining linear-complexity temporal modeling.
-
The reported results are competitive, and the ablations on individual components explain the design choices for the KPE and KS⁴ components.
Critical weaknesses
-
The technical novelty of the proposed framework is not clearly established against the current literature. Independently processing patches with spatially parallelized temporal SSMs has already been explored by many existing works, such as TV3S (CVPR 2025) for video segmentation and STAC (ECCV 2026) for video reasoning segmentation. While MoMa (ICML 2025) has explored the integration of Selective SSMs into image-pretrained models for video recognition, and MotionSqueeze (ECCV 2020) has investigated extracting motion features through local inter-frame correspondences. With most of the components well covered by the literature, it is unclear what the paper produces as a contribution other than their combination, nor is it clear why the proposed combination is effective.
-
The motivation and effectiveness of the proposed KPE is not clearly justified, as the proposed priors are essentially created with local and temporal feature correlation and differences, which are assumed to be better motion-aware transitions. While the results show the gains observed, it is unclear which has the biggest impact on performance, the proposed kinematic formulation or the introduced temporal processing.
-
The reported training efficiency is questionable, as the claimed 10.7x reduction in training compares VideoMamba-M800e, which performs a full retraining + fine-tuning (~830 epochs), with PKS⁴, which only performs fine-tuning(~20 epochs). While both configurations benefit from image/CLIP initialization, the difference in the video domain schedule is what is seen in the reported reduction figure, making it hard to attribute the claimed efficiency directly to the proposed architecture.
-
The performance comparison missing the relevant and more recent methods, especially a closely related work like MoMa (ICML 2025), is concerning. A more comprehensive comparison is needed to establish the current work's actual performance and the accuracy-efficiency advantage.
Questions for authors
-
What is the key novelty of the proposed framework, given that several of the proposed components have already been explored in prior works? And what is the advantage of making use of Kinematic priors with parallel SSMs over the existing approaches?
-
While the authors have established that the improvements arise from KPE specifically due to better motion modelling, it is unclear whether the improvements are indeed due to the correlation and variation operator or the addition of temporal processing. This needs to be clarified.
-
Can the authors clarify how the claimed 10.7x trading-efficiency improvement is attributed to the architecture, especially with the presence of substantial differences in their training schedules?
-
Given that there are other recent and closely related approaches, such as MoMa, the authors would have to justify the claimed performance and efficiency competitiveness of their proposed framework.