- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether differences in frame-order sensitivity between pretrained video models persist after downstream end-to-end fine-tuning. The authors primarily compare VideoMAE checkpoints from different pretraining stages and evaluate their frame-order sensitivity on two fall-detection datasets, OF-Syn and OOPS. They observe that the differences between frozen checkpoints largely disappear after fine-tuning on OF-Syn, while the results on OOPS are less conclusive. Additional experiments involving an order-defined task, layer-wise probing, and other video architectures are conducted to investigate whether these changes reflect differences in temporal information availability or its task-specific utilization.
Critical strengths
-
The paper focus on temporal representation learning and transfer, which are important topics in video understanding.
-
The comparison between VideoMAE checkpoints before and after supervised Kinetics training provides a relatively controlled setting for investigating the effects of different pretraining stages.
-
The authors conduct extensive experimental analyses, including multiple random seeds, different fine-tuning schedules, alternative shuffling strategies, and layer-wise probing.
Critical weaknesses
- Insufficient motivation and unclear significance of the research question.
Understanding temporal representation learning and transfer is important. However, the paper does not adequately explain why differences in frozen frame-order sensitivity should be expected to persist after end-to-end fine-tuning, which explicitly adapts the entire network to a new task. The authors also fail to establish that existing studies rely on this assumption or that its violation has meaningful practical consequences. Therefore, the significance of the specific research question remains unclear.
- A mismatch between the evaluation tasks and the research objective.
The main experiments rely on fall detection, which can potentially be solved using static visual cues rather than temporal ordering. This limitation is evident in the authors' own results: frozen VideoMAE checkpoints differ substantially in sensitivity on OF-Syn Fall (0.151 vs. 0.048), but show similar sensitivity on Up/Down (0.325 vs. 0.303). These task-dependent measurements do not directly characterize temporal representation capabilities, weakening the connection between the experiments and the paper's broader conclusions.
- Inconsistent findings and limited generalizability.
The main finding is primarily supported by VideoMAE on synthetic OF-Syn. On real-world OOPS, the reported κ = 0.70 [-0.41, 1.34] is inconclusive. Furthermore, TimeSformer and Hiera retain their sensitivity differences after fine-tuning on the same OF-Syn Fall task. Although the authors acknowledge these discrepancies, they do not establish what determines whether the differences persist, leaving the conclusions highly architecture- and task-dependent.
Questions for authors
see weakness