- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces YOLO-ST, combining VideoMAE, a frozen YOLO11 backbone, and a frozen motion pathway, with dense heads predicting frame geometry and tube queries predicting actor ownership and temporal extent. It evaluates the framework on four benchmarks.
Critical strengths
-
Scoring the same predictions under different conventions produces frame mAP ranging from 88.47 to 95.10, exposing consequential differences in frame selection, AP integration, and bounding-box conventions.
-
Geometry snap improves UCF101-24 test video AP50 and AP50:95 by 0.46 and 0.79 points, respectively, providing concrete evidence that dense detections can improve tube localization.
Critical weaknesses
This paper is obviously low quality and should be rejected. Without any innovation or technical contribution shown in paper.
-
The central benefit of ownership–geometry decoupling is not established. TubeR [1] and STAR [2] already provide explicit tube modeling, so this paper must demonstrate the additional value of its division of responsibilities through controlled comparisons against dense-only and fully shared dual-head alternatives. Tube-extent gating fails to improve action-free-frame detection, while JHMDB and MultiSports ultimately use linked dense detections because these outperform tube queries (section 5.3), leaving the necessity of the central ownership branch unproven. Also, the performance comparisons do not compensate for the missing methodological evidence.
-
The training objective is underspecified at precisely the points supporting the main claim. In this paper, equation (3) does not mathematically define key terms or specify the Hungarian matching cost and supervision of unmatched queries. For AVA, the paper explains only dense-loss masking, leaving unclear how the tube branch’s softmax classification and interval/boundary supervision accommodate sparse, multilabel annotations; consequently.
-
The frozen motion trunk was trained using labels from the Tune and Confirm partitions, removing it reduces Confirm frame mAP by 3.13 points but test frame mAP by only 0.30, exposing severely inflated validation evidence (section 4.2, Table 2). The selection protocol is contaminated, and retrospective freezing cannot repair it.
-
Geometry snap lacks an identity-preserving association mechanism. Section 3.4 moves each tube box 75% toward the highest-IoU detection of the same class, without describing actor-identity constraints or one-to-one assignment across tubes. This rule permits identity switches or multiple tubes snapping toward the same detection when same-class actors overlap, and no corresponding error analysis is provided.
In the experiment, the ablations do not justify the architectural complexity. Removing the task adapters and pyramid adapter improves Confirm frame mAP by 0.30 and 0.40 points, respectively, while removing cross-clip memory has essentially no effect; even within the authors’ contaminated validation protocol, these results do not support retaining those components. Their video-mAP ablations are omitted, preventing assessment of whether they improve temporal ownership sufficiently to justify their inclusion.
[1] Zhao, J., Zhang, Y., Li, X., Chen, H., Shuai, B., Xu, M., Liu, C., Kundu, K., Xiong, Y., Modolo, D., Marsic, I., Snoek, C.G.M. and Tighe, J., 2022, June. TubeR: Tubelet transformer for video action detection. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 13598–13607).
[2] Gritsenko, A.A., Xiong, X., Djolonga, J., Dehghani, M., Sun, C., Lucic, M., Schmid, C. and Arnab, A., 2024, June. End-to-end spatio-temporal action localisation with video transformers. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 18373–18383).
Questions for authors
-
What operational definition of “decoupling” is used, and what binds dense geometry to tube identity?
-
Under identical backbones, data, and training budgets, how do dense-only, fully shared dual-head, and partially isolated models compare?
-
Does the tube branch improve temporal boundary error, temporal-only IoU, or false positives on action-free frames?
-
How does geometry snap prevent identity switches and many-to-one associations? What are the identity-error rates before and after snapping?
-
What are the exact definitions of all losses in Equation (3), the matching cost, and unmatched-query supervision?
-
What ensures that the two interval queries within an actor slot represent the same person?
-
Is the tube branch enabled on AVA? How are multilabel classification and missing interval annotations handled?