- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PhyTraj for detecting physical violations in generated videos. The method uses estimated object trajectories to query visual features from a frozen VideoMAE encoder, followed by object-level fusion and classification. The authors evaluate the method on 2,334 human-labeled videos covering six physical-rule categories. In exploratory experiments, the full model achieves 0.758 AUROC versus 0.683 for the video-only baseline. The problem is interesting, and the results are promising. However, I have several concerns about the novelty of the method and whether the experiments adequately support the proposed motion-query mechanism.
Critical strengths
-
Using object trajectories to guide visual feature extraction is a reasonable approach to physical violation detection, particularly for contact and collision-related events.
-
The paper includes useful ablations and evaluates on held-out object compositions. I also appreciate the group-level bootstrap analysis and the discussion of experimental limitations.
Critical weaknesses
-
The method combines fairly standard components, including object tracking, temporal Transformers, cross-attention, and feature aggregation. Related work such as VAMP and PDI-Bench has already explored motion-based physics evaluation. The main distinction appears to be using trajectories as attention queries, but the paper lacks comparisons with simpler approaches such as feature concatenation, late fusion, or trajectory-only classification. It is therefore difficult to assess the benefit of the proposed design.
-
In Table 3, the full 3D model achieves 0.758 AUROC, while shuffled trajectories and 2D trajectories both reach around 0.753. These differences are quite small. Although the shuffling experiment preserves local motion features, the results raise questions about how much the model actually benefits from temporal ordering and 3D motion information. More convincing ablations would strengthen the main claim.
-
All videos come from Seedance 2.0 under fixed-camera settings. Holding out object compositions is useful, but generator-specific artifacts and similar event templates may still appear across splits. The method also relies on metadata-provided object names and excludes videos without valid tracks. It would be helpful to evaluate on another video generator and examine how performance changes with tracking quality.
-
The representation ablations were added after examining results on the same test set. In the separate frozen-feature study, the AUROC improvement over VideoMAE-only is 0.045, with a 95% confidence interval of [-0.0006, 0.0865]. Also, comparisons with VideoPhy-2 and other evaluators are not entirely fair, since PhyTraj is trained on the target dataset while those methods are evaluated zero-shot. Stronger matched baselines or an independent test set would make the results more convincing.
Questions for authors
-
How does trajectory-query attention compare with simpler fusion methods under similar model capacity? Is there evidence that the proposed attention mechanism provides benefits beyond adding trajectory features?
-
How well does PhyTraj generalize across video generators and different levels of tracking quality? Could the authors report per-category results with uncertainty estimates to clarify where gains arise?