- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
To test whether video generators actually follow physical laws rather than just rendering plausible-looking frames, the authors build PCBench, combining 3,093 real videos, 4,049 generated clips from six generator families, and 945 Unity simulations with controlled physical perturbations. A four-axis human annotation protocol scores general physical plausibility (SCS-ge) before revealing the prompt to separate it from prompt-specific phenomenon realization (SCS-sp), showing that generators fail at the requested physical event far more often than at general coherence. For seven mechanics families, the paper fits analytical motion equations to Grounded-SAM2 tracks to compute a Motion-Law Conformity (MLC) score, and reuses those motion programs in Blender to drive and re-verify Wan2.1-VACE-1.3B generation.
Critical strengths
-
Having annotators score general physical plausibility (SCS-ge) before seeing the prompt is a clean, effective way to separate "looks like plausible physics" from "actually produced the requested physical phenomenon" (SCS-sp). Paired with the 945 controlled Unity videos across 18 motion-perturbation pairs, the benchmark provides both a clear empirical finding across six generators and a well-calibrated synthetic testbed for trajectory evaluators.
-
The paper is unusually candid about negative results and uncertainty, and the temporal permutation test in Section 5 is a great diagnostic. Scrambling the middle control frames preserves visual appearance and endpoints while breaking dynamics; showing that trajectory forecasting catches this while VideoScore2 fails (and correlates inversely with the pass/fail gates) clearly demonstrates why VLM judges alone are insufficient.
Critical weaknesses
-
It is unclear whether MLC reliably ranks generated clips within a given motion type, or if the pooled correlation (rho = 0.382 on the 216 generated mechanics clips) is mostly a between-family scaling artifact. In Appendix D (Tables 15, 17, and 19), within-family correlation is only 0.125 (CI includes zero), six of seven families have near-zero or negative correlations, and replacing min-max normalization with percentile ranks drops Test95 generated correlation from 0.381 to 0.048.
-
The "agentic" framing feels overstated relative to the actual experiments. In Section 4, the motion family comes from the ground-truth category, equations are fixed, and humans manually chose the target track in 227 of 421 multi-object clips (54%). In Section 5, Blender scenes and programs are manually supplied (real-video reconstruction in Appendix E.13 failed), and on the two repair cases the agent action simply turns on full 49-frame control.
Questions for authors
-
Why is within-family correlation near zero for most mechanics families, and does applying the coverage and motion gates from Section 5 to the benchmark clips improve within-family ranking or robustness to percentile normalization? If gating resolves this, I would be glad to take that into account in my final rating.
-
How much does the MLC correlation on the 216 generated clips change if you replace manual track selection on the multi-object clips with an automated heuristic (such as GroundingDINO confidence or a VLM prompt)?