- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores whether physical quantities can be identified in the internal activations of large video diffusion models. Using simulator-generated videos with ground-truth physical quantities, the authors train linear probes to decode properties such as speed, direction, and gravity. The paper further analyzes where and when these signals appear in the model. It also evaluates transfer across changes in confounding factors. The results show strong linear decodability of physical information in large video diffusion models.
Critical strengths
-
The experiments cover several large video diffusion models. This work analyzes three different backbones and evaluates the signals across model compoents, which provides a relatively comprehensive empirical characterization.
-
The paper makes a meaningful effort to control possible confounding factors. In addition to matched latent baselines, the transfer experiments independently vary across confounding factors. These controls improve the rigor of the empirical analysis and help test whether the decoded signals are tied only to particular scene configurations.
Critical weaknesses
-
The main conclusion relies heavily on linear probing, but the paper does not sufficiently justify why linear decodability should be interpreted as evidence of an internal physical representation. Without nonlinear comparisons or further analysis, the experiments establish linear accessibility, but not the rigorous representation form of the underlying physical information.
-
The connection between the controlled variables and the claimed physical concepts has not been rigorously identified. A probe predicting gravity, speed, or momentum could still rely on correlated visual or dynamical features. This means that the decoded directions cannot be uniquely attributed to the claimed physical quantities.
-
The practical value of these linearly decodable signals remains unclear. Given that prior work has already probed physical information in video-model representations, the paper does not demonstrate a concrete application showing how these representations can be used to improve video generation (in terms of physical correctness) or the diagnosis of generated videos.
Questions for authors
-
To what extent do the controlled variations isolate the intended physical concept? While the confounder tests seem useful, it remains unclear whether or why they are sufficient to rule out correlations and uniquely attribute the decoded direction to the claimed physical quantity
-
Beyond model diagnosis, what practical uses do the authors see for these representations? How can they be used to improve physical generation or control in a concrete way? Or even to diagnose a model or the generated video?