- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies OOD generalization of physical laws in video generation: a model trained on a restricted "primitive" regime of a physical law (e.g., purely vertical projectile motion, or collision with a stationary target) must generalize to a structurally different regime governed by the same invariant law (arbitrary-direction projectile motion, or collision between two moving balls), as formalized in Definition 3.1. To address this, the paper proposes Physics-AdaNet, combining (1) Adaptive Physics Alignment (AdaAlign), a spatial+temporal Token Relation Distillation loss (Eq. 9-10) that aligns the diffusion backbone's intermediate features with relational structure extracted by a pretrained visual encoder from ground-truth video, with a temporal masking mechanism that exempts the I2V conditioning frames from this alignment; and (2) a NAS-based Adaptive Projector that searches (via a weight-sharing supernet + evolutionary search, Eq. 12-14) a scenario-specific architecture for the feature projector used in the alignment, rather than using a fixed projector architecture.
The method is evaluated on two synthetic benchmark tasks built on CogVideoX-5B (Newton's law of universal gravitation, and conservation of momentum), each with an ID split (training-distribution dynamics) and a structural-OOD split (Table 1). Physics-AdaNet outperforms CogVideoX, PISA, and VideoREPA on OOD kinematic error in both tasks while remaining competitive on ID scenes and standard visual-fidelity metrics (FVD/SSIM/PSNR/LPIPS). An ablation (Table 2, Appendix C) attributes gains to both AdaAlign and the NAS projector, and the paper reports that a projector architecture searched on one task degrades when transferred to the other task, which is used to argue the searched architectures are scenario-dependent.
Critical strengths
The problem formulation (Definition 3.1) cleanly isolates structural OOD shift from the scalar-parameter OOD shift studied in prior work (Kang et al., 2024), and the two benchmark tasks operationalize this distinction concretely and verifiably via closed-form kinematics (Eq. 5-8).** Because ground-truth trajectories are known exactly, the paper can report a well-defined kinematic error metric rather than relying only on pixel-level fidelity, which is the right evaluation axis for the claim being made (physical-law generalization, not just visual quality), and the CogVideoX failure cases in Fig. 2(b) make the targeted failure mode concrete.
Critical weaknesses
W1. The benchmark consists of only two synthetic, low-dimensional kinematic scenarios (single ball under gravity; two-ball 1D elastic collision), and all headline claims rest entirely on this benchmark.
Both tasks have closed-form, low-order-polynomial solutions and a handful of scalar parameters (position, velocity, radius, color). This is a reasonable minimal testbed to demonstrate the structural-OOD concept, but it is a narrow basis for the paper's broader framing (Introduction, Conclusion) that the method addresses "OOD generalization of physical laws" in video generation generally. There is no scenario with more than two interacting bodies, no rotational dynamics, no scenario where the structural shift could plausibly be confused with a simple interpolation of training trajectories, and no systematic variation of how large the structural gap is. The real-world qualitative result (Fig. 3b, one scene) is explicitly preliminary and not quantified. Given this, the quantitative superiority in Table 1 is good evidence for the two tasks as constructed, but weaker evidence for the paper's stated general claim than the framing suggests.
W2. The NAS component's claimed benefit (scenario-specific architecture adaptation) is undercut by the paper's own cross-task transfer result, and no comparison is made against simpler alternatives to NAS.
Section 4.3 reports that a projector searched on one task "degrades when retrained on the other task," which the paper frames as confirming architectures are scenario-dependent. An equally plausible reading is that the search is overfitting to per-task noise in the small AdaAlign-loss proxy signal rather than discovering a genuinely different and better-matched architecture per scenario; the paper does not report variance across repeated searches, nor compare against a plausible non-NAS baseline (e.g., a single reasonably-sized fixed projector, or a random architecture of matched size) that would show the search is doing more than selecting a well-sized network. Without that control, the ablation (ID/OOD with and without the NAS projector, Table 2) establishes that some projector configuration helps, but not that the evolutionary search over the specific space in Table 3 (Appendix E) is necessary.
Minor Comments
- The baselines PISA and VideoREPA are adapted from 3D-physical-video and general T2V alignment settings respectively (Sec. 4.1); PISA in particular drops its optical-flow/semantic-segmentation supervision since depth maps are unavailable in 2D. This is disclosed, which is appreciated, but it leaves open whether the baselines are tuned as well for this specific benchmark as Physics-AdaNet.
- (the AdaAlign weight) is fixed via small-scale preliminary experiments rather than jointly searched with the architecture (Sec. 3.3); this is a reasonable practical simplification but means the reported configuration may not be at the actual optimum for either component.
Questions for authors
- NAS vs. simpler baselines. Does the searched projector outperform a fixed, reasonably-sized projector of comparable parameter count, or a few manually chosen configurations, on the same two tasks? This would clarify whether the evolutionary search is finding something non-trivial versus mainly selecting adequate capacity.
- Robustness of the "scenario-dependent" claim. Is the degradation observed when transferring the searched architecture across tasks consistent across repeated independent searches (different random seeds for the evolutionary search), or could it reflect search variance rather than a genuine scenario-specific optimum?
5 comments
Response to Reviewer dTnF (2)
At the same time, we acknowledge that precise physical parameters of the real-world footage remain unavailable and that these measurements use the corresponding 2D velocity sequences as references. We will therefore present the combined qualitative and quantitative results as preliminary transfer evidence beyond synthetic appearance, rather than as comprehensive validation of unrestricted real-world physical dynamics.
W2 and Q1. Architecture-selection controls: manual, random, and grid alternatives
We agree that the current cross-task degradation alone cannot distinguish genuine scenario dependence from variance in the evolutionary search or its proxy. Similarly, comparison with only the heavier hand-crafted three-layer MLP does not establish that evolutionary search is necessary.
To address this issue, we add the following controls:
- A manually designed, parameter-matched two-layer MLP projector, which provides a shallow manual alternative with manually chosen hyperparameters following VideoREPA, except for the number of layers and the feature dimensions.
- Random search, which uniformly samples 455 candidate architectures from the same search space and selects the candidate with the lowest . Unlike evolutionary search, it does not use previous evaluation results to guide subsequent sampling.
- Grid search with a depth prior, which exhaustively evaluates 576 architectures with the projector depth fixed to . Specifically, the reduced grid consists of alignment layers ; normalization methods ; and all ordered module-type combinations , where , allowing different module types in the two layers. The hidden dimension satisfies . The four activation-function configurations are given by , where .
This yields candidates. The candidate with the lowest is selected for standalone training. To keep the evaluation budget broadly comparable, we use a reduced grid with a two-layer depth prior and the restricted axis choices specified above. Exhaustively extending this grid to the full depth range and all per-layer architecture combinations would be computationally prohibitive due to its exponential growth with depth.
All variants use the same masking strategy, alignment objective, value of , training data, and 8,000 final standalone-training steps. Only the architecture-selection strategy changes.
Table R2. Comparison of architecture-selection strategies on OOD scenes. All searched and manually designed alternatives use the same final training protocol.
| Selection strategy | Evaluated candidates | Task 1 | Task 2 | Task 2 |
|---|---|---|---|---|
| Manually designed three-layer MLP | 1 | 6.8020 ± 0.1501 | 7.4039 ± 0.2475 | 8.2445 ± 0.1951 |
| Manually designed two-layer MLP | 1 | 6.5141 ± 0.1468 | 6.7932 ± 0.1884 | 8.0492 ± 0.2467 |
| Random search | 455 | 6.0361 ± 0.1736 | 6.3685 ± 0.1697 | 8.4528 ± 0.2379 |
| Grid search | 576 | 4.9208 ± 0.1612 | 5.7320 ± 0.1813 | 7.9244 ± 0.2635 |
| Evolutionary search | 455 | 4.3210 ± 0.1355 | 5.1170 ± 0.1593 | 7.5075 ± 0.2703 |
The shallow manual control improves over the three-layer default on all three metrics, while its comparison with random search is mixed. This suggests that projector depth and manual configuration explain part of the gain, but do not account for the consistent further improvements achieved by evolutionary search. Grid search is the strongest non-evolutionary alternative. Importantly, its evaluation cost is kept manageable by using the reduced grid described above, including the two-layer depth prior and the restricted axis choices. Extending the exhaustive grid to deeper projectors would be computationally prohibitive due to the exponential growth of per-layer architecture combinations. Even with this depth prior and a slightly larger candidate-evaluation budget than evolutionary search (576 versus 455), grid search remains inferior to evolutionary search on all three final OOD metrics. These results indicate that the gains are not attributable merely to selecting a suitably sized two-layer network or to a larger search budget; the architecture-selection strategy itself contributes to identifying a projector better suited to each scenario.
Response to Reviewer dTnF (3)
Q2. Search-seed sensitivity on a fixed supernet
To directly assess the variability introduced by evolutionary search itself, we repeat the search five times on the same trained supernet, using the same held-out validation set and 455 candidate evaluations per run. Only the random seed controlling population initialization, parent selection, crossover, and mutation changes. Tables R3a and R3b report the selected architectures and their corresponding best values.
Table R3a. Architectures selected by five independent evolutionary searches on the same supernet for Task 1.
| Search run | Alignment layer | Normalization | Depth | Hidden dimension | Module sequence | Activation sequence | Best |
|---|---|---|---|---|---|---|---|
| Run 1 | 18 | BatchNorm | 2 | 1024 | (Linear, Linear) | (SiLU, Identity) | 0.0574 |
| Run 2 | 18 | BatchNorm | 2 | 1024 | (Linear, Linear) | (SwiGLU, Identity) | 0.0576 |
| Run 3 | 18 | BatchNorm | 2 | 1024 | (Linear, Linear) | (SwiGLU, Identity) | 0.0576 |
| Run 4 | 18 | BatchNorm | 2 | 1024 | (Linear, Linear) | (SiLU, Identity) | 0.0574 |
| Run 5 | 18 | BatchNorm | 2 | 1024 | (Linear, Linear) | (SiLU, Identity) | 0.0574 |
Table R3b. Architectures selected by five independent evolutionary searches on the same supernet for Task 2.
| Search run | Alignment layer | Normalization | Depth | Hidden dimension | Module sequence | Activation sequence | Best |
|---|---|---|---|---|---|---|---|
| Run 1 | 12 | RMSNorm | 2 | 2048 | (Linear, Linear) | (SwiGLU, Identity) | 0.0593 |
| Run 2 | 12 | RMSNorm | 2 | 2048 | (Linear, Linear) | (SiLU, Identity) | 0.0597 |
| Run 3 | 12 | RMSNorm | 2 | 2048 | (Linear, Linear) | (SwiGLU, Identity) | 0.0593 |
| Run 4 | 12 | RMSNorm | 2 | 2048 | (Linear, Linear) | (SwiGLU, Identity) | 0.0593 |
| Run 5 | 12 | RMSNorm | 2 | 2048 | (Linear, Linear) | (SwiGLU, Identity) | 0.0593 |
Across five independent evolutionary searches on the same trained supernet, the selected architectures exhibit strong stability in their core structural choices. For Task 1, all runs select layer 18, BatchNorm, a two-layer Linear projector, and a hidden dimension of 1024. Only the first-layer activation varies between SiLU and SwiGLU, with the originally reported SiLU configuration recovered in three of the five runs. For Task 2, all runs select layer 12, RMSNorm, a two-layer Linear projector, and a hidden dimension of 2048. The originally reported SwiGLU configuration is recovered in four of the five runs, while the remaining run differs only in using SiLU as the first-layer activation. The two-layer Linear design is preserved in 5/5 runs for both tasks, and all architectural choices other than the first-layer activation remain unchanged across the repeated searches.
The best values also show limited dispersion. Task 1 scores range from to , and Task 2 scores range from to . Conditional on the same trained supernet and the same held-out validation set, these results show that the core architecture-selection outcomes are insensitive to the random seed of the evolutionary search. In particular, each task repeatedly recovers the same alignment layer, normalization method, hidden dimension, and shallow Linear projector family, with variation limited to the first-layer activation. We therefore interpret the selected projectors as repeatedly selected candidate designs under the fixed-supernet protocol.
Response to Reviewer dTnF (1)
Response to Reviewer dTnF
We thank the reviewer for the thoughtful assessment. We narrow the scope of our physical-law claims, add manual, random, and reduced-grid architecture-selection controls, analyze search-seed sensitivity conditional on a fixed supernet, and clarify PISA adaptation and λ selection. We also report guidance-referenced velocity measurements for the real-world generations, while acknowledging their limitations as independent real-world physical validation.
W1. Narrow benchmark scope and the broader “physical-law” framing
We agree that the phrase “learning physical laws” is broader than what our experiments establish. Our study does not recover explicit governing equations, identify symbolic physical laws, or demonstrate general-purpose physical reasoning. We also do not evaluate interactions involving more than two bodies, rotational dynamics, interpolation-controlled structural shifts, or systematic variation of the structural gap; these remain outside the scope of our current empirical evidence.
Our evidence supports a narrower claim: when the governing mechanism remains fixed but a condition variable frozen during training becomes active at test time, Physics-AdaNet improves the transfer of motion regularities consistent with task-specific physical constraints. The reduced OOD velocity errors indicate improved law-consistent motion generation under the evaluated structural shifts, rather than explicit equation recovery. We will therefore replace broad instances of "learning physical laws" with "learning transferable physical priors for structural OOD generalization". We will also limit our empirical claims to the two controlled benchmarks and describe the real-world results as preliminary qualitative and guidance-referenced quantitative evidence beyond synthetic appearance.
Nevertheless, we believe the two controlled settings remain meaningful. Although their governing equations are simple, the compared baselines exhibit substantially larger OOD kinematic errors on these structural shifts:
Thus, the results support the narrower conclusion that Physics-AdaNet effectively addresses these two nontrivial structural-motion shifts, without implying a general solution to arbitrary physical laws.
To assess how faithfully the real-world generations preserve the supplied motion guidance, we are adding quantitative velocity measurements for the generations in Figure 3(b) and Figures 14–16. Object velocities will be extracted from the generated real-world videos and compared with the corresponding 2D ground-truth velocity sequences.
Table R1a. Quantitative velocity errors for real-world gravitation scenes, using the corresponding 2D ground-truth velocities as references.
Table R1b. Quantitative velocity errors for real-world collision scenes, using the corresponding 2D ground-truth velocities as references.
These results show that the improvements in both sustained horizontal motion and collision-induced velocity changes persist across different object appearances. The overall error magnitudes and relative method rankings are broadly consistent with the measurements on the 2D videos. This similarity is expected because the real-world generation process uses the corresponding 2D videos as motion maps: their trajectories guide the warping of the initial noise, while each model generates subsequent frames from the same warped noise and initial frame. The dominant motion structure is therefore shared between the 2D guidance and the real-world generation process, even though their visual appearances differ. Nevertheless, the consistent relative improvements across balls and cubes support the robustness of Physics-AdaNet’s kinematic gains under changes in object morphology and visual appearance.