- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper asks whether pretrained video generators contain subnetworks with stronger physical behavior than the full model. It samples structured masks over attention heads and feed-forward neurons, measures solution density across model families and scales, and selects reusable masks using reference-free physics screeners. Selected subnetworks improve Physics-IQ and gravity metrics over generation-seed best-of-(N) and a matched-compute baseline, transfer across scenes and seeds, and preserve perceived visual quality. Additional analyses identify category specialists and generalists and attribute gravity improvements to specific attention heads.
Critical strengths
- Structured masking offers a novel, training-free way to expose and reuse physically stronger generator subnetworks.
- Matched budgets, held-out scenes, fresh seeds, component interventions, and blinded human studies provide unusually thorough validation.
Critical weaknesses
-
The term “physics experts” overstates what the experiments establish. Broad masking may improve learned metrics through redundancy, regularization, reduced motion, trackability, or generic distribution shifts rather than exposing localized physical knowledge. Comparable visual quality does not exclude these alternatives, and causal head attribution is demonstrated only for gravity in one model.
-
The scaling and robustness evidence remains underdetermined. Each checkpoint uses one mask pool and only three deployment runs; model size co-varies with architecture and dropout tolerance, while Seed BoN reuses one fixed pool. Hyperparameter sweeps and benchmark-best baseline variants lack nested testing or multiplicity correction, and Cosmos gravity gains reverse at .
Questions for authors
-
After controlling for generic fidelity, temporal smoothness, motion magnitude, V-JEPA predictability, and tracking success, do the overall and category-specific mask advantages remain? Please include unmasked and population-level controls rather than only selected outputs.
-
Does the scale trend persist across independently resampled mask pools and masking-rate sweeps that equalize absolute removed units, output perturbation, or generic-quality degradation? Please provide uncertainty estimates for the resulting solution densities.
-
Can Seed BoN be repeated with independent seed pools paired to each mask-selection run, with hierarchical confidence intervals over candidate pools, generation seeds, and scenario clusters?
-
Can the component-attribution result be reproduced across fresh seeds, both model families, and additional physical domains, while reporting gravity error conditional on successful tracking?
5 comments
W1-Q4: Does component attribution replicate with fresh seeds, in both families and in more domains, with gravity error conditional on tracking?
A-Q4: Yes. We repeated the head intervention of App. E.1 at four new seeds, on solid mechanics and fluids (the two largest Physics-IQ categories) and on free fall, in both Wan2.2 and Cosmos. For free fall, we report the gravity error conditional on tracking, that is, only on the scenes tracked in both models being compared.
Rebuttal Table 16: All six differences favor the attributed heads. Results are at four new seeds.
| Gain over removing 16 random heads ↑ [95% CI] | Wan2.2-A14B | Cosmos3-Super |
|---|---|---|
| Physics-IQ score, solid mechanics | +3.32 [+0.29, +6.73] | +3.54 [+0.59, +7.12] |
| Physics-IQ score, fluids | +2.63 [+0.26, +5.16] | +1.14 [−0.57, +3.56] |
| Free-fall log-RMSE reduction, tracked scenes | +0.050 [−0.005, +0.114] | +0.110 [+0.043, +0.171] |
W2: Robustness of the scaling and selection results
A2: The scaling trend persists across independently resampled mask pools and matched masking rates (Q2). Our advantage largely persists when Seed BoN is repeated with independent seed pools (Q3). We answer the remaining two points here.
Nested testing and multiplicity. K = 16 and M = 200 were fixed before experiment. The sweeps in Tables 15 and 16 of the paper only measure sensitivity, and no setting was chosen from them. In these sweeps, Ours has lower error than Seed BoN in settings with K < M on both models, so the direction of the result does not depend on which setting is reported. We take "benchmark-best baseline variants" mentioned by you to mean Proprio, since it's our only baseline with variants. To keep our comparison fair, we ran its three scoring variants and report the best one for each model and benchmark. Global scoring is Proprio's best variant for both models on both benchmarks (Table 5 of the paper). Ours, without such preselection, still beats this best variant.
Cosmos at K = 200. The sign changes only when K equals the pool size M. The comparison is then between the consensus votes of the two whole pools. Our advantage comes from a higher density of experts in the mask pool, not from the whole pool being better, which is why Ours is not necessarily ahead at K = M.
Q2: Does the scale trend hold across resampled mask pools and masking rates matched on removed units, output perturbation or quality loss?
A-Q2: Yes, the scale trend persists across independently resampled mask pools and masking-rate sweeps.
Q2.1. Independently resampled mask pools. For each model we compare three pools of 50 masks. Each model's density is similar across pools, and the order 5B < A14B and Edge < Nano < Super holds in every pool.
Rebuttal Table 17: Independently resampled mask pools.
| Solution density ↑ [95% CI] | Pool 1 | Pool 2 | Pool 3 |
|---|---|---|---|
| Wan2.2-5B | 2% [0, 10] | 0% [0, 7] | 0% [0, 7] |
| Wan2.2-A14B | 74% [60, 84] | 82% [69, 90] | 72% [58, 83] |
| Cosmos3-Edge | 6% [2, 16] | 8% [3, 19] | 10% [4, 21] |
| Cosmos3-Nano | 32% [21, 46] | 22% [13, 35] | 28% [18, 42] |
| Cosmos3-Super | 58% [44, 71] | 58% [44, 71] | 54% [40, 67] |
Q2.2. Masking-rate sweeps.
(a) Equal absolute removed units. Removing as many units from the smaller model as from the larger one hurts the smaller model more than the paper's percentage masks do.
Rebuttal Table 18: Equal absolute removed units.
| Solution density ↑ [95% CI] | Larger model | Smaller model, same units as the larger |
|---|---|---|
| Wan2.2: A14B vs 5B | 78% [69, 85] | 0% [0, 11] |
| Cosmos3: Super vs Nano | 58% [48, 67] | 0% [0, 11] |
| Cosmos3: Nano vs Edge | 27% [19, 36] | 0% [0, 11] |
(b) Equal output perturbation. We measure perturbation as the LPIPS distance (Zhang et al., 2018) between a masked video and the unmasked video from the same seed. With the paper's masks, 5B is perturbed more than A14B. We adjust the masking rates until the perturbation matches. This raises 5B's density, but the larger model stays far ahead in both families.
Rebuttal Table 19: Equal output perturbation.
| Model | Perturbation (LPIPS) | Solution density ↑ [95% CI] |
|---|---|---|
| Wan2.2-A14B | 0.100 | 78% [69, 85] |
| Wan2.2-5B | 0.098 | 23% [12, 41] |
| Cosmos3-Super | ≈ 0.10 | 37% [22, 55] |
| Cosmos3-Nano | ≈ 0.10 | 6% [2, 16] |
Cosmos3-Edge cannot get down to this perturbation level, because the mask rule removes at least one of its 16 heads in every layer.
(c) Equal generic-quality degradation. With the paper's masks, VBench quality changes very little for every model, about as much as a change of seed. So the paper's scale trend is already measured at equal quality loss.
Rebuttal Table 20: Equal generic-quality degradation.
| Masked − unmasked (same seed) | VBench imaging quality ↑ | VBench aesthetic quality ↑ |
|---|---|---|
| Wan2.2-5B | −0.009 | −0.002 |
| Wan2.2-A14B | −0.005 | −0.007 |
| Cosmos3-Edge | +0.013 | −0.003 |
| Cosmos3-Nano | +0.002 | +0.001 |
| Cosmos3-Super | −0.002 | −0.003 |
References: Zhang et al., The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (CVPR 2018).
Q3: Can Seed BoN be repeated with independent seed pools?
A-Q3: Yes. On Physics-IQ, we repeat Seed BoN with three independent pools of 100 seeds per model, one for each mask-selection run.
Rebuttal Table 21: Seed BoN with independent seed pools.
| Performance gain over Seed BoN ↑ | Wan2.2-A14B | Cosmos3-Super | Both models |
|---|---|---|---|
| Physics-IQ [hierarchical 95% CI] | +2.58 [−0.36, +5.86] | +1.51 [−2.02, +5.28] | +2.04 [−0.30, +4.66] |
The intervals resample scenario clusters, generation seeds with their paired pools, and the candidates in each pool, rerunning the selection each time. Ours wins in all six runs.
We thank the reviewer for the constructive and thorough feedback. We address each point below.
W1-Q1: Do the overall and category-specific mask advantages remain after controlling for fidelity, smoothness, motion, V-JEPA predictability and tracking?
A-Q1: Yes. We go through the five factors one by one.
Generic fidelity. In Physics-IQ, only the MSE term measures generic fidelity. It compares pixels with the real video. The other three terms score where, when and how much things move. Our lead over Seed BoN comes from these three terms, and the MSE term adds almost nothing (Rebuttal Table 10). Without the MSE term, the population-level solution density also stays the same (Rebuttal Table 11).
Rebuttal Table 10: Physics-IQ lead by term.
Rebuttal Table 11: Solution density without MSE.
Temporal smoothness. We measure it with VBench's motion smoothness and temporal flickering on all 198 videos of each of the 100 masks and 100 unmasked seeds per family. A score difference of 0.001 is about a quarter of one gray level per pixel. So the differences between masks and seeds are tiny (Rebuttal Table 12).
Rebuttal Table 12: Temporal smoothness.
Motion magnitude. Physics-IQ does not reward moving more or moving less. It checks whether the video moves where and when the real recording moves, and stays still where the recording is still. Physics-IQ Verified (Rädsch et al., 2026) further removes motion in the real recordings that has nothing to do with the physical event being tested, so that only the relevant motion is scored. On Verified, our generator also beats Seed BoN (Rebuttal Table 13).
Rebuttal Table 13: Physics-IQ Verified.
V-JEPA predictability. We measure it with V-JEPA 2 surprise, the score our reward model uses (lower = more predictable), on all 198 videos of each of the 100 masks and 100 unmasked seeds per family. Masks and seeds are about equally predictable (Rebuttal Table 14), so the comparison is fair to both sides.
Rebuttal Table 14: V-JEPA 2 surprise.
Tracking success. It only affects free fall, since Physics-IQ uses no tracker. Across all 200 masks and 200 unmasked seeds per family, on the balls that are tracked, masks are still more accurate than seeds (Rebuttal Table 15). Fidelity, smoothness and V-JEPA predictability cannot change the free-fall score in any other way, because the score uses only the ball's motion.
Rebuttal Table 15: Free fall on tracked balls.
References: Rädsch et al., Physics-IQ Verified (arXiv 2026).