acceptodds
Under review as a conference paper at ICLR 2027

Your Video Generator Secretly Contains Physics Experts

Abstract

Video generation models can produce high-quality, realistic videos, yet they often violate basic physical laws. Existing work largely treats these failures as evidence of missing capabilities that must be added through physical supervision, simulators, or additional training. In this paper, we find video generators capable of producing physically better videos already exist within pretrained video models. We can find those physics experts by applying random masks in the video models. Therefore, the physical accuracy of video generators can be improved without training. Across two model families, we also observe that at a fixed masking rate, these physics experts become denser as model scale increases. In practice, they can be selected without ground-truth videos, reused across scenes and obtained with comparable visual quality. Our results offer a new perspective on physical failures in video generation: physical capabilities may not be absent from pretrained models, but rather distributed across the diverse subnetworks they contain.

Primary area:
generative models
Keywords:
Video generation, Physical consistency, Training-free adaptation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_acAGOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper asks whether pretrained video generators contain subnetworks with stronger physical behavior than the full model. It samples structured masks over attention heads and feed-forward neurons, measures solution density across model families and scales, and selects reusable masks using reference-free physics screeners. Selected subnetworks improve Physics-IQ and gravity metrics over generation-seed best-of-(N) and a matched-compute baseline, transfer across scenes and seeds, and preserve perceived visual quality. Additional analyses identify category specialists and generalists and attribute gravity improvements to specific attention heads.

Critical strengths
  • Structured masking offers a novel, training-free way to expose and reuse physically stronger generator subnetworks.
  • Matched budgets, held-out scenes, fresh seeds, component interventions, and blinded human studies provide unusually thorough validation.
Critical weaknesses
  • The term “physics experts” overstates what the experiments establish. Broad masking may improve learned metrics through redundancy, regularization, reduced motion, trackability, or generic distribution shifts rather than exposing localized physical knowledge. Comparable visual quality does not exclude these alternatives, and causal head attribution is demonstrated only for gravity in one model.

  • The scaling and robustness evidence remains underdetermined. Each checkpoint uses one mask pool and only three deployment runs; model size co-varies with architecture and dropout tolerance, while Seed BoN reuses one fixed pool. Hyperparameter sweeps and benchmark-best baseline variants lack nested testing or multiplicity correction, and Cosmos gravity gains reverse at K=200K=200.

Questions for authors
  • After controlling for generic fidelity, temporal smoothness, motion magnitude, V-JEPA predictability, and tracking success, do the overall and category-specific mask advantages remain? Please include unmasked and population-level controls rather than only selected outputs.

  • Does the scale trend persist across independently resampled mask pools and masking-rate sweeps that equalize absolute removed units, output perturbation, or generic-quality degradation? Please provide uncertainty estimates for the resulting solution densities.

  • Can Seed BoN be repeated with independent seed pools paired to each mask-selection run, with hierarchical confidence intervals over candidate pools, generation seeds, and scenario clusters?

  • Can the component-attribution result be reproduced across fresh seeds, both model families, and additional physical domains, while reporting gravity error conditional on successful tracking?

5 comments

We thank the reviewer for the constructive and thorough feedback. We address each point below.

W1-Q1: Do the overall and category-specific mask advantages remain after controlling for fidelity, smoothness, motion, V-JEPA predictability and tracking?

A-Q1: Yes. We go through the five factors one by one.

Generic fidelity. In Physics-IQ, only the MSE term measures generic fidelity. It compares pixels with the real video. The other three terms score where, when and how much things move. Our lead over Seed BoN comes from these three terms, and the MSE term adds almost nothing (Rebuttal Table 10). Without the MSE term, the population-level solution density also stays the same (Rebuttal Table 11).

Rebuttal Table 10: Physics-IQ lead by term.

Physics-IQ gain over Seed BoN ↑Wan2.2-A14BCosmos3-Super
Spatiotemporal IoU+0.54+2.00
Spatial IoU+1.03+0.01
Weighted spatial IoU+1.12+0.10
MSE [95% CI]+0.05 [+0.00, +0.10]−0.04 [−0.09, +0.00]
Total (Table 1 of the paper, mean of 3 runs)+2.75+2.08

Rebuttal Table 11: Solution density without MSE.

Solution density ↑ [95% CI]Wan2.2-A14BCosmos3-Super
Full Physics-IQ (as in the paper)78% [69, 85]58% [48, 67]
Physics-IQ without the MSE term78% [69, 85]59% [49, 68]

Temporal smoothness. We measure it with VBench's motion smoothness and temporal flickering on all 198 videos of each of the 100 masks and 100 unmasked seeds per family. A score difference of 0.001 is about a quarter of one gray level per pixel. So the differences between masks and seeds are tiny (Rebuttal Table 12).

Rebuttal Table 12: Temporal smoothness.

Masks − seeds [95% CI]Wan2.2-A14BCosmos3-Super
VBench motion smoothness ↑ (0–1)+0.00062 [+0.00052, +0.00072]−0.00012 [−0.00020, −0.00004]
VBench temporal flickering ↑ (0–1)+0.00119 [+0.00104, +0.00134]+0.00008 [−0.00003, +0.00017]

Motion magnitude. Physics-IQ does not reward moving more or moving less. It checks whether the video moves where and when the real recording moves, and stays still where the recording is still. Physics-IQ Verified (Rädsch et al., 2026) further removes motion in the real recordings that has nothing to do with the physical event being tested, so that only the relevant motion is scored. On Verified, our generator also beats Seed BoN (Rebuttal Table 13).

Rebuttal Table 13: Physics-IQ Verified.

Physics-IQ Verified ↑Wan2.2-A14BCosmos3-Super
Seed BoN34.8540.35
Ours (mean of 3 runs)36.7641.56
Solution density, 100 masks [95% CI]82% [73, 88]59% [49, 68]

V-JEPA predictability. We measure it with V-JEPA 2 surprise, the score our reward model uses (lower = more predictable), on all 198 videos of each of the 100 masks and 100 unmasked seeds per family. Masks and seeds are about equally predictable (Rebuttal Table 14), so the comparison is fair to both sides.

Rebuttal Table 14: V-JEPA 2 surprise.

V-JEPA 2 surprise ↓Wan2.2-A14BCosmos3-Super
100 unmasked seeds0.4110.406
100 masks0.4080.406
Masks − seeds, relative to seeds [95% CI]−0.5% [−0.7%, −0.4%]+0.1% [−0.1%, +0.3%]

Tracking success. It only affects free fall, since Physics-IQ uses no tracker. Across all 200 masks and 200 unmasked seeds per family, on the balls that are tracked, masks are still more accurate than seeds (Rebuttal Table 15). Fidelity, smoothness and V-JEPA predictability cannot change the free-fall score in any other way, because the score uses only the ball's motion.

Rebuttal Table 15: Free fall on tracked balls.

Free-fall log-RMSE ↓, tracked balls onlyWan2.2-A14BCosmos3-Super
200 unmasked seeds1.1640.783
200 masks1.0330.756
Masks − seeds [95% CI]−0.131 [−0.192, −0.067]−0.027 [−0.075, +0.022]

References: Rädsch et al., Physics-IQ Verified (arXiv 2026).

W1-Q4: Does component attribution replicate with fresh seeds, in both families and in more domains, with gravity error conditional on tracking?

A-Q4: Yes. We repeated the head intervention of App. E.1 at four new seeds, on solid mechanics and fluids (the two largest Physics-IQ categories) and on free fall, in both Wan2.2 and Cosmos. For free fall, we report the gravity error conditional on tracking, that is, only on the scenes tracked in both models being compared.

Rebuttal Table 16: All six differences favor the attributed heads. Results are at four new seeds.

Gain over removing 16 random heads ↑ [95% CI]Wan2.2-A14BCosmos3-Super
Physics-IQ score, solid mechanics+3.32 [+0.29, +6.73]+3.54 [+0.59, +7.12]
Physics-IQ score, fluids+2.63 [+0.26, +5.16]+1.14 [−0.57, +3.56]
Free-fall log-RMSE reduction, tracked scenes+0.050 [−0.005, +0.114]+0.110 [+0.043, +0.171]

W2: Robustness of the scaling and selection results

A2: The scaling trend persists across independently resampled mask pools and matched masking rates (Q2). Our advantage largely persists when Seed BoN is repeated with independent seed pools (Q3). We answer the remaining two points here.

Nested testing and multiplicity. K = 16 and M = 200 were fixed before experiment. The sweeps in Tables 15 and 16 of the paper only measure sensitivity, and no setting was chosen from them. In these sweeps, Ours has lower error than Seed BoN in settings with K < M on both models, so the direction of the result does not depend on which setting is reported. We take "benchmark-best baseline variants" mentioned by you to mean Proprio, since it's our only baseline with variants. To keep our comparison fair, we ran its three scoring variants and report the best one for each model and benchmark. Global scoring is Proprio's best variant for both models on both benchmarks (Table 5 of the paper). Ours, without such preselection, still beats this best variant.

Cosmos at K = 200. The sign changes only when K equals the pool size M. The comparison is then between the consensus votes of the two whole pools. Our advantage comes from a higher density of experts in the mask pool, not from the whole pool being better, which is why Ours is not necessarily ahead at K = M.

Q2: Does the scale trend hold across resampled mask pools and masking rates matched on removed units, output perturbation or quality loss?

A-Q2: Yes, the scale trend persists across independently resampled mask pools and masking-rate sweeps.

Q2.1. Independently resampled mask pools. For each model we compare three pools of 50 masks. Each model's density is similar across pools, and the order 5B < A14B and Edge < Nano < Super holds in every pool.

Rebuttal Table 17: Independently resampled mask pools.

Solution density ↑ [95% CI]Pool 1Pool 2Pool 3
Wan2.2-5B2% [0, 10]0% [0, 7]0% [0, 7]
Wan2.2-A14B74% [60, 84]82% [69, 90]72% [58, 83]
Cosmos3-Edge6% [2, 16]8% [3, 19]10% [4, 21]
Cosmos3-Nano32% [21, 46]22% [13, 35]28% [18, 42]
Cosmos3-Super58% [44, 71]58% [44, 71]54% [40, 67]

Q2.2. Masking-rate sweeps.

(a) Equal absolute removed units. Removing as many units from the smaller model as from the larger one hurts the smaller model more than the paper's percentage masks do.

Rebuttal Table 18: Equal absolute removed units.

Solution density ↑ [95% CI]Larger modelSmaller model, same units as the larger
Wan2.2: A14B vs 5B78% [69, 85]0% [0, 11]
Cosmos3: Super vs Nano58% [48, 67]0% [0, 11]
Cosmos3: Nano vs Edge27% [19, 36]0% [0, 11]

(b) Equal output perturbation. We measure perturbation as the LPIPS distance (Zhang et al., 2018) between a masked video and the unmasked video from the same seed. With the paper's masks, 5B is perturbed more than A14B. We adjust the masking rates until the perturbation matches. This raises 5B's density, but the larger model stays far ahead in both families.

Rebuttal Table 19: Equal output perturbation.

ModelPerturbation (LPIPS)Solution density ↑ [95% CI]
Wan2.2-A14B0.10078% [69, 85]
Wan2.2-5B0.09823% [12, 41]
Cosmos3-Super≈ 0.1037% [22, 55]
Cosmos3-Nano≈ 0.106% [2, 16]

Cosmos3-Edge cannot get down to this perturbation level, because the mask rule removes at least one of its 16 heads in every layer.

(c) Equal generic-quality degradation. With the paper's masks, VBench quality changes very little for every model, about as much as a change of seed. So the paper's scale trend is already measured at equal quality loss.

Rebuttal Table 20: Equal generic-quality degradation.

Masked − unmasked (same seed)VBench imaging quality ↑VBench aesthetic quality ↑
Wan2.2-5B−0.009−0.002
Wan2.2-A14B−0.005−0.007
Cosmos3-Edge+0.013−0.003
Cosmos3-Nano+0.002+0.001
Cosmos3-Super−0.002−0.003

References: Zhang et al., The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (CVPR 2018).

Q3: Can Seed BoN be repeated with independent seed pools?

A-Q3: Yes. On Physics-IQ, we repeat Seed BoN with three independent pools of 100 seeds per model, one for each mask-selection run.

Rebuttal Table 21: Seed BoN with independent seed pools.

Performance gain over Seed BoN ↑Wan2.2-A14BCosmos3-SuperBoth models
Physics-IQ [hierarchical 95% CI]+2.58 [−0.36, +5.86]+1.51 [−2.02, +5.28]+2.04 [−0.30, +4.66]

The intervals resample scenario clusters, generation seeds with their paired pools, and the candidates in each pool, rerunning the selection each time. Ours wins in all six runs.

Reviewer_vBveOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Physics Thickets, a training-free approach to improving the physical consistency of video generation models. By randomly masking a subset of attention heads and feed-forward neurons in pretrained video generators, the authors discover subnetworks with specialized physical capabilities in domains such as solid mechanics, fluid dynamics, and optics. Experiments across Wan2.2 and Cosmos3 show that the proportion of high-performing subnetworks generally increases with model scale. By selecting subnetworks using reference-free video scoring or physics-based metrics, the proposed method improves physical accuracy while maintaining comparable visual quality. The authors further demonstrate that some selected subnetworks retain their advantages across different generation seeds and unseen scenarios.

Critical strengths
  1. The paper proposes a novel training-free approach to improving the physical consistency of video generation by identifying specialized subnetworks through structured masking, without updating model parameters.
  2. The proposed method outperforms generation-seed selection on Physics-IQ under comparable inference budgets, with further experiments demonstrating improvements in physical plausibility while maintaining comparable visual quality.
  3. Experiments across two model families show that the density of high-performing subnetworks generally increases with model scale, providing evidence that larger models contain more readily accessible physical expertise.
Critical weaknesses
  1. The paper provides limited mechanistic insight into why masking certain attention heads and feed-forward neurons improves physical consistency, leaving the underlying causes of the discovered expertise insufficiently understood.
  2. The effectiveness of the approach depends on the candidate selection strategy. The reference-free video reward model may not perfectly align with physical correctness, and the method does not consistently outperform existing approaches across all evaluated benchmarks.
  3. The experimental evaluation is limited to five model variants and a relatively narrow set of physical benchmarks. Further validation on more diverse real-world scenarios and complex physical interactions is needed to establish broader generalizability.
  4. The qualitative results do not demonstrate a sufficiently clear visual advantage over the baseline. Although the paper presents examples of improved physical behaviors, the differences are not consistently compelling, making it difficult to assess the practical significance of the proposed method from visual comparisons alone.
Questions for authors

N/A

3 comments

We thank the reviewer for the constructive feedback. We address each point below.

W1: Why does masking certain heads and neurons improve physics?

A1: One theoretical intuition is that masking heads and FFN neurons in the early denoising steps changes the motion plan of the video, and some masks switch the model to a more physical plan than the one the base model picks. Recent work shows that video diffusion models fix the motion plan in the first few denoising steps, and later steps mostly change visual details (Newman et al., 2026). At these earlier steps, a few attention heads decide how objects move, and several possible plans are still competing (Li et al., 2026). We mask heads and FFN neurons only in the early, high-noise steps, so a mask can switch the model to a different plan.

The base model does not always pick the most physical plan, because its training loss is mostly about pixels and says little about motion (Chefer et al., 2025). Masks that switch it to a more physical plan give the better subnetworks we find.

Larger models have more of these subnetworks because they lose less when units are removed (Shevchenko & Mondelli, 2020), so a mask can change the plan without breaking the rest of the video. Smaller models have fewer. A small model has to pack many features into each unit, and features that matter little to the loss, like motion, end up sharing units with others (Elhage et al., 2022). So a mask that changes the motion also breaks other things.

References: Newman et al., Video Models Reason Early (arXiv 2026); Li et al., Why Do Video Diffusion Models Violate Physics? (arXiv 2026); Chefer et al., VideoJAM (ICML 2025); Shevchenko & Mondelli, Dropout Stability of SGD Solutions (ICML 2020); Elhage et al., Toy Models of Superposition (arXiv 2022).

W2.1: The approach depends on the candidate selection strategy, and the reference-free reward model may not perfectly align with physical correctness.

A2.1: Yes, the effectiveness of our method depends on the selection strategy, and a better strategy would further improve it. We use two kinds of reward, which together cover most tasks. For general physical plausibility, we use WMReward (Yuan et al., 2026). It identifies physical plausibility across rigid-body dynamics, fluids, buoyancy, gravity, conservation of mass and optics, and the human study also confirms it. Best-of-N with WMReward raises the Physics-IQ score of the generators we tested, consistent with the results in their paper. For tasks with a precise physical quantity, we measure that quantity in each video and use how close it is to the correct value as the reward. On free fall, for example, we track the ball and measure gravity from its fall.

References: Yuan et al., Inference-time Physics Alignment of Video Generative Models (CVPR 2026).

W2.2: The method does not consistently outperform existing approaches across all evaluated benchmarks.

A2.2: In the paper, we use a small population size due to compute constraints. This understates our gain, because the performance gain has a positive correlation with the population size (see Rebuttal Table 6). We double the population size from 200 to 400, and Ours beats both Seed BoN and Proprio with the same compute. With this, our gain is consistent across all model families and benchmarks we test, and the performance gain increases as the population grows.

Rebuttal Table 6: There is a positive correlation between population and performance gain. Ours is averaged over three runs.

Free-fall log-RMSE ↓ (Cosmos3-Super)Seed BoNProprioOurs
M = 500.7030.6970.716
M = 2000.7290.6600.687
M = 4000.6910.6800.647

W3.1: Five models in two families are not enough.

A3.1: We choose Wan2.2 (from Alibaba, the company behind Qwen) and Cosmos3 (from NVIDIA) in the paper because they are representative video generators. To address your concern, we add two more models. HunyuanVideo 1.5 (Wu et al., 2025) is Tencent's latest open-source video model, and LTX-2 (HaCohen et al., 2026) is a widely used open model from Lightricks. For both models, with a population of 50 candidates, the subnetwork our method selects beats Seed BoN on Physics-IQ.

Rebuttal Table 7: Ours beats Seed BoN on Physics-IQ in two more model families.

Physics-IQ ↑HunyuanVideo 1.5LTX-2
Seed BoN32.3331.73
Ours33.5032.70

Together with our results on Wan2.2 and Cosmos3, our method shows performance gains over Seed BoN in all four model families, which are among the most widely used open video generators.

More on why we choose Wan2.2 and Cosmos3: Together they cover general-purpose video generation (Wan2.2) and physical AI such as robotics and autonomous driving (Cosmos3). Both are open families released at several sizes, which our model-size analysis needs. Using one or both of these families to test research hypotheses is standard practice (e.g., Huang et al., 2026; Zheng et al., 2026); we use both to show that our method generalizes.

References: Wu et al., HunyuanVideo 1.5 Technical Report (arXiv 2025); HaCohen et al., LTX-2 (arXiv 2026); Huang et al., PhyMotion (NeurIPS 2026); Zheng et al., Score-Regularized Continuous-Time Consistency (ICLR 2026).

W3.2: More benchmarks with diverse real-world scenarios and complex physical interactions are needed.

A3.2: To address your concern, we added two benchmarks which are built for testing diverse real-world scenarios and physical interactions. PisaBench (Li et al., 2025) tests how a variety of everyday objects fall in real-world scenes. Principia (Thozhiyoor et al., 2026) tests whether two objects in a real scene obey the same physical law. We use its restitution (bouncing) and spring (elastic force) tasks. For both Wan2.2-A14B and Cosmos3-Super, Ours selects from 200 masks and Seed BoN from 200 seeds.

Rebuttal Table 8: Ours beats Seed BoN on all three PisaBench metrics in both models.

PisaBenchWan2.2-A14B L2 ↓Wan2.2-A14B Chamfer ↓Wan2.2-A14B IoU ↑Cosmos3-Super L2 ↓Cosmos3-Super Chamfer ↓Cosmos3-Super IoU ↑
Seed BoN0.1150.2920.08670.1400.3890.0783
Ours0.1050.2560.08790.1370.3820.0805

Rebuttal Table 9: Ours beats Seed BoN on Principia's restitution and spring tasks in both models.

PrincipiaWan2.2-A14B restitution ↑Wan2.2-A14B spring ↑Cosmos3-Super restitution ↑Cosmos3-Super spring ↑
Seed BoN0.5990.1320.2240.089
Ours0.6200.2780.3060.151

Together, these results show that our method is better than Seed BoN on all four benchmarks, covering diverse real-world scenarios and complex interactions.

More details on the benchmarks

The two existing benchmarks in our paper cover many aspects of physical plausibility, one broadly and one precisely. Physics-IQ is the broad test: a widely used benchmark of diverse real-world scenarios with complex physical interactions, across five areas of physics (solid mechanics, fluid dynamics, optics, thermodynamics and magnetism). The free-fall benchmark is the precise test: it measures the acceleration of falling objects and checks it against Earth's gravity.

PisaBench tracks the dropped object in the generated video and in the real video and compares them frame by frame. L2 and Chamfer measure how far the generated object is from the real one (L2 between their centers, Chamfer between their outlines). IoU measures how much the two objects overlap, on frames in which the real object is detected.

Principia's restitution task measures how high two identical balls bounce after being dropped from different heights. Its spring task measures how far two identical springs stretch under two different masses. The score measures how well the two objects agree as the physical law requires.

References: Li et al., PISA Experiments (ICML 2025); Thozhiyoor et al., Principia (NeurIPS 2026).

W4: Visibility of the improvements in the qualitative results

A4: Our method improves the physical correctness of the videos, not their general visual quality, so the improvement may not stand out at first glance. On a closer look, the physical errors it removes are clear. For example, Fig. 1 shows:

  • Optics. In the original video, the teapot in front of the mirror shows two handles, while its reflection shows one handle and a spout. The mirror does not reflect the teapot as it is, which violates the law of reflection. With the optics expert, both the teapot and its reflection show one handle and one spout.
  • Solid mechanics. A stick on a turntable knocks over a row of dominoes. In the original video, the single purple domino turns into three purple blocks, although a rigid domino cannot break apart like this. With the mechanics expert, each domino pushes the next one over and stays one rigid block.
  • Fluid dynamics. Red liquid is poured onto a rubber duck in a tray. In the original video, the duck crumples and loses its beak, although liquid poured over it cannot change its shape. With the fluids expert, the liquid runs down the duck and fills the tray, and the duck keeps its shape.
Reviewer_BgAvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents an empirical finding in large pretrained video generation models where applying structured masking to the model's weights can yield subnetworks that outperform the base model on Physics-IQ tasks and demonstrate improved physical correctness at matched video fidelity. These subnetworks are more prevalent in larger models.

Critical strengths
  • Interesting empirical finding: Random structured masking can improve physical plausibility without additional training, with a higher prevalence of beneficial subnetworks in larger models.

  • Consistent experimental evidence: Results across five models from two families, with improvements generalizing across generation seeds and held-out scenarios, including evidence of domain specialization.

  • Practical applicability: The approach outperforms seed-based selection on physics-related metrics while maintaining comparable visual quality, offering a training-free alternative.

  • Clear presentation: The paper is well-written, clearly structured, and easy to follow.

  • Generally sound methodology: The experimental design is reasonable, with appropriate baseline comparisons and several robustness checks.

Critical weaknesses
  • Limited understanding of the underlying mechanism: the paper does not offer an explanation of the mechanism that causes the observed phenomenon. Why masking reveals such subnetworks? why they exist? how training produces them? why the base model performs worse than the subnetworks within it?
  • Limited technical novelty: The method combines established techniques (random masking and best-of-N selection) without substantial algorithmic innovation. Its novelty lies primarily in the empirical findings, whose underlying mechanisms remain unclear.
  • Overstated claims: Five models across two families provide encouraging evidence, but are insufficient to establish general claims about scaling and the existence of physics experts in general.
  • The paper does not address Physics-IQ Verified [1], which identifies shortcomings in its primary benchmark. Independent evaluation would strengthen the findings. [1] Rädsch, T., Asano, Y. M., Kuehne, H., Bauer, S., Jaini, P., Geirhos, R., & Lüth, C. T. (2026). Physics-IQ Verified. arXiv preprint arXiv:2606.18943.
  • Reproducibility: No open code.
Questions for authors
  • Have you checked conjectures of pruning and generalization as the machanism behind the observed phenomena? Have you checked that literature? Have you investigated other possible underlying mechanisms that could explain the observed phenomena?
2 comments

We thank the reviewer for the constructive feedback. We address each point below.

W1: What's the underlying mechanism?

A1: One theoretical intuition is that masking heads and FFN neurons in the early denoising steps changes the motion plan of the video, and some masks switch the model to a more physical plan than the one the base model picks.

  • Why masking reveals such subnetworks. Recent work shows that video diffusion models fix the motion plan in the first few denoising steps, and later steps mostly change visual details (Newman et al., 2026). At these earlier steps, a few attention heads decide how objects move, and several possible plans are still competing (Li et al., 2026). We mask heads and FFN neurons only in the early, high-noise steps, so a mask can switch the model to a different motion plan.
  • Why they exist. Since several plans are still competing at these steps, a more physical plan is often only a small change away, and masks that switch to it give the better subnetworks we find. Larger models have more of them because they lose less when units are removed (Shevchenko & Mondelli, 2020). A small model has to pack many features into each unit, and features that matter little to the loss, like motion, end up sharing units with others (Elhage et al., 2022), so a mask that changes the motion also breaks other things.
  • How training produces them, and why the base model is worse. Pretraining minimizes a loss that is mostly about pixels and says little about motion (Chefer et al., 2025). It does not push the plan toward real physics, so the base model's plan is not always the most physical one, and some of its subnetworks pick a better one.

References: Newman et al., Video Models Reason Early (arXiv 2026); Li et al., Why Do Video Diffusion Models Violate Physics? (arXiv 2026); Shevchenko & Mondelli, Dropout Stability of SGD Solutions (ICML 2020); Elhage et al., Toy Models of Superposition (arXiv 2022); Chefer et al., VideoJAM (ICML 2025).

Q1: Have you examined pruning and generalization as the mechanism?

A-Q1: Yes. We use pruning theory in W1 to explain why larger models have more of these subnetworks. A mask helps only when the gain from a better motion plan is larger than the loss from the removed units. Studies of pruning and dropout show that larger networks lose less when units are removed (Shevchenko & Mondelli, 2020), so more of their masks help. The work on pruning and generalization studies pruning followed by training, where pruning acts as a regularizer (Bartoldson et al., 2020). We do not train the masked model, so we think this is less likely to be the main reason here.

References: Bartoldson et al., The Generalization-Stability Tradeoff in Neural Network Pruning (NeurIPS 2020).

W2: The algorithm is too simple and needs more innovation.

A2: It is worth noting that our main claim is a new finding: a pretrained video generator already contains many subnetworks that are better at physics than the base model. Based on this finding, we carefully choose the algorithm, random masking with best-of-N, because it makes good use of these many subnetworks and is comparable to or better than more complex methods.

Applying random masks to the base model often improves its performance, which shows that better subnetworks are dense around the pretrained model. Best-of-N leverages these diverse experts. Random masking followed by best-of-N (Ours) beats Seed BoN on both model families and both benchmarks (Tables 1 and 2 in the paper). In particular, on Physics-IQ solid-mechanics scenes, Ours gains 4.28 points over Seed BoN on Wan2.2-A14B and 4.07 on Cosmos3-Super.

Many methods improve physics in video generators by adding simulators, physical supervision or more training (Liu et al., 2024; Pathak et al., 2026). To our knowledge, our results are the first in video generation to show that subnetworks of a pretrained model can generate physically better videos than the base model, and that they can be used without training.

References: Liu et al., PhysGen (ECCV 2024); Pathak et al., PhysVid (CVPR 2026).

W3: Five models in two families are not enough.

A3: We choose Wan2.2 (from Alibaba, the company behind Qwen) and Cosmos3 (from NVIDIA) in the paper because they are representative video generators. To address your concern, we add two more models. HunyuanVideo 1.5 (Wu et al., 2025) is Tencent's latest open-source video model, and LTX-2 (HaCohen et al., 2026) is a widely used open model from Lightricks. For both models, with a population of 50 candidates, the subnetwork our method selects beats Seed BoN on Physics-IQ.

Rebuttal Table 4: Ours beats Seed BoN on Physics-IQ in two more model families.

Physics-IQ ↑HunyuanVideo 1.5LTX-2
Seed BoN32.3331.73
Ours33.5032.70

Together with our results on Wan2.2 and Cosmos3, our method shows performance gains over Seed BoN in all four model families, which are among the most widely used open video generators.

More on why we choose Wan2.2 and Cosmos3: Together they cover general-purpose video generation (Wan2.2) and physical AI such as robotics and autonomous driving (Cosmos3). Both are open families released at several sizes, which our model-size analysis needs. Using one or both of these families to test research hypotheses is standard practice (e.g., Huang et al., 2026; Zheng et al., 2026); we use both to show that our method generalizes.

References: Wu et al., HunyuanVideo 1.5 Technical Report (arXiv 2025); HaCohen et al., LTX-2 (arXiv 2026); Huang et al., PhyMotion (NeurIPS 2026); Zheng et al., Score-Regularized Continuous-Time Consistency (ICLR 2026).

W4: Do the results hold on Physics-IQ Verified?

A4: Yes. We re-scored our results with the official Physics-IQ Verified code. In both models, all three runs beat Seed BoN.

Rebuttal Table 5: Ours beats Seed BoN when scored with Physics-IQ Verified. Ours is averaged over three runs.

Physics-IQ Verified ↑Wan2.2-A14BCosmos3-Super
Seed BoN34.8540.35
Ours36.7641.56

W5: Reproducibility

A5: We will release the code, all mask configurations, and per-candidate scores together with the paper.

Reviewer_P2vhOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether structured masking can expose better physical behavior in pretrained video generators without training. Across Wan and Cosmos models, it studies the frequency and specialization of useful subnetworks, then selects masks through screening without reference videos. Physics-IQ, free-fall experiments, transfer tests, and human judgments provide evidence that some masks offer reusable improvements over generation-seed selection.

Critical strengths

This paper investigates whether structured masking can expose better physical behavior in pretrained video generators without training. Across Wan and Cosmos models, it studies the frequency and specialization of useful subnetworks, then selects masks through screening without reference videos. Physics-IQ, free-fall experiments, transfer tests, and human judgments provide evidence that some masks offer reusable improvements over generation-seed selection.

Critical weaknesses
  1. Reuse is demonstrated on new scenarios and seeds within the evaluated tasks, but portability across task distributions remains unclear. Physics-IQ and free-fall experiments use separate screening procedures. Whether a selected mask remains useful on a different benchmark without another search is an important open question for practical use.
  2. The practical gains are less consistent than the evidence that useful subnetworks exist. Human preference on Physics-IQ is modest, and masking does not consistently outperform Proprio in the Cosmos free-fall setting. The practical benefit appears task-dependent and currently less compelling than the broader empirical finding.
  3. Although the subnetwork results cover several physical domains, the targeted component-intervention experiment is limited to Wan-A14B on free fall. This supports the importance of particular heads in that setting, but the corresponding evidence for richer physical interactions remains limited. An intervention study on another type of dynamics would strengthen the interpretation of physical specialization.
Questions for authors

see weaknesses

1 comment

We thank the reviewer for the constructive feedback. We address each point below.

W1: Does a selected mask remain useful on a different benchmark?

A1: Yes, it remains useful when the two tasks share the relevant physics. For example, masks picked on solid mechanics transfer to free fall, while masks picked on optics, which is unrelated to free fall, do not. To show this, we took the top three masks on Physics-IQ's solid-mechanics scenes and the top three on its optics scenes, and ran them on free-fall tasks at new sampling seeds (3 masks × 3 seeds per group and model). Each number is how much the masks lower free-fall log-RMSE compared with the unmasked model, so higher is better.

Rebuttal Table 1: Masks picked on solid mechanics transfer to free fall, while masks picked on optics do not.

Free-fall log-RMSE reduction ↑Wan2.2-A14BCosmos3-SuperAverage
Solid-mechanics masks (from Physics-IQ)+0.049+0.084+0.066
Optics masks (from Physics-IQ)−0.202−0.085−0.144

W2: Concern about the consistency of practical gains across benchmarks

A2: In the paper, we use a small population size due to compute constraints. This understates our gain, because the performance gain has a positive correlation with the population size (see Rebuttal Table 2). We double the population size from 200 to 400, and Ours beats both Seed BoN and Proprio with the same compute. With this, our gain is consistent across all model families and benchmarks we test, and the performance gain increases as the population grows.

Rebuttal Table 2: There is a positive correlation between population and performance gain. Ours is averaged over three runs.

Free-fall log-RMSE ↓ (Cosmos3-Super)Seed BoNProprioOurs
M = 500.7030.6970.716
M = 2000.7290.6600.687
M = 4000.6910.6800.647

W3: Does the component intervention also hold for other kinds of dynamics?

A3: Yes. We repeated the head intervention of App. E.1 on solid mechanics and fluids (the two largest Physics-IQ categories), on both Wan2.2 and Cosmos3. The 16 heads were picked on one half of each category's scenes and tested on the other half, at four new seeds. Each number is the Physics-IQ score with these heads removed minus the score with 16 matched random heads removed.

Rebuttal Table 3: Removing the attributed heads gives a higher Physics-IQ score than removing 16 matched random heads.

Physics-IQ gain ↑Wan2.2-A14BCosmos3-Super
Solid mechanics+3.32+3.54
Fluids+2.63+1.14

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Rebuttal and Additional Findings

We thank all reviewers for their constructive feedback. We are glad to see that our method is "a novel training-free approach" to improving the physical plausibility of video generation (vBve, acAG). Reviewers also note that our experiments provide "consistent experimental evidence" and "unusually thorough validation" (BgAv, acAG), and that the paper is "clear and easy to follow" (P2vh, BgAv, vBve). We have answered all weaknesses and questions raised by the reviewers point by point. We summarize the main new results below.

  • We add more model families and benchmarks, and Ours shows consistent gains on all four model families and all four benchmarks. Besides the paper's two representative model families, Wan2.2 and Cosmos3, we add two widely used open video generators, HunyuanVideo 1.5 (Tencent) and LTX-2 (Lightricks). Besides the paper's two benchmarks, Physics-IQ (physics in diverse real-world scenarios) and free fall (the acceleration of falling objects), we add PisaBench (how everyday objects fall in real-world scenes) and Principia (whether objects in a real scene obey the same physical law). We also rescore Physics-IQ with Physics-IQ Verified, which scores only the motion relevant to the tested physical event. Ours outperforms Seed BoN on all of them. Our gain also grows with the population size, so our performance can be increased further with a larger population size. Furthermore, selected masks can also transfer to new tasks from other benchmarks that share the same physics.

  • We provide theoretical intuitions to explain the mechanism behind the physically better subnetworks in pretrained video generators. Specifically, we answer 1) why masking reveals these subnetworks, 2) why they exist, and 3) why the base model is worse than them (see W1 of our reply to Reviewer BgAv).

  • We add ablation experiments, which show that our gain comes cleanly from physical correctness and that our scaling evidence is robust. Our gain remains after controlling for generic fidelity, temporal smoothness, motion magnitude, V-JEPA predictability and tracking success, indicating that it reflects better physics rather than these factors. The scaling trend, that larger models contain more of these subnetworks than smaller ones, persists across resampled mask pools and matched masking rates.

Details are in our responses to each reviewer. We will add all new results to the revision and release our code, and we are happy to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.