- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper works on whether simple and non-photorealistic simulations can improve physical behavior in photorealistic video generation. The authors construct about 297K simulation clips covering object permanence, collision dynamics, and wave propagation. They fine-tune Cosmos3-Super using 30% simulation data and 70% real videos, with detailed captions describing the scenes and events. They evaluate specialist models and a joint model across simulation, synthetic, and photorealistic domains. The results show improvements on the three target tasks and Physics-IQ-Verified, while VBench++ scores remain similar to the baseline.
Critical strengths
S1. The paper addresses a useful question with a simple but efficient approach. Improving physical behavior without expensive photorealistic simulation or additional inference-time components is valuable. The results suggest that simple simulation data can improve the selected skills in photorealistic scenes. The method also keeps the standard image-to-video inference process.
S2. The experiments cover several complementary settings. The authors evaluate three physicial skills across three visual domains and compare specialist training with joint training. The joint model improves all nine task/domain combinations over real-only-SFT. The human comparison, confidence intervals, and failure analysis provide additional support for the observed improvements.
Critical weaknesses
W1. The idea of mixture training recipe is similar as Zhao et al., Synthetic Video Enhances Physical Fidelity in Video Synthesis (ICCV 2025). Zhao et al. already study synthetic-real video mixing, detailed captions, synthetic-domain tags, and the trade-off between physical fidelity and synthetic appearance. Several parts of the proposed recipe closely follow the same approach. This paper appears to extend an existing approach to additional physical scenarios, and I am not yet convinced that the novelty is sufficient.
Reference: Zhao et al., Synthetic Video Enhances Physical Fidelity in Video Synthesis, ICCV 2025. https://arxiv.org/abs/2503.20822
W2. The scope of the learned physical skills is not sufficiently verified. The VLM judge agrees well with human rankings, but agreement on score-gap magnitude is only moderate (Spearman 0.42–0.51; MAE 0.17–0.23). Robustness to another judge is also not reported, so the meaning of the numerical gains remains unclear. Generalization is mainly tested within the same activity families. Despite parameter variation in training, systematic tests on unseen physical properties, such as mass ratios, restitution, or rope tension, are limited. The recipe is also tested on only one backbone. These results support task-specific improvements, but provide limited evidence for broader physical generalization.
Questions for authors
Q1: Evaluation robustness. Results from another VLM judge, such as Qwen or Gemini, would help check whether the rankings and score gains are consistent. The number of human annotators and agreement between them also need clarification.
Q2: Physical generalization. The train/test split should show which object properties and physical parameters are unseen during training. Tests on new mass ratios, restitution values, or rope configurations would help separate visual transfer from physical generalization. The paper should also state whether hidden physical properties are given in the prompt.
Q3: Applicability. The recipe is tested on only Cosmos3-Super. Results on another video model would help show whether it works more generally.
4 comments
W2 (judge) & Q1: Robustness to another judge and human agreement.
1. Second VLM judge. We evaluated the same photorealistic videos of the five models in Table 2 (excluding the added Wan2.2 rows) with an independent VLM judge from another model family, Gemini 4 Argon (Argon), using the same rubrics. All rankings align with the original VLM judge (GPT): the five-model ranking is identical on every task (Kendall τ = 1 for object permanence under occlusion (OP), rigid-body collision dynamics (CD), and wave propagation along deformable ropes (WP)), and the gains of Sim-∞ over Pretrained remain significant on all three tasks (GPT: +0.18 / +0.49 / +0.59, 95% CIs [0.10, 0.25] / [0.45, 0.53] / [0.54, 0.62]; Argon: +0.11 / +0.42 / +0.46, 95% CIs [0.05, 0.18] / [0.39, 0.44] / [0.43, 0.50]; Table R7).
| Model | OP (GPT / Argon) | CD (GPT / Argon) | WP (GPT / Argon) |
|---|---|---|---|
| MiniMax-H3 | 0.73 / 0.62 | 0.53 / 0.41 | 0.74 / 0.46 |
| Gemini Omni Flash | 0.45 / 0.41 | 0.38 / 0.30 | 0.49 / 0.42 |
| Cosmos3-Super (Pretrained) | 0.54 / 0.48 | 0.28 / 0.23 | 0.17 / 0.11 |
| real-only-SFT | 0.48 / 0.44 | 0.32 / 0.25 | 0.29 / 0.19 |
| Sim-∞ | 0.72 / 0.59 | 0.77 / 0.65 | 0.76 / 0.57 |
Table R7: Both VLM judges scored the same videos from one seed (seed 16); the paper's Table 2 averages three seeds, so values differ from it.
2. Human annotators. We added a second independent human annotator, who rated the same 190 videos (95 scene pairs) blind to the first annotator and to all VLM judge scores. The two human annotators prefer the same video in every pair where both express a preference (24 of 30 OP, 29 of 30 CD, and 35 of 35 WP pairs), and their per-video scores agree closely (Krippendorff's α = 0.82 / 0.90 / 0.91 for OP / CD / WP).
3. The meaning of the numerical gains. The size of a single-pair gap is noisy for any two evaluators, including the two humans: the VLM judge scores each video independently, and gap size is inherently subjective. The moderate agreement between the VLM judge and a human on single-pair gap size in Appendix B.1 is at the same level as the agreement between the two humans (Table R8). The performance gains and conclusions of this paper, however, are drawn from scores averaged over all cases. At that level, the gains measured by both VLM judges are equal to or smaller than those measured by the human mean (Table R8, last column: 0.00 to −0.26).
| A vs. B (OP / CD / WP) | Same preferred video (non-tied pairs) | Single-pair gap: Spearman | Single-pair gap: MAE | Mean-gain difference (A − B) |
|---|---|---|---|---|
| Human 2 vs. Human 1 | 100 / 100 / 100% | 0.57 / 0.44 / 0.27 | 0.22 / 0.20 / 0.18 | −0.13 / −0.11 / −0.05 |
| VLM judge (GPT) vs. Human 1 (Appendix B.1) | 100 / 96 / 97% | 0.47 / 0.51 / 0.42 | 0.23 / 0.17 / 0.21 | −0.06 / −0.10 / −0.16 |
| VLM judge (GPT) vs. human mean | 95 / 97 / 97% | 0.56 / 0.66 / 0.29 | 0.20 / 0.14 / 0.21 | 0.00 / −0.04 / −0.13 |
| VLM judge (Argon) vs. human mean | 91 / 90 / 94% | 0.62 / 0.56 / 0.41 | 0.17 / 0.19 / 0.28 | −0.14 / −0.05 / −0.26 |
Table R8: Agreement between evaluators. Mean gain: an evaluator's average score gain of Sim-∞ over real-only-SFT across the 95 pairs; the last column is A's mean gain minus B's. Human mean: the average of the two human annotators' scores.
We have added this analysis to Appendix B.1 (marked in red). The remaining points of W2 (unseen physical properties and broader physical generalization; a single backbone) are addressed in Q2 and Q3.
W2 (unseen physical properties) & Q2: Unseen properties and physical information in the prompt.
1. Train/test split. Every photorealistic evaluation case contains configurations that never appear in simulation training (Table R9).
| Skill | Simulation training | Photorealistic evaluation (cases) |
|---|---|---|
| Object permanence under occlusion (OP) | One ball size per scene: diameter ratio 1.0, equal masses | Different ball sizes and masses in a scene, diameter ratio 1.8–3.7 (60/100); configurations absent from all training rollouts (37/100); ball fully hidden at frame 0 (3/100) |
| Rigid-body collision dynamics (CD) | 40–60 cm balls; 2.2 m Newton's cradle | 2–24 cm everyday balls; tabletop Newton's cradle (300/300) |
| Wave propagation along deformable ropes (WP) | Rope driven by a rigid controller | Rope swung by a person (178/178) |
| All | Plain materials and floor | Real objects and surfaces (all cases) |
Table R9: Simulation training versus photorealistic evaluation. Sizes are the values requested when constructing the first frames. We have added this table to Appendix A.3 (marked in red).
2. Physical generalization. The photorealistic scenes use real objects, materials, and physical scales, so material-dependent properties such as restitution and friction differ from those in simulation, although we do not measure their exact values. On these scenes, Sim-∞ still improves OP, CD, and WP by +0.28, +0.47, and +0.50 over real-only-SFT (Table 2), so the learned skills generalize to real objects of different materials and to physical scales absent from simulation. Where a hidden property is needed to define the event (e.g., the ball-to-block mass ratio, the rope's wave speed), the prompt states it; material-dependent properties such as restitution and friction are not given in the prompt.
W2 (single backbone) & Q3: Applicability to another video model.
We applied the same recipe to Wan2.2-TI2V-5B (Team Wan et al., 2025), a 5B model from a different family, without backbone-specific tuning: the same simulation data, the same real-video pool, the same joint training on all three skills, the same number of steps (7,500), and our 480p, 189-frame setting rather than Wan's native 720p and 121 frames. Over Pretrained Wan2.2-TI2V-5B, Sim-∞ improves all three skills by +0.41 / +0.46 / +0.58 on OP / CD / WP and Physics-IQ-Verified by +3.64 (33.40 to 37.04), while VBench++ stays at 89.43 vs. 89.10 (Table R10).
| Model | OP | CD | WP | Physics-IQ-Verified | VBench++ (overall) |
|---|---|---|---|---|---|
| Wan2.2-TI2V-5B (Pretrained) | 0.20 | 0.12 | 0.11 | 33.40 | 89.10 |
| + Sim-∞ | 0.61 | 0.58 | 0.69 | 37.04 | 89.43 |
Table R10: Photorealistic physics scores (OP/CD/WP) are averaged over three seeds, as in Table 2. We have added these results to Table 2 (marked in red).
Reference
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.
W1: The pipeline is shared; the task and research question are not, and Zhao et al.'s results do not predict ours.
Thank you for taking the time to review our paper and for your feedback and suggestions. We address each weakness and question below.
W1: The pipeline is shared; the task and research question are not, and Zhao et al.'s results do not predict ours.
Synthetic–real mixing, detailed captions, and domain tags describe a broad pipeline used across synthetic-data fine-tuning (e.g., Zhao et al., 2025; Bai et al., 2025). A shared pipeline does not carry over across tasks by default: in Zhao et al.'s own experiments, low-quality assets or low-cost rendering sharply reduce transfer (their Table 2).
1. Task and research question. Zhao et al. ask whether high-fidelity CGI improves text-to-video human-pose integrity, 3D consistency under camera orbits, and layer decomposition, with a single object per scene (their Sec. 2), leaving multi-object interactions to future work (their Sec. 6). We ask whether simple, non-photorealistic simulation, which their Table 2 would predict to fail, improves physics (e.g., object permanence under occlusion (OP), rigid-body collision dynamics (CD), and wave propagation along deformable ropes (WP)) in photorealistic image-to-video generation; our experiments show that it does.
2. Step-by-step differences. The pipeline is shared, but each step is implemented differently from Zhao et al. (Table R6):
Table R6: Step-by-step comparison with Zhao et al. (2025).
References
Qi Zhao, Xingyu Ni, Ziyu Wang, Feng Cheng, Ziyan Yang, Lu Jiang, and Bohan Wang. Synthetic video enhances physical fidelity in video synthesis. In Proceedings of the International Conference on Computer Vision (ICCV), 2025.
Jianhong Bai, Menghan Xia, Xintao Wang, Ziyang Yuan, Zuozhu Liu, Haoji Hu, Pengfei Wan, and Di Zhang. SynCamMaster: Synchronizing multi-camera video generation from diverse viewpoints. In International Conference on Learning Representations (ICLR), 2025.