- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work deals with the problem of generateing visually impressive and physically plausible videos of phase transitions from image(s). The authors build a pipeline, combining the abilities of vlm reasoning, physical simulation, and video diffusion models: vlm decides the physical process and final visual results, physical simulators update the physical states based on the reconstructed 3D scenes, video diffusion models improve the visual details to make the results more realistic. The main contributions are the integration of approximate phase-transition simulation and the temporal Gaussian mapping mechanism. Authors compare their methods with several video generation baseline models and WonderPlay, qualitative and quantitative results are competitive.
Critical strengths
Video generation of phase transitions is a hard problem. The presented results in this work demonstrate that the proposed pipeline can deal with the phase change and produce visually convincing results.
Critical weaknesses
- The claimed physical grounding is weak from the thermal part. The paper neglects the modeling of thermal effects like temperature evolution and heat transfer. Consequently, it's difficult for the method to model spatially varying transitions and boundary conditions, such as melting that begins near a localized heat source.
- The ablation of the method is sufficient. Authors didn't show the intermediate results of each pipeline components, e.g., the generated rgb before vdm refinement and how the gaussian temporal updater makes effects.
- The video model baselines authors compared in the paper are weak. The authors should test several much better methods, like Runway4.5, Seedance2.5.
Questions for authors
- Could the authors clarify the amount of manual configuration and tuning required to produce the reported results? In particular, which simulation parameters are inferred automatically, and which must be specified or adjusted for each scene?
- The anonymous link in the paper is expired. Could the authors provide a working link to the video results?
- Regarding the writing, the authors spend a lot of efforts connecting this method with "World model", which i see as irrelevant, what this work tries to deal with is just a specific video generation task.
- Although the solver formulation includes a temperature field in 3.2, the described phase-transition mechanisms mainly use time-dependent schedules for dynamic parameters like stiffness and damping. Could the authors clarify how temperature is updated and are these time-dependent schedules prescribed per scene or derived from thermal conditions?
- How does the method handle the gap between the simulated geometry and the AI-generated target state? Since optimal transport establishes spatial correspondences, could the authors discuss or demonstrate the method’s robustness to such mismatches?”
5 comments
Thanks for you reply! According to you reply and revised manuscript, I have 2 follow-up questions:
-
Fidelity of the final video to the simulated motion. In some of the intermediate results in Appendix G like Fig.11/12, the motion in the final video appears to differ noticeably from that in the simulation render. To what extent does the video diffusion model override the simulated dynamics during refinement?
-
Reliability of AI-proposed simulation parameters. The response states that the reported examples used AI-proposed parameters without manual adjustment. Since several parameters jointly determine the simulated behavior, and their effective values in an approximate solver may differ from real-world material values, I doubt the reliablity of a one-shot vlm inference. Could the authors report the acceptance criterion or the first-attempt success rate across the evaluated scenes?
Thanks so much for your follow-up questions!
For your Q1, as we state in the paper, we use the same pipeline as WonderPlay for final video refinement. The coarse render and the simulated flow reach the video generator through two separate control paths:
- Motion path (flow-warped noise). The pixel-space flow F, rendered from the particle velocities, warps a Gaussian noise sequence into structured noise N(F). This noise is fused with random noise, and Go-with-the-Flow uses it as the starting noise, so the simulated motion is imposed on the generated frames.
- Appearance path (SDEdit). The coarse video Ṽ, rendered from the Gaussian-updated particles, is noised to timestep s₁ and then denoised jointly over all frames from that point, following WonderPlay Eq. (4): V_{s₁} = α_{s₁} Ṽ + √(1 − α²_{s₁}) N(F). The render is the starting point of denoising, not a conditioning signal applied throughout, and a larger s₁ gives the generator more freedom to change it.
We totally agree with your observations, but since the render is just the starting point of denoise, it is ok that the rendered video is not what we expect for the final video. This is the expected behaviour of the appearance path, and it is controlled by s₁. By your comment we think that you are referring to the ocean wave example. If you check out our Figure 3 - the predicted final state of this case, we want to hightlight that the predicted final state (Figure 3e) is the main source of the mismatch: the generator is pulled toward an appearance that the final-state prediction did not capture well. We will attribute it to the “very off” prediction of the final stage by AI, instead of our pipeline failure. If the users provide better “expected appearance of the ice”, the results will be better.
For your Q2, we really appreciate your questioning about reliablity of a one-shot vlm inference. Indeed, we consider that to let the model create fully controlled and satisfied physics-grounded video in one single shot, there is quite a long path to go. We apologize if any sentence in our paper makes you think that is our claim. As we mention in the main reply, our user interviews indicate that “realistic” or acceptable physics-grounded video is highly subjective and varies from person to person. For this reason, we believe it is still currently difficult to quantify an acceptance rate in a strict and scientific way, based on the state-of-arts.
However, all the results in Figure 3 is actually the best of three runs per model, with slightly different prompts. We already reported the best results we got from each model. For our pipeline, if we consider our shown video is “acceptable”, roughly our success rate of the first attempt on average is about 66% (one run will be off and not good per example). However, the results have shown that our model indeed can provide better control on the final video creation, compared with similar methods / generative AI models without our pipeline, considring all the shown results of the baseline is not “acceptable” with the success rate of 0% in three runs.
We hope these can answer your questions well, and please feel free to ask us more questions if there are still any confusions!
Thank you for your comments! Please first check out Our "Reply to the major comments from all reviewers". We address each point from you below:
For weakness #1, We agree that PhasePlay does not model heat transfer or spatially varying thermal fields. The phase-change mechanisms are phenomenological approximations, and we have revised the abstract and introduction to state this. As mentioned in the "reply to all reviewers", This simplified design is deliberate, because it lets users without substantial physics knowledge operate the pipeline. The reasoning module handles the parameter choices, and the video refinement stage handles appearance. We agree that some scenarios need finer control over the physics. The Limitations section notes this, and the field-valued extension (replacing the scalar λ with a spatially varying field) is a natural direction for users with sufficient physics knowledge. We consider this extension as a less common scenario, which is not considered in this work. However, we have modified the conclusion to admit this limitation.
For weakness #2, We agree that intermediate outputs help readers assess each component. The added Appendix G now shows, for each example, the input and inpainted background, the depth map, the SAM mask candidates, the object multiview, the simulated flow magnitude, the Gaussian-updated render, and the final video. These results show that the predictions at some stages are sometimes incorrect. However, the correct candidate results is always shown in the output pools for user to select. Hence the pipeline can provide reliable results as long as each stage ouput is confirmed by users.
For weakness 3, We have added Seedance-2.5 in Appendix C, with qualitative results in Figure 4. Seedance-2.5 renders the wood ignition and ocean wave examples smoothly, but shows physical and thermodynamic inconsistencies in the other cases, including butter melting and the ice-related scenes. This still show that exisitng state-of-art is not sufficinet / reliable for generating physics-plausible video.
For the Q1, please check out the main reply further, The reasoning module generates the simulation hyperparameters for each scene with AI reasoning automatically.
For Q2, We apologise. the link is activated now.
For Q3, We agree that the task studied here is physics-aware video generation from a single image with user-specified actions. We mention world models in the related work because they also predict future states from action inputs, which partially falls in this category. We will consider reducing the length or removing in the final version.
For Q4, we apologize the confusion, we tried to make the framework more general in theory and we admit that is our writing incorrectness that cuase the confusion. In practice, the calculation is simplified: the AI reasoning module takes the stated thermal condition T(t) and generates the schedule parameters for each scene / physics, completely by AI reasoning. Then the particle states are updated based on the schedule parameters and other input such as forces.
For your Q5, Optimal transport establishes the correspondence between simulated particles and the target Gaussians. Appendix G shows cases where the shape changes less, and the mismatch is acceptable there. However, we want to highlight that the final state is just a reference for how the new generated particles should look like in RGB color, in the end the final video is still largely controlled by the simulation results, as shown in the Appendix G. So the model will not get confused under this case.