acceptodds
Withdrawn from ICLR 2027

Phase-Aware Video Generation for Physics-Grounded Dynamics and Interactions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating physically plausible videos for solid–gas dynamics is challenging as different phases exhibit distinct dynamics yet remain coupled through physical interactions. We present PAVG, a Phase-Aware Video Generator for solid–gas dynamics and interactions. It employs a dual-branch architecture to explicitly model the distinct dynamics of solids and gases, while spatiotemporal cross-attention captures their physical interactions. This design enables PAVG to preserve phase-specific motion characteristics while producing physically consistent responses across phases. To facilitate this task, we further construct a simulation corpus comprising over 700K physical trajectories across diverse solid, gas, and solid–gas interaction scenarios. Extensive evaluations demonstrate that our PAVG produces videos with improved motion adherence, physical plausibility, and visual quality compared with existing approaches.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Controllable video generation, Physics-grounded video generation, Phase-aware motion modeling, Solid-gas interaction, 3D trajectory prediction, Diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_znPtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces PAVG, a framework designed to address the lack of physical plausibility in existing video generation models when handling multi-phase dynamics, such as solid-gas interactions. PAVG features a dual-branch trajectory predictor that explicitly models the distinct dynamics of solids and gases separately, while employing a Cross-Phase Interaction (CPI) module to capture their physical coupling. To support this, the authors construct a large-scale trajectory simulation corpus comprising over 70K physical trajectories. Extensive evaluations demonstrate that PAVG outperforms existing approaches in motion adherence, physical plausibility, and video quality.

Critical strengths

PAVG effectively addresses the distinct physical dynamics of solids and gases through an explicit dual-branch design, coupling them via spatial and temporal Cross-Phase Interaction (CPI) modules. This architecture successfully preserves phase-specific motion characteristics while accurately capturing cross-phase physical coupling without requiring expensive online solvers.

Critical weaknesses
  1. Reliance on Pre-defined Physical Conditions & Limited Interaction Type:

While PAVG introduces an interesting dual-branch trajectory predictor to handle solid-gas interactions, its applicability is heavily constrained by the reliance on explicit, pre-defined physical conditions (e.g., applied forces, material properties, initial velocities) and manual phase masks. Furthermore, this paper restricts the interaction model to a strict one-way coupling: only the gas-solid interaction and future neglect the effect of gas onto solid. It oversimplifies real-world physics and limits its capability to handle bi-directional dynamic exchanges (e.g. solid-solid interactions, gas-gas interactions, or interactions between other phases).

  1. Physical Rigor and Evaluation:

The framework's reliance on data-driven trajectory priors and localized point-cloud approximations rather than rigorous online physical solvers leaves room for underlying mechanical inaccuracies. Coupled with the fact that evaluation relies primarily on human preference studies and multimodal LLM judges rather than absolute physical ground truth, the model measures perceived visual plausibility rather than guaranteed physical correctness.

Questions for authors
  1. Given that the synthetic testbed provides complete ground-truth simulation data, have the authors considered evaluating absolute physical errors—such as L2L_2 velocity field discrepancy or momentum conservation residuals? Would such rigorous quantitative physical metrics help rule out visually plausible yet physically non-compliant hallucinations?

  2. Can the proposed dual-branch architecture and Cross-Phase Interaction (CPI) design be smoothly generalized to scenarios involving other combinations of phases such as gas-gas, solid-solid. What if we want to introduce other materials like liquids?

  3. Could the authors provide more qualitative results, such as failure cases, longer video rollouts, or comparisons under highly cluttered/dense real-world scenes, to better demonstrate the robustness and boundary limits of the proposed trajectory guidance?

  4. Is there a pathway to relax or simplify these strict input requirements? Specifically, can the framework be extended to automatically estimate or infer necessary physical parameters and phase regions directly from the input image or text, thereby shifting from a controlled simulation paradigm to a more generalizable feed-forward text/image-to-video pipeline?

Reviewer_mBMXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

PAVG proposes a two-branch trajectory diffusion model for solid and gas dynamics, with each branch building on PhysCtrl's design and a gated cross-attention connecting the branches to model solid-to-gas influences. Following PhysCtrl, the predicted trajectories control a frozen off-the-shelf video generator for video synthesis. A large-scale trajectory dataset and a 32-scene video benchmark are introduced to support the new task of paired solid-gas dynamics generation.

Critical strengths
  • The task of physics-grounded video generation involving solid–gas interactions is relatively underexplored, so the large-scale trajectory dataset and benchmark would be meaningful contributions that could benefit further research.
  • The reported quantitative results seem to indicate that PAVG outperforms baselines.
Critical weaknesses
  • Despite video generation being a main application, there are almost no qualitative video comparisons beyond the one sample in the teaser. Most claims rely on VLM-based scores and human preference studies, yet there are no sufficient visual examples and comparisons to baselines to verify the results or understand failure cases. While providing code is appreciated, it should not substitute for presenting qualitative comparisons.

  • The architectural contribution and ablations are not sufficiently convincing. The method largely extends PhysCtrl with separate solid/gas branches and cross-attention. From my understanding Table 3 only compares unified and separate predictors in Stage 1, rather than comparing the full frameworks under equal training budgets. Table 4 disables cross-attention at inference, which demonstrates reliance on this pathway but not its benefits over a model trained without it, or over bidirectional (solid-to-gas and gas-to-solid) cross-attention.

  • I have concerns about the fairness of the video baseline comparisons. As stated in Appendix C.5, PhysCtrl is trained only on solid data but evaluated on gas and solid–gas cases, which account for 24 of the 32 benchmark scenes. This doesn't seem like a controlled comparison.

Questions for authors
  • Could the authors provide representative video comparisons with baselines across solid, gas, and solid–gas interaction scenes, including failure cases?

  • Why does the unified predictor underperform the phase-specific specialists in Table 3? What is lost during joint training if they have access to equal training budget? Since Table 3 only evaluates Stage 1, how would a unified predictor compare against the full two-stage framework under matched training budgets?

  • While the specialists outperform the unified predictor in Table 3, the final PAVG model outperforms the specialist baselines on single-phase tasks in Table 2. Could the authors discuss where these additional gains come from?

  • Could the authors report video-generation metrics separately for solid-only, gas-only, and solid–gas interaction scenes to clarify where the improvements are most significant?

Reviewer_FqTTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

PAVG predicts phase-specific 3D point trajectories and uses them to condition a frozen video generator. Building on PhysCtrl, it introduces separate solid and gas branches, directional cross-phase attention, and additional physical supervision. Evaluation includes a synthetic trajectory benchmark and a 32-scene video benchmark with multimodal-model ratings and human preferences.

Critical strengths

Separating phase-specific dynamics while learning their interaction is a reasonable approach. The simulation-based trajectory evaluation provides useful quantitative evidence that the interaction pathway improves gas-trajectory prediction.

Critical weaknesses
  1. Insufficient evidence for the central video-generation claims. The provided materials contain limited static qualitative comparisons, but I could not locate playable final generated videos; the repository’s diagnostic tracking animations are explicitly not model results. The 32-scene evaluation relies on sampled-frame model ratings and a small human study. Without diverse video comparisons and failure cases, it is difficult to establish whether improved trajectories translate into convincing temporal dynamics or meaningful practical advantages.
  2. The scope and validation of gas modeling remain unclear. The prominent dust example does not by itself establish gas dynamics: dust consists of solid particles, and treating it as a passive tracer of airflow requires justification. Although the method includes gas-specific constraints, the evidence does not sufficiently distinguish gas-like transport from generic point-cloud deformation. Given its substantial inheritance from PhysCtrl and its one-way solid-to-gas coupling, the claimed advance needs stronger validation beyond the authors’ simulation setup.
Questions for authors
  1. Could you provide videos comparing your method with the baselines on the same solid–gas scenes, including failure cases? It would also be helpful to test relevant gas or interaction scenarios from VideoPhy or Physics-IQ.
  2. Your examples include dust, which you model as particles that follow the gas flow. When is this a reasonable approximation? Could you clarify which examples show dust being carried by airflow, rather than moving mainly under its own inertia and gravity, and when your model would no longer be appropriate?
  3. Would your method still outperform a single model that jointly predicts solid and gas trajectories, if both models had similar capacity and were trained on the same data? Turning off cross-attention only at inference does not show whether a jointly trained model could achieve similar results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.