- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces PhysiPlay, which is an action-conditioned generative simulator that directly predicts vertex trajectories of meshes in 3D space. The model is initialized from PhysiFormer, with additional action conditioning injected through global AdaLN and local FiLM to control actuated objects. PhysiPlay uses a chunk causal autoregressive generation strategy to support streaming generation and interactively take in commanded actions of the actuated objects during inference. To improve rollout stability, training incorporates ground-truth, noised ground-truth, and self-generated histories, along with geometric perturbations. Experiments compare PhysiPlay with video generators and 3D trajectory diffusion models and show improvements on selected metrics.
Critical strengths
- Compared to video models, the exploicit 3D mesh representation provides a persistent state of complete objects, and allows the model to maintain geometric consistency across viewpoints.
- PhysiPlay makes modifications on top of the base model to support flexible object conditioning and interactive streaming generation.
Critical weaknesses
-
The modifications on top of PhysiFormer have limited justification. It is unclear whether the causal adaptation provides a clear improvement over straightforward chunk-by-chunk generation using the original base model with action conditioning. It would be helpful to include a comparison to justify this adaptation.
-
PhysiPlay compares its predictions with video generation models and 3D trajectory prediction models. However, PhysiPlay receives privileged information such as complete object states and structured actions, while video models receive only an initial frame and textual descriptions. The synthetic Pong renderings may also be out of distribution for the video models. Is there a more closely matched evaluation setting that would make this comparison more meaningful?
-
The comparisons with 3D trajectory prediction models seem limited and exclude active paddle interactions. Furthermore, PhysiPlay underperforms PhysCtrl on two out of the three reported metrics.
-
The ablations do not seem to ablate the different forcing mechanisms used in the paper. Could the authors include comparisons showing the effectiveness of the forcing mechanisms?
-
Despite introducing techniques to improve long horizon rollout stability, the model still appears to have substantial trajectory divergence according to Table 3.
-
The data is simulated at 120 Hz, so a four frame chunk is ~0.033 seconds. A 243 frame rollout represents ~2.03 seconds of simulated time (243/120), but takes 34.5 seconds to generate, which is roughly 17 times slower than real time. This seems to limit the practical use of the model for interactive simulation.
Questions for authors
Besides the weaknesses, could the authors report the number of unique training trajectories or total simulated frames used to train the model?