acceptodds
Withdrawn from ICLR 2027

PhysChain: Constraint-based Physical Chains for Visual State and Transition Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

While existing methods for video generation leveraging the physical reasoning of Vision-Language Models (VLMs) have made impressive progress, they focus on injecting the knowledge into video generation through designated representations (e.g., physical descriptions, property labels, or motion trajectories), bounding the dynamics they can express. In this work, we present PhysChain, a framework that simulates a chain of future states with VLMs to generate physically valid videos from a single image. Our key idea is to simulate the chain of states and transitions under individually verifiable physical constraints to tractably handle real-world complex dynamics, highly entangled and implicit. For consistent and reliable simulations, we design a predict-and-correct loop with sampling-based constraint derivation, and further introduce a novel confidence-adaptive rendering scheme that tolerates the prediction errors that still remain after the simulations. In the experiments, PhysChain consistently outperforms competing physics-aware methods across five physics benchmarks, ranking first on most of them even among large-scale video foundation models, and generalizes to complex real-world dynamics with various potential applications. Videos are available at : https://anonymous-submission-384590248.github.io/anonymous-page.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Generation, Physically plausible video generation, Vision Language Model

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_6HaWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a framework designed to generate long-horizon, physically plausible videos from a single input image. Its core highlights include building a structured observable-cue memory using Vision-Language Models (VLMs) to translate complex physical dynamics into atomic physical constraints, employing a global iterative predict-and-correct loop to prevent error accumulation typical in traditional causal simulations, and introducing a confidence-adaptive rendering mechanism that dynamically schedules the video model's denoising process to preserve verified physical states while refining details through generative priors.

Critical strengths
  1. The framework's paradigm emulates human cognitive reasoning, allowing it to bypass the constraints of explicit trajectory representations and natively handle complex, implicit real-world dynamics like fluid and thermal behaviors.

  2. It addresses the gap between VLM reasoning and pixel synthesis, leading to decoupling of physical planning from image editing, successfully turning the VLM into an effective constraint-formulating planner and judge rather than forcing a monolithic model to directly bridge logic and generation.

Critical weaknesses

1. Heavy Cost: The framework's heavy reliance on multi-stage VLM reasoning loops and iterative predict-and-correct cycles introduces prohibitive time costs and computational complexity, transforming what should be a practical video generator into a sluggish offline optimization pipeline. Generating a short video requires minutes of heavy API calls and single-GPU rendering per clip, making the approach fundamentally unviable for real-time applications, interactive simulators, or large-scale dataset production.

2. Physical Properties Relying on VLM: The system's physical validity is bottlenecked by the initial VLM's capacity to flawlessly parse complex physical properties, leaving the entire architecture highly vulnerable to garbage-in, garbage-out failures. Because the framework depends on VLM self-evaluation to compute confidence scores, any instance of "confident hallucination", where the VLM misjudges a physically invalid state as correct, causes the adaptive rendering mechanism to falsely lock in and protect the flawed physics, neutralizing its own error-correction safeguards.

Questions for authors
  1. Memory Generalizability: How robust is the Chain-of-Thought observable-cue memory construction when scaling from controlled benchmarks to unstructured, open-world environments with highly ambiguous physics?

  2. Warping Robustness: Since warp-based temporal resampling relies heavily on dense correspondences, how does the renderer handle severe occlusions, object disappearances, or large topological deformations in complex physical chains?

  3. VLM Reliability & Confident Hallucination: How does the framework prevent cascading failures if the VLM misjudges a physically invalid state with high confidence, which would otherwise cause the adaptive rendering to erroneously lock in the error?

Reviewer_NeKROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PhysChain, a framework for generating physically plausible videos from a single image. The method first uses a VLM to organize the predicted physical process into an observable-cue memory containing events and individually verifiable visual facts. It then generates a sequence of key visual states with an image editing model and iteratively verifies and corrects their content and timing according to the sampled constraints. Finally, a confidence-adaptive rendering scheme converts the key states into a continuous video while preserving high-confidence states and allowing the video prior to refine less reliable ones. Experiments on five physics-related benchmarks show consistent improvements over the video backbone and several physics-aware baselines.

Critical strengths
  1. The paper addresses an important problem: improving physical plausibility without relying on a fixed representation such as bounding boxes or drag trajectories.

  2. The observable-cue memory and predict-and-correct loop provide a clear and intuitive framework.

  3. The method covers a broad range of physical domains and performs well across five benchmarks.

Critical weaknesses
  1. The method relies heavily on the VLM to predict the physical process and verify whether the constraints are satisfied, as well as on the image editing model to generate the key states. It is unclear how failures at each stage affect the final video or which component has the greatest impact on performance. The paper lacks a systematic analysis of these failure cases.

  2. The paper does not report dedicated image or video quality metrics. Therefore, it is unclear whether latent overwriting and confidence-adaptive denoising introduce artifacts or degrade visual quality, temporal smoothness, and identity consistency.

Questions for authors
  1. How do failures in the VLM reasoning, constraint verification, and image editing stages affect the final video? Could you provide a stage-wise failure analysis and clarify which component has the largest impact on performance?

  2. How are the initial key time points selected from the observable-cue memory? Could you provide the exact sampling algorithm and study the effect of the number and density of sampled states?

  3. How robust is the warping-based temporal resampling under occlusion, object disappearance, and large deformation? Do you have quantitative results or failure examples for these cases?

  4. Does latent overwriting affect visual quality, temporal smoothness, or identity consistency? Could you report dedicated image/video quality metrics or a human evaluation?

  5. Could you report the four underlying Physics-IQ Verified components—Spatial IoU, Spatiotemporal IoU, Weighted Spatial IoU, and MSE—as well as results across multiple runs with standard deviations?

  6. Since some benchmarks rely on VLM-based judges, have you evaluated the reliability of these judgments or conducted a human study to validate the physical plausibility of the final videos?

Reviewer_xynhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PhysChain is a training-free framework for physically plausible image-to-video generation. It combines structured observable-cue memory, iterative correction of visual states and event timing, and confidence-adaptive video rendering. Experiments on five benchmarks assess visual plausibility, agreement with real trajectories, and compliance with physical laws. Reported results outperform competing physics-aware methods overall, while ablations support the memory and rendering components.

Critical strengths
  1. The design of this framework is reasonable, but lacks novelty.
  2. Experiments span five physical video benchmarks, combining VLM-based judgments with objective measurements against real recordings, motion trajectories, and physical laws.
  3. The paper is clearly written and well organized. The motivation and overall pipeline are easy to follow. The video demo looks good.
Critical weaknesses
  1. I don't think the training-free paradigm is a good way to solve physics-related problems. Without physical data involved in training, it's hard to truly solve problems about physical objects. In effect, this method just improves the success probability of video generation models at inference time through a few simple tricks, which I consider insufficient in terms of both theoretical grounding and workload.
  2. Using the same VLM for VLIPP is helpful, but other differences remain. The video backbone, additional image-generation model, and inference budget are not clearly matched. I would like to see a more controlled comparison to understand how much of the improvement comes from the proposed constraint mechanism.
  3. It is unclear whether the constraint satisfaction rate is a reliable confidence score. A state could pass many easy checks while still violating a key physical requirement. Does this score correlate with the physical errors measured by PISA or Morpheus? The current ablations on PhyGenBench do not answer this question.
  4. This work needs a user study to further prove its effectiveness.
Questions for authors

see above

Reviewer_ArDaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PhysChain generates future visual states through a VLM-driven predict-and-correct loop. A persistent memory of observable cues supplies constraints for checking states and transitions, and the resulting confidence scores control how strongly a video model preserves each state during rendering. The method is evaluated on five physics benchmarks, with both closed and open reasoning and image-generation models.

Critical strengths
  1. Compared with trajectory-based planning such as VLIPP, checking observable states and transitions offers a flexible way to represent physical events. Keeping a shared reference and correcting the whole sequence are sensible choices for maintaining consistency across interacting events.
  2. The evaluation covers five complementary benchmarks, including measurements of trajectories and Newtonian dynamics. The open-model variant and the use of the same VLM in planning baselines strengthen the evidence that the framework contributes beyond a stronger proprietary reasoner.
Critical weaknesses
  1. The fixed constraint memory provides consistency, but also makes the correction process depend heavily on the initial physical reasoning. When an important interaction is omitted or incorrectly predicted, satisfying those constraints may preserve the same mistake. The paper does not establish how often this occurs or how well the framework handles it. This is a substantive limitation of using the initial VLM expectations as the correction target.
  2. The aggregate benchmark gains are credible, but the distinctive claim about handling complex physical chains has more limited evidence. Applications such as Rube Goldberg machines are shown qualitatively; the experiments do not quantify whether the advantage over existing planners grows or persists as the number of dependent interactions increases.
  3. The evaluation does not directly characterize the tradeoff between improved physics and visual quality. Since rendering involves warping predicted states and partially denoising them, a dedicated assessment of appearance preservation and temporal artifacts would be useful. The current examples and benchmark scores provide limited evidence about this tradeoff, particularly around occlusions and large appearance changes.
Questions for authors
  1. In unsuccessful cases, how often are the initial physical constraints themselves wrong or incomplete, and can the correction loop recover from these errors?
  2. Can the authors quantify success on multi-event physical chains against an existing state-planning baseline, including whether errors accumulate as more interactions are added?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.