acceptodds
Withdrawn from ICLR 2027

CoDance for Physically Consistent Image-to-Video Generation via Coupled Reinforcement Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image-to-video models can produce visually realistic videos while violating the physical constraints governing motion and interactions, limiting their utility for embodied prediction and planning. Some recent attempts employ physical reasoning to refine the user prompt before feeding it into the video generation model, making the intended dynamics and interactions more explicit. However, reasoning over the prompt alone does not ensure that the generator can faithfully realize these inferred dynamics. This motivates jointly optimizing reasoning and generation to address the mismatch between inferred physical dynamics and their realization in generated videos. We introduce CoDance with improved physical quality by jointly optimizing a vision-language reasoner and a flow-based video generator with reinforcement learning. The resulting training pipeline, dubbed CoDanceRL, combines three parts: Coupled Policy Optimization, Physics-Aware Feedback, and Hierarchical Reward Normalization. CoDance obtains an IQ-Score of 45.4 on Physics-IQ Verified and an average embodied video generation score of 69.4 on RBench, compared with 34.8 and 58.1 for its Cosmos3-Nano base model. It also achieves an average closed-loop action-planning success rate of 41.5% across two WorldArena tasks, demonstrating its utility for downstream planning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Models, Video Generation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_rsv2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoDance trains the reasoner for prompt enhancement and the video diffusion model for generation together, for the improvement of video physical plausibility. CoDanceRL uses shared nested rollouts, physical fidelity reward and geometric trajectory reward to update both components. The model performs well on Physics-IQ Verified, RBench and WorldArena.

Critical strengths
  1. Improvements on the held-out VLM judge and downstream planning can prove that training produces useful changes rather than only improving motion plausibility.
  2. Ablations on coupled training (compared with reasoner-only, generator-only, sequential) are compelling.
Critical weaknesses
  1. CoDance only reports 2 tasks of WorldArena, which is hard to prove its general advantage
  2. Metrics of Physics-IQ are used as reward which is very unfair for comparison.
  3. RBench covers a very small range of models to compare.
Questions for authors
  1. Does the coupled/sequential advantage persist across training seeds since the distinction is very small?
  2. Which exact RBench judge settings were used, and were the imported baseline scores obtained with a compatible evaluator?
  3. What are the performances of CoDance on the whole WorldArena benchmark?
Reviewer_LMaYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoDance, which jointly optimizes a physical reasoning module and a video generator through reinforcement learning to reduce the mismatch between textual guidance and the dynamics realized in generated videos. For each input, the method samples multiple prompt rewrites and corresponding videos, evaluates them using motion-consistency and geometric trajectory rewards against reference videos, and updates the two modules using hierarchical reward normalization. Experiments cover Physics-IQ Verified, RBench, and two WorldArena closed-loop tasks, with comparisons against single-component training, sequential training, and different combinations of trained components. The results show improvements over both the base model and its supervised fine-tuning baseline.

Critical strengths
  1. Joint adaptation is well motivated. The reasoner continually receives feedback from the generator as it is updated, which can help reduce the mismatch between textual guidance and the resulting videos. The hierarchical reward scheme also reasonably distinguishes prompt quality from variation in video quality under the same prompt.
  2. The experimental design is well considered. Beyond video generation benchmarks, the paper includes evaluation by a VLM judge that is not used for reward optimization, as well as closed-loop control tasks. It also examines single-component training, sequential training, and combinations of components from different training stages.
Critical weaknesses
  1. The reward design does not fully align with the goal of physical consistency. The rewards primarily measure how closely a generated video matches a single reference in appearance and two-dimensional motion, although the same input may admit multiple physically plausible futures. A valid but different trajectory may be penalized, while a similar trajectory does not necessarily imply correct contact relationships or dynamics. The method therefore more directly optimizes the reproduction of reference motion; whether this translates into better physical correctness requires independent validation.
  2. The effects of hierarchical normalization and the geometric reward are not sufficiently separated. Normalizing rewards separately can mitigate differences in scale, but it may also amplify small variations caused by tracking noise and does not inherently resolve conflicts between rewards. Table 4 lacks a key control that uses only the physical-fidelity reward with the full hierarchical normalization scheme. It is therefore unclear whether the final gains come from the geometric reward, the normalization scheme, or their interaction.
Questions for authors
  1. How does the reward handle generated videos whose trajectories differ from the reference but are equally physically plausible? Do you have human evaluations or controlled experiments that distinguish reference matching from physical correctness?
  2. How are rewards computed when objects disappear, are incorrectly occluded, or cannot be tracked reliably? How do you handle cases with no valid points or point pairs?
  3. With only one optimizer update per batch, do policy-ratio clipping and dual clipping actually take effect? Could you report the fraction of samples for which clipping is activated?
  4. Why does performance on multi-entity and long-horizon tasks decline relative to supervised fine-tuning? Can the available error analysis distinguish sampling variability from a systematic bias introduced by the reward design in these complex interactions?

I would raise my score if the authors satisfactorily address these concerns.

Reviewer_4bESOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CoDance which is an image to video model that jointly trains a VL reasoner and a flow-based video generator to improve the physical consistency of its generations. CoDanceRL, works by sampleing multiple prompt rewrites and multiple videos per rewrite, then it updates both components above using shared feedback from the videos. The feedback combines reference-based motion/reconstruction metrics with tracked-point geometry, timing, and displacement scores. Hierarchical reward normalization separates the advantages for the reasoner and generator based on the nested rollout hierarchy.

The results are shown on Physics-IQ Verified, RBench, and two WorldArena planning tasks. CoDance improves Physics-IQ IQ-Score from 34.8 for the base model and 38.4 after SFT to 45.4, RBench average from 58.1 and 63.1 to 69.4, and mean planning success from 14.0% and 22.5% to 41.5%. Ablations examine reward components, normalization, single-component training, sequential training, and different inference-time component combinations.

Critical strengths
  1. since the physical-fidelity reward incorporates Physics-IQ metrics, it's good that CoDance added held-out VLM-judge scores and a downstream planning experiment. Improvements on these evaluations provide evidence that the method achieves benefits beyond increasing its directly optimized benchmark metrics.

  2. Ablation: CoDance compares the reasoner-only, generator-only, and sequential training, alongside other component combinations. These comparisons show the importance of contributions from both the reasoner and generator. The reward-normalization ablations also show that combining the unnormalized rewards performs worse than using physical fidelity alone.

Critical weaknesses
  1. Table 8 doesn't seem to show that coupled training leads to robust outperformance. At the same time, the lack of error bars makes its hard to assess the significance of these results. The most relevant sequential baseline achieves 44.1 IQ-Score versus 45.4 for CoDance. Also the baseline’s allocation of updates, generated samples, and compute is not specified in the paper. As a result, a 1.3-point difference does not show the benefits of coupling from differences in optimization budget in a convincing way. This is importang given that the coupling in the main methodological contribution.

  2. It's not clear how well the models learns general physics knowledge. Both rewards encourage agreement with a particular recorded groundtruth, which may penalize other alternatives that are also physically valid. The held-out judge also evaluates against reference videos and is not validated against human judgments so we don't know how reliable/realistic it is. Similarly there are some mixed results on transfers to other physics. (1) on RBench, manipulation, multi-entity, and long-horizon scores are mixed relative to SFT. (2) On planning, it covers only two tasks with ten distinct initial conditions each. Its not clear how this supports broad physics generalization.

Questions for authors

Can you describe more clearly the compute and optimizer updates per training phase per generated videos for CoDance and also the sequential baselines that CoDance is compared to (those that share the backbone). Also see above regarding uncertainty estimates.

How well does the method generalize to arbitrary physics knowledge especially those that are different from the data but equally valid?

The abstract motivates the coupled training primarily through the issues from prompt-only refinement, but the paper itself talks about other (non-prompt-only) methods that already adapt the generator or both components. Can the authors clarify the remaining limitation of these stronger approaches and why coupled optimization addresses these?

Reviewer_jb1UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper, CoDance, proposes a method to improve physical consistency in image-to-video generation using RL that jointly optimizes a VLM for physical reasoning and a video generation model. The video generator is based on NVIDIA's Cosmos3-Nano, and the VLM is based on Qwen3-VL. The main idea is to have the VLM first generate more detailed descriptions of how objects should move and interact, and then use these descriptions to guide video generation. Instead of training both models separately, the authors propose to optimize them jointly using feedback from the generated videos. The RL framework combines existing GRPO for the reasoning model and DanceGRPO for the video generator, with rewards based on motion consistency, geometric trajectories, and comparisons with reference videos. A hierarchical reward normalization is introduced to balance the different rewards and separately evaluate which video generations are better and which prompt descriptions lead to better results. The paper reports improvements on several benchmarks, including physical consistency and robotic planning. No playable video results are provided in the submitted material, but the figures show improved results on several selected examples compared with existing SOTA video models.

Critical strengths

The paper addresses an important problem in video generation, which is the lack of physical consistency, adn remains a major issue in most existing video generation models and is also one of the core challenges in robotics and embodied AI. The proposed method is relatively straightforward and can be easily implemented using existing RL techniques. The quantitative results do show improvements across several benchmarks, and some of the qualitative examples in the figures also look convincing and visibly better than those generated by certain existing SOTA video models. Overall, the approach is technically sound and demonstrates promising results.

Critical weaknesses

My main concern is that the proposed method seems rather incremental, as it mostly combines existing RL techniques and reward functions. The improvement in physical consistency seems to be coming mainly from better alignment between the reasoning model and the video generator, rather than from learning more real physical principles. The improvement over sequential training is also relatively small (45.4 vs. 44.1 on Physics-IQ), which makes the contribution of joint optimization less convincing. Another concern is why the authors compare against Seedance 1.5 on RBench instead of Seedance 2.0, which is already used for Physics-IQ and would be a more relevant baseline. Furthermore, physical consistency is mainly evaluated through comparisons with reference videos, and the results are limited to short clips. Some RBench results are also worse than the SFT baseline, and the robotic planning experiments cover only two tasks, making it difficult to assess how well the method generalizes.

Questions for authors

1/ Why did you use Seedance 1.5 instead of Seedance 2.0 for the RBench evaluation, while Seedance 2.0 was used for Physics-IQ? Wouldn't it be more appropriate to use the newer model for both benchmarks?

2/ Would it make sense to also evaluate your method using physics-based simulated data, where more accurate ground truth for object trajectories, collisions, and physical properties is available? This could help assess whether the model actually learns physically consistent behavior rather than simply matching reference videos.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.