acceptodds
Under review as a conference paper at ICLR 2027

PhysPilot: Top-Down Reinforcement Learning of Physical Priors for Video Generation

Abstract

Recent video generation models are capable of generating visually realistic videos, but often fail to adhere to physical laws, limiting their ability to generate physically plausible videos and serve as "world models". To address this issue, we propose PhysPilot, a generalizable physical law learning framework designed to enhance the physical plausibility of video generation. Specifically, PhysPilot is based on the image-to-video task where the model is expected to predict physically plausible dynamics from the input image. Since the input image provides physical priors like positions, materials, and interactions of objects in the scenario, we devise a physical encoder, PhysEncoder, to extract such physics-relevant visual cues as an extra condition. However, how to help the model learn to utilize these physical priors to assist generation remains an open question. Without a well-established definition for physical representation, we can not conduct straightforward supervision of PhysEncoder for training. To solve this challenge, we adopt a top-down optimization strategy, where the PhysEncoder and the video generation model are jointly optimized based on the physical plausibility of the final generated videos using reinforcement learning (RL). Through this top-down optimization, the PhysEncoder can effectively capture implicit physical cues from the starting image, and the video model learns to better utilize them, thereby implicitly learning physical laws. Experiment results prove that our method significantly enhances the model’s physics-awareness as a plug-in, and demonstrates strong generalization on both specialized proxy tasks and diverse open-world physical scenarios.

Primary area:
generative models
Keywords:
physics-aware video generation, representation learning, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NdoZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PhysPilot, which enhances the physical plausibility of video generation by learning physical priors from the input image. It introduces a PhysEncoder to extract physics-relevant representations and employs a top-down DPO-based optimization strategy to improve both physical cue extraction and their utilization by the video generation model. Extensive experiments across specialized physical tasks and diverse open-world scenarios demonstrate the effectiveness and generalization of the proposed method.

Critical strengths

1、The proposed top-down reinforcement learning strategy is effective, and the ablation studies provide convincing evidence for the contribution of the different training stages.

2、The experimental evaluation is comprehensive, covering proxy physical tasks, related-task transfer, and diverse real-world scenarios, demonstrating the effectiveness and generalization of PhysPilot.

Critical weaknesses

1、The necessity of the trainable physical head is not fully justified. The proposed PhysEncoder already adopts a frozen pretrained DINOv2 encoder, while only the physical head is optimized. However, the experiments do not compare against directly using the frozen pretrained visual representation as the physical condition. Such a baseline would help clarify whether the improvements mainly come from the pretrained representation itself or from the proposed physical representation adaptation.

2、The choice of DPO as the reinforcement learning objective is not sufficiently justified. Although the proposed method is built upon a flow-based video diffusion model, the paper does not discuss why Flow-DPO is preferred over other reinforcement learning objectives for flow-based generation, such as Flow-GRPO. A comparison or discussion of alternative RL objectives would help justify the design choice.

3、Table 1 contains a formatting issue that affects its readability. The authors should carefully revise the table layout and ensure that all entries are properly aligned and formatted.

Questions for authors

see Weaknesses.

5 comments

We thank the reviewer for the thoughtful suggestions on our baselines, training objective, and presentation.

R3-W1: Comparison of Frozen-representation baseline and the trainable physical head

We thank the reviewer for this suggestion. Following it, we add a baseline that directly uses the frozen pretrained representation as the physical condition. Its construction, the comparison with our pipeline, and the related representation-level evidence are given below.

1. Frozen-representation baseline

The frozen pretrained features (DINOv2-B tokens; the 64-channel dense output of the pretrained Depth Anything V2 head) do not match the 8-channel latent space of the video DiT, so "directly using the frozen pretrained visual representation as the physical condition" requires a dimensional mapping. We therefore instantiate the baseline with a non-learned mapping: Depth Anything V2's encoder and pretrained DPT head are kept frozen; their 64-channel dense features are projected to 8 channels by PCA, channel-wise affine-normalized to the latent statistics, and injected at exactly the same position as in our method (temporal concatenation). No part of this conditioning branch is trained: in all training stages, the DiT is the only learnable module. The baseline follows the same three-stage schedule and budget as our pipeline.

2. Comparison under the same schedule and budget

We compare three pipelines that differ only in the physical condition: no PhysEncoder, the frozen-representation baseline, and ours. All three follow the same three-stage schedule (SFT, followed by two DPO rounds) with the same budget. We report the results after SFT alone and after the full three stages.

After SFT alone.

Table a: Physical-condition comparison after SFT alone.

Training stagesSeen L2 (↓)CD (↓)IoU (↑)Unseen L2 (↓)CD (↓)IoU (↑)Avg L2 (↓)CD (↓)IoU (↑)
No PhysEncoderSFT for vθv_\theta (Table 1, row 2)0.0532\mathbf{0.0532}0.134\mathbf{0.134}0.137\mathbf{0.137}0.05120.05120.1330.1330.135\mathbf{0.135}0.0522\mathbf{0.0522}0.134\mathbf{0.134}0.136\mathbf{0.136}
Frozen-representation (new)SFT for vθv_\theta, frozen representation as condition0.05610.05610.1400.1400.1360.1360.05030.05030.1300.1300.1340.1340.05320.05320.1350.1350.1350.135
Ours (Stage I; Table 1, row 3)SFT for vθv_\theta & EpE_p0.05680.05680.1410.1410.137\mathbf{0.137}0.0498\mathbf{0.0498}0.128\mathbf{0.128}0.1340.1340.05330.05330.134\mathbf{0.134}0.1350.135

After SFT alone, neither the frozen representation nor the jointly trained PhysEncoder brings a stable gain over the model without PhysEncoder. Both improve the trajectory metrics (L2 and CD) on the Unseen split of the evaluation set, but both are worse on the trajectory metrics of the Seen split, and neither improves the average results. Adding a physical condition through SFT alone, whether it is frozen or trainable, is therefore not sufficient.

After the full three stages.

Table b: Physical-condition comparison after the full three-stage schedule.

Training stagesSeen L2 (↓)CD (↓)IoU (↑)Unseen L2 (↓)CD (↓)IoU (↑)Avg L2 (↓)CD (↓)IoU (↑)
No PhysEncoderSFT for vθv_\theta + DPO for vθv_\theta + 2nd DPO round on vθv_\theta0.05520.05520.1400.1400.146\mathbf{0.146}0.04530.04530.1220.1220.1300.1300.05030.05030.1310.1310.1380.138
Frozen-representation (new)SFT for vθv_\theta + DPO for vθv_\theta + 2nd DPO round on vθv_\theta0.05020.05020.1300.1300.1370.1370.04580.04580.1200.1200.1360.1360.04800.04800.1250.1250.1370.137
Ours (Stage III; Table 1, row 8)SFT for vθv_\theta & EpE_p + DPO for vθv_\theta + 2nd DPO round on EpE_p, vθv_\theta frozen0.0489\mathbf{0.0489}0.120\mathbf{0.120}0.1410.1410.0450\mathbf{0.0450}0.115\mathbf{0.115}0.145\mathbf{0.145}0.0470\mathbf{0.0470}0.118\mathbf{0.118}0.143\mathbf{0.143}

The comparison between the first two rows measures the contribution of the pretrained representation itself, and the comparison between the last two rows measures the contribution of the physical representation adaptation.

Frozen-representation vs. no PhysEncoder. Directly using the frozen pretrained representation improves the trajectory metrics (L2 and CD) on average, and most of this gain is obtained on the Seen split of the evaluation set. It does not improve IoU on average, and on the Unseen split its effect is small and not consistent across the three metrics. The pretrained representation itself therefore accounts for a part of the improvement, mainly in trajectory accuracy on objects and backgrounds seen during training.

Ours vs. frozen-representation. With the physical head adapted by SFT and DPO, our model outperforms the frozen-representation baseline on every metric of both splits. On the Unseen split, the improvement brought by the adaptation is larger than the improvement brought by the frozen representation on all three metrics, and the improvement in average IoU comes entirely from the adaptation.

3. Representation-level evidence in the submission

The comparisons above evaluate the representations through the generated videos. The probes in Sec. 4.2 examine the representations themselves, without any training, and address the same question: whether the physical information is already present in the frozen pretrained feature ("DA dense"), or is acquired through adaptation. Each probe is evaluated on the frozen pretrained feature and on the adapted features, so that they can be compared directly.

Probe 1: physical grouping across semantic categories (Table 4 in the submission).This probe tests whether objects with the same physical behavior have similar features even when they belong to different semantic categories. We assign 185 objects from nine semantic categories to three physical groups by deformation behavior, and measure how well the feature similarity follows these physical groups. Pairs from the same semantic category are excluded, so semantic similarity cannot account for the result.

Table c: Physical grouping across semantic categories.

FeatureContrast (↑)NN@5 (↑)RSA (↑)
DA dense (frozen pretrained)0.280.280.3430.3430.0090.009
Stage I dense0.340.340.3470.3470.0400.040
Stage I latent0.640.640.3510.3510.0610.061
Stage III dense0.480.480.399\mathbf{0.399}0.0560.056
Stage III latent0.69\mathbf{0.69}0.3610.3610.111\mathbf{0.111}

The frozen pretrained feature follows the physical groups the least on all three metrics. The agreement increases after Stage I and increases further after Stage III. The physical grouping is therefore not inherited from the pretrained representation; it is formed by the adaptation.

Probe 2: sensitivity to physical versus appearance factors (Table 3 of the submission, with the frozen pretrained feature added). This probe changes one factor of a scene at a time and measures how much the feature changes. The factors are either physical (the support state of the object, i.e., whether it is in contact with the ground or in the air; its height) or related to appearance (background, lighting).

Table d: Sensitivity to physical and appearance factors.

Changed factorDA dense (frozen pretrained)Stage III latent
Support state (air / ground)0.02960.02960.12170.1217
Height0.01270.01270.03520.0352
Rotation0.01520.01520.01380.0138
Background0.01180.01180.01150.0115
Lighting0.00170.00170.00040.0004

After Stage III, the feature changes about ten times more under a change of a physical factor (the support state) than under a replacement of the whole background, and it is almost unaffected by lighting. For the frozen pretrained feature, the responses to physical factors and to appearance factors are much closer. The adaptation therefore makes the feature sensitive to the physical factors of the scene rather than to its appearance.

Probe 3: separation of physics laws in open-world scenes. This probe repeats the test beyond the proxy task. Scenes from the WISA test set are labeled by their governing physics law (dynamics, thermodynamics, optics), and we measure how much more similar the features of scenes under the same law are than the features of scenes under different laws. This separation is the weakest for the frozen pretrained feature and increases after each stage of adaptation. After Stage III, it is more than three times that of the frozen pretrained feature. The conclusion of Probe 1 therefore also holds for general physical scenes.

Summary. These results answer the reviewer's question in three steps. (1) After SFT alone, using the frozen pretrained representation as the physical condition brings no stable gain. (2) After the full three-stage training (one SFT stage and two DPO stages), the frozen representation improves over the pipeline without any representation, but this improvement is smaller than the one obtained with the physical representation adaptation. (3) In the training-free probes, the frozen pretrained feature is clearly weaker than the adapted feature. The pretrained representation itself therefore explains only a part of the improvement, and the remaining part is obtained through our three-stage physical adaptation, which is why the physical head needs to be trainable.

R3-W2: Flow-DPO and comparison with Flow-GRPO

Below we explain why Flow-DPO is chosen, which follows from the form of supervision available in our setting, and report a direct comparison with Flow-GRPO on the proxy task, where an online reward is available.

Why Flow-DPO. Online RL objectives such as Flow-GRPO need a reward for every video sampled during training. Our pipeline targets the open-world setting, where such a reward is not available. We tried open-source VLMs as automatic judges, but they are weak at comparing the physical plausibility of a pair of videos (see also the Limitations), and optimizing the generator online against an unreliable judge would not reliably improve physical plausibility. Human annotation, in turn, cannot be provided for every video sampled during training. Physical plausibility is therefore judged by human annotators as pairwise preferences, which are collected before training. Flow-DPO can be trained directly on such a fixed set of preference pairs, without a reward model and without any reward query during training. We use the same form of supervision on the proxy task, where candidate videos are ranked against simulated ground truth by mask IoU, so one objective covers both settings. The proxy task also admits an automatic reward, which allows the direct comparison with Flow-GRPO below.

Comparison on the proxy task The proxy task does admit online RL: the same automatic protocol we use to label preference pairs (VAE decode → SAM 2 segmentation → IoU against simulated ground truth) can serve as a per-rollout reward. We therefore implemented Flow-GRPO (Liu et al., 2025) on our generator, following its ODE-to-SDE conversion for stochastic sampling and denoising-reduction strategy. The policy parameters are the same LoRA (r=64) trained by our Stage II, and both methods are trained for the same number of steps on top of the same Stage-I checkpoint:

Table e: Flow-DPO and Flow-GRPO on the proxy task in Stage II.

Method (after Stage I)Seen L2 (↓)CD (↓)IoU (↑)Unseen L2 (↓)CD (↓)IoU (↑)Avg L2 (↓)CD (↓)IoU (↑)
+ DPO for vθv_\theta (Stage II, ours)0.05200.05200.1250.1250.1330.1330.04540.04540.1160.1160.1360.1360.04870.04870.1200.1200.1340.134
+ Flow-GRPO for vθv_\theta0.05610.05610.1380.1380.1360.1360.04810.04810.1250.1250.1360.1360.05210.05210.1320.1320.1360.136

Under the same number of training steps, the two objectives improve different aspects. Flow-DPO gives the better trajectory metrics (L2 and CD) on both the Seen and Unseen splits of the evaluation set. Its improvement over the Stage-I checkpoint (Table 1, row 3) on these metrics is clearly larger than that of Flow-GRPO, which improves them only slightly. Flow-GRPO is slightly better on IoU on the Seen split and on average, and the two are equal on the Unseen split.

A plausible explanation for the better trajectory metrics (L2 and CD) of Flow-DPO in this comparison is the form of the optimization feedback. Both methods use the same IoU criterion. Flow-DPO converts it into fixed preference pairs and compares the preferred and the non-preferred video under the same noise and timestep. The loss therefore depends only on the difference between the two videos, and the gradient directly reflects what makes the preferred video better. Flow-GRPO instead estimates group-relative advantages from the IoU of the generated videos, and applies this video-level feedback to all stochastic denoising transitions. The reward indicates the overall quality of a video, but not which transitions caused a physical error, so the credit assignment is indirect. This makes the updates less effective at correcting the object motion, while Flow-GRPO remains competitive on IoU, the quantity it maximizes directly.

In conclusion, Flow-DPO is the more suitable objective for our pipeline. On the proxy task, it improves trajectory accuracy more than Flow-GRPO under the same number of training steps, with comparable IoU. It does not need to sample and score videos during training, and it also applies to the open-world setting, where no reliable online reward is available.

Reviewer_8nDdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PhysPilot improves the physical plausibility of image-to-video generation by introducing a PhysEncoder that extracts additional conditioning features from the initial image. Training proceeds through joint supervised fine-tuning, DPO on the generator’s LoRA parameters, and DPO on the physical head with the generator frozen. Experiments cover falling and collision scenarios, adaptation to related tasks, and broader physical phenomena.

Critical strengths
  1. The paper uses video-level preferences to train an image-conditioning encoder, without requiring explicit labels for physical properties. This is a useful idea, and the staged training pipeline offers a practical way to implement it.
  2. The evaluation covers controlled tasks, downstream adaptation, and broader scenarios. Ablations on the training stages and trainable components help show how different parts of the method contribute to its performance.
Critical weaknesses

The ablations do not clearly explain why the final design was chosen. Table A1 shows trade-offs between L2- and IoU-based preferences, and Table A4 does not show a consistent advantage for either the depth-based or DINOv2-based encoder. These results suggest that the best choice depends on the task, so claims that one option is better should be more cautious.

Questions for authors
  1. Could you specify the video generator’s architecture and parameter count?
  2. Appendices B and C.1 mention using a “test set” to construct DPO preference pairs. Is this a separate dataset from the final evaluation set? In particular, do the two sets share any scenes or source videos?
3 comments

We thank the reviewer for the helpful comments on our design choices and the clarity of the manuscript.

R2-W1: Design choices

We agree with the reviewer. Tables A1 and A4 show trade-offs, and no option is better on every metric. Our default choices were made for generalization: we prefer the option that performs better on the Unseen split of the evaluation set, which contains novel objects and backgrounds. On this split, the IoU criterion gives the best trajectory accuracy (L2 0.04540.0454, CD 0.1160.116), and the Depth-based encoder is the best on all three metrics. These two results determine our current choice of options.

More importantly, the different options lead to similar conclusions. For the preference criterion, all three criteria improve the average L2 and CD over Stage I. The improvement from DPO therefore does not rely on a particular criterion. The Depth-based and DINOv2-based encoders are also close on average. We take the option with better generalization as the representative setting for the remaining experiments.

R2-Q1: Generator architecture and parameter count

Our video generator is a latent rectified-flow image-to-video DiT with 1B parameters. A 3D causal VAE encodes RGB videos into 8-channel latents with 8×8\times spatial and 4×4\times temporal compression; text is encoded by a T5 encoder. For I2V generation, the clean first-frame VAE latent and the 8-channel physics latent produced by PhysEncoder are prepended to the noisy future-video latents along the temporal dimension.

PhysEncoder contains ~97.6M parameters in total: a frozen DINOv2-B backbone, plus the trainable DPT-style physical head and the 64-to-8-channel latent adapter.

R2-Q2: Separation of preference and evaluation data

The set called "test set" in Appendices B and C.1 is a DPO preference pool held out from SFT training; the final evaluation uses PisaBench's simulated (Seen/Unseen) and real-world subsets. We audited the overlap programmatically at two levels: source videos, zero overlap between the pool and each evaluation subset; scenes/object assets, the evaluation Unseen split shares none with either the SFT set or the pool, while the Seen split shares asset and background with training and the pool by definition of that split following PisaBench protocol. The real-world subset is from a different domain entirely.

Thank you for your detailed response and the effort you put into addressing my comments. After reading the rebuttal, I will maintain my score.

Dear Reviewer,

Thank you so much for your positive assessment and encouraging attitude throughout the review process. We are glad that our responses have addressed your concerns, and we believe that the additional experiments and clarifications have further strengthened the paper. We sincerely appreciate your continued support, thoughtful feedback, and generous consideration. Thank you again for your time and encouragement!

Best regards,

The Authors

Reviewer_MMiXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

PhysPilot proposes a method for improving the physical plausibility of image-to-video generation. It introduces PhysEncoder, an auxiliary module with a backbone derived from Depth Anything, that extracts features from the initial image to guide the video generation model. The generator and encoder are trained in 3 stages: one stage of SFT, jointly training both the generator and encoder, and two stages of DPO, the first training the generator through LoRA updates with the encoder frozen, and the second training the encoder with the generator frozen. Preferences for DPO are derived from agreement with reference object masks for free-fall trajectories and from human judgments for more diverse scenarios. The approach is experimentally validated on free-fall evaluations and related tasks, as well as on broader physics-related scenarios, and demonstrates improvements in the physical plausibility of the generated videos.

Critical strengths
  1. PhysPilot demonstrates strong empirical performance in terms of the physical consistency of generated videos.

Taken as a whole, the overall training pipeline results in a video generation model which outperforms relevant existing models, such as PhysGen and PISA, and also demonstrates meaningful improvements in qualitative user studies compared to the base model. The paper also provides various informative ablations on physical encoder architectures (e.g. CLIP vs DINOv2 vs Depth-Anything), training strategy (SFT-only vs SFT+DPO, LoRA vs full finetuning), and criteria used to construct preference data (Chamfer vs L2 vs IoU).

Critical weaknesses
  1. The experiments do not conclusively demonstrate that the PhysEncoder provides a meaningful performance improvement.

The paper presents the PhysEncoder, a model component used to "extract physical priors from the input image", as one of its primary contributions. However, the paper lacks experimental evidence that demonstrate that the performance improvements can be specifically attributed to the PhysEncoder rather than being solely driven by DPO post-training. In particular, the strongest results in Table 1 and Table 8 are obtained with two stages of DPO training, while the ablated variants without the PhysEncoder receive only one, implying fewer overall steps of DPO post-training and/or fewer preference examples, making this an unfair comparison. The training step count ablation in Table A.2 notably excludes 1000 steps, i.e. the total length of DPO training for stage-3 models, so this ablation also provides little insight into the performance difference between stage-II and stage-III models. The few fair comparisons between encoder and no-encoder variants described in the paper show mixed results, e.g. the no-encoder variant having better IoU and L2 than the variant with the encoder after stage-2 training. Overall, the ablations fail to isolate the contribution of PhysEncoder, making it difficult to determine whether the paper's demonstarted experimental improvements can be attributed to its primary methodological novelty.

Questions for authors
  1. Appendix B states that the "test set includes 500 videos for preference data construction", presumably for DPO training, and appendix C states "We create random subsets of 500, 1500, and 2500 samples from the full test dataset and finetune our model for the same number of steps in Stage II on each subset". Does performing DPO on 'test' data imply that data leakage occurred between training and evaluation?

  2. Table 7 compares PhysPilot to several text-to-video models, e.g. Wan2.1-T2V-1.3B, PhyT2V. However, the paper's approach is based on I2V generation, and VideoPhy is generally a text-to-video benchmark without reference seed frames. How were image conditions for the proposed model created?

  3. Could more details about training configuration and hyperparameters be provided? In particular, what was the architecture of the physical head, the LoRA rank of the generator, the set of layers which had trainable LoRA layers, DPO regularization strength, and training hardware?

5 comments

We thank the reviewer for the detailed feedback and constructive suggestions on our experimental evaluation.

R1-W1: Isolating the contribution of PhysEncoder

We thank the reviewer for this careful reading. In Tables 1 and 8, the ablated variants without PhysEncoder received one stage of DPO while our final model received two, i.e., fewer total DPO steps and/or fewer preference pairs. Below we (1) report two budget-matched controls that satisfy both conditions simultaneously, (2) clarify what Table A2 actually varies, and (3) clarify the scope of our claim regarding the stage-2 comparisons.

1. Two budget-matched controls (same total DPO steps and same number of preference pairs)

  • Control A — no PhysEncoder mirror. This pipeline is identical to our full pipeline except that PhysEncoder is never used. It consists of SFT for vθv_\theta, followed by two successive DPO rounds on vθv_\theta (LoRA). Following the construction protocol of preference data pair in Sec. 3.3 ("candidate pairs are generated using the same prompt and first frame with different sampling seeds from the preceding-stage model"), the 500 pairs for the second DPO round are generated by the model after the first round. Such setting is exactly the no-PhysEncoder mirror of our full pipeline with the same total budget of 500 + 500 DPO steps and 500 + 500 data pairs.

  • Control B — no DPO-on-PhysEncoder mirror. This pipeline is identical to our full pipeline except that DPO is never applied on PhysEncoder but only on vθv_\theta. Up to Stage II it is identical to our full pipeline: it starts from the same Stage-II checkpoint used by our Stage III; in Stage III, the second DPO round instead trains the DiT LoRA on the same 500 preference pairs, with the same reference model and step count, while PhysEncoder stays frozen. Since these pairs were generated by the shared Stage-II checkpoint, they are on-policy for Control B as well. The only difference from Stage III is which parameters are updated.

Results on the proxy task(protocol of Table 1, PisaBench).

Table a: Budget-matched controls on the proxy task.

Training stagesSeen L2 (↓)Seen CD (↓)Seen IoU (↑)Unseen L2 (↓)Unseen CD (↓)Unseen IoU (↑)Avg L2 (↓)Avg CD (↓)Avg IoU (↑)
SFT for vθv_\theta + DPO for vθv_\theta (Table 1, row 4)0.05600.05600.1430.1430.144‾\underline{0.144}0.0446\mathbf{0.0446}0.115\mathbf{0.115}0.1280.1280.05030.05030.1290.1290.1360.136
Control A: row 4 + 2nd DPO round on vθv_\theta (self-generated pairs)0.05520.05520.1400.1400.146\mathbf{0.146}0.04530.04530.1220.1220.1300.1300.05030.05030.1310.1310.138‾\underline{0.138}
SFT for vθv_\theta & EpE_p + DPO for vθv_\theta (Stage II; Table 1, row 5)0.05200.05200.1250.1250.1330.1330.04540.04540.116‾\underline{0.116}0.1360.1360.04870.04870.120‾\underline{0.120}0.1340.134
Control B: Stage II + 2nd DPO round on vθv_\theta, EpE_p frozen (same pairs as Stage III)0.0512‾\underline{0.0512}0.124‾\underline{0.124}0.1370.1370.04560.04560.1180.1180.137‾\underline{0.137}0.0484‾\underline{0.0484}0.1210.1210.1370.137
Stage III (ours): Stage II + 2nd DPO round on EpE_p, vθv_\theta frozen (Table 1, row 8)0.0489\mathbf{0.0489}0.120\mathbf{0.120}0.1410.1410.0450‾\underline{0.0450}0.115\mathbf{0.115}0.145\mathbf{0.145}0.0470\mathbf{0.0470}0.118\mathbf{0.118}0.143\mathbf{0.143}

With the overall pipeline budget matched, the three pipelines(Control A, Control B, Ours) in the table behave differently.

(1) More DPO on the generator does not improve stably. Control A improves over row 4 on all Seen metrics, but on the Unseen split only IoU improves marginally while L2/CD regress. We attribute this to continued LoRA-DPO on the generator: more pairs and more steps fit the model more strongly to the domains seen during training, helping the Seen split of the evaluation set at the cost of generalization to the Unseen split: consistent with Table A2, where more preference pairs likewise improve the Seen split but degrade the Unseen split. Control B shows the same pattern in milder form relative to row 5.

(2) Our pipeline allocates the two rounds differently, and this is where the design pays off. Stage II first aligns the generator’s outputs with physical plausibility, which also “paves the way” by letting the next stage draw higher-quality candidate videos; Stage III then fine-tunes only PhysEncoder, with the aim of producing a better physical representation that acts on the generated videos implicitly, as a condition; the generator itself stays frozen and is never further fitted to the preference data, which avoids the overfitting-to-Seen and degraded-generalization problem above. Stage III is the only second-round option in the table that improves all six cells, on both the Seen and Unseen splits (Unseen IoU 0.136→0.1450.136 \to \mathbf{0.145}).

(3) Together, the two controls isolate what the reviewer asks about. Control A shows that removing PhysEncoder and spending the same budget on the generator for SFT and DPO does not reach our endpoint; Control B shows that keeping PhysEncoder but applying DPO only to the generator does not either. Under matched budgets, the remaining improvement is attributable specifically to DPO-adapting the physical representation.

Open-domain counterpart (protocol of Table 8, VideoPhy).

We run the same comparison on general open-world scenarios:

Table b: Budget-matched controls on open-world scenarios.

Training stagesSA (↑\uparrow)PC (↑\uparrow)
SFT for vθv_\theta + DPO for vθv_\theta (Table 8)0.640.640.350.35
Control A: SFT for vθv_\theta + DPO for vθv_\theta + 2nd DPO round on vθv_\theta (self-generated, newly annotated pairs)0.65‾\underline{0.65}0.350.35
SFT for vθv_\theta & EpE_p + DPO for vθv_\theta (Stage II; not reported in the submission)0.630.630.340.34
Control B: Stage II + 2nd DPO round on vθv_\theta, EpE_p frozen (same pairs as Stage III)0.620.620.36‾\underline{0.36}
Stage III (ours) (Table 8)0.67\mathbf{0.67}0.40\mathbf{0.40}

Spending the second DPO round on the generator does not achieve this: Control A barely moves over its first round (SA +0.01, PC unchanged), and Control B trades the two metrics: a small gain in physical commonsense at the cost of semantic adherence (SA 0.63→0.620.63 \to 0.62, PC 0.34→0.360.34 \to 0.36). This trade-off is expected: the human preference labels reward physical plausibility only, so pushing the generator itself onto these 500 pairs can sacrifice semantics for the preferred-physics signal. Stage III, spending the same budget on PhysEncoder, is again the only option that improves both metrics, and by the largest margins over the shared Stage-II checkpoint (SA +0.04, PC +0.06), reaching 0.67\mathbf{0.67} / 0.40\mathbf{0.40}, with the generator frozen, Stage III cannot degrade the semantic behavior the generator already has; the adapted condition only supplies better physical cues. That the same pattern holds under two different forms of supervision: automatic IoU ranking on the proxy task and human pairwise preferences here, further supports attributing the final model's gains to the DPO-adapted physical representation rather than to additional generator training.

2. Clarification of Table A2: it varies preference-pair count at a fixed step count

The reviewer’s arithmetic is right: the stage-3 model’s total DPO budget is 500 + 500 = 1000 optimization steps and, coincidentally, also 1000 preference pairs. However, Table A2 could not have settled this question even if it had contained a 1000-sample row: as stated in Appendix C.1, every row is trained “for the same number of steps in Stage II”, i.e., a fixed 500 steps, varying only the number of preference pairs: a 1000-pair row would still cover only half of the total optimization steps. This is exactly why we constructed Controls A and B above, which match the full 500 + 500-step schedule (and the 1000-pair total) explicitly.

Table A2 is nevertheless informative on the data axis: the 1500- and 2500-pair variants already exceed the 1000 pairs consumed by our full pipeline, yet still underperform Stage III on all three average metrics: increasing preference data alone does not account for Stage III’s advantage.

The main pipeline uses the following schedule: Stage I (SFT) 1,000 steps on 7k samples; Stage II 500 steps on 500 pairs; Stage III 500 steps on 500 pairs. The submission reports only wall-clock hours (20 / 15 / 8 h, Sec. 4.1) and batch sizes with learning rates (192 / 96 / 72, lr 1e-5 / 1e-6, Appendix B), which we believe caused this confusion, and we apologize for it.

3. Scope of our claim, and the mixed stage-2 comparisons

We agree that the budget-matched comparisons available in the submission (rows 4 vs 5) are mixed: the encoder variant improves average L2 and CD, while the no-encoder variant has higher average IoU. The encoder variant improves average L2 and CD, while the no-encoder variant has higher average IoU. Our claim, however, has never been that attaching the encoder helps at every intermediate stage. The submission itself reports that after SFT alone the encoder does not help (rows 2 vs 3) and attributes the encoder's value to the final-stage optimization ("DPO unlocks PhysEncoder's potential"): at Stage II the encoder is still its SFT version, so mixed results there are expected under our own hypothesis. The decisive test of the claim is the budget-matched endpoint comparison, which is precisely what Controls A/B provide.

R1-Q1: DPO preference pool and train–test separation

No data leakage occurred between training and evaluation. The set referred to as the "test set" in Appendix B and C.1 is a held-out split relative to the SFT training set, used solely to construct DPO preference pairs. It is disjoint from the evaluation benchmark, and no evaluation video was used in any training stage. We agree that the current naming invites this reading.

R1-Q2: Image conditions for VideoPhy

Protocol. For each of the 344 VideoPhy prompts, we first rewrite the video prompt into an image caption with GPT-6, so that the caption describes the initial state of the scene. From this caption we generate one first frame with GPT-Image-2. The same frame per prompt is then shared by every I2V method we evaluate and by all of our own variants (e.g., the base model, Stage I, Stage III, and the no-encoder pipeline in Table 8). All within-table comparisons in the I2V setting are therefore under identical input conditions.

Expanded comparison under the I2V setting (shared first frames). Under these unified input conditions, we further evaluate more I2V models. The results are shown in the table below.

Table c: VideoPhy comparison under the I2V setting.

ModelSizeSA (↑\uparrow)PC (↑\uparrow)
HunyuanVideo-1.5-I2V8B0.650.650.300.30
Wan2.2-TI2V-5B5B0.600.600.230.23
VLIPP5B0.650.650.180.18
Our base model1B0.590.590.290.29
PhysPilot (ours)1B0.67\mathbf{0.67}0.40\mathbf{0.40}

With a 1B backbone, PhysPilot outperforms I2V baselines of up to 8B parameters on both metrics. The largest margin is on physical commonsense, the metric our method targets: PC 0.400.40 vs. at most 0.300.30 among the I2V baselines. The gap in semantic adherence is small (0.670.67 vs. 0.650.65 for HunyuanVideo-1.5-I2V).

R1-Q3: Training configuration and hyperparameters

We provide the requested details below.

Architecture of the physical head. Following the architecture of Depth Anything V2 (Yang et al., 2024a), PhysEncoder consists of a frozen DINOv2-B backbone (initialized from the Depth Anything V2 encoder weights) and a trainable DPT-style physical head. The head aggregates multi-scale intermediate features into a 64-channel dense representation, which a convolutional latent adapter then projects from 64 to 8 channels to match the latent dimensionality of the video DiT. The resulting 8-channel physics latent is concatenated with the first-frame image latent and the noisy future-frame latents along the temporal dimension before being fed into the DiT. The trainable parameters of PhysEncoder are the DPT head and the latent adapter; the DINOv2-B backbone stays frozen in all stages.

LoRA configuration of the generator. LoRA with rank r=64r=64, α=128\alpha=128 is attached to every linear layer of the attention and FFN modules in each of the 28 DiT blocks: the q/k/v/outq/k/v/\text{out} projections of spatial self-attention, temporal attention, and text cross-attention, plus the three SwiGLU FFN projections.

DPO regularization strength. We use a DPO coefficient of β=0.1\beta = 0.1, with a fixed MSE scaling factor of 0.50.5 and no timestep weighting.

Training hardware and schedule. All stages run on 16× A800 80GB: Stage I (SFT) 1000 steps at global batch size 192 (lr 1e-5); Stage II (DPO on LoRA) 500 steps at batch size 96; Stage III (DPO on PhysEncoder) 500 steps at batch size 72 (both DPO stages at lr 1e-6). Inference runs on a single A800 (26 s for a 5-second video).

Comments on the paper

1 comment

First and foremost, we sincerely thank all Reviewers, Area Chairs, and Program Chairs for their time and thoughtful feedback. We have provided detailed point-by-point responses to all comments from (Reviewer MMiX, Reviewer 8nDd, and Reviewer NdoZ).

We are encouraged by the reviewers' positive recognition of our work:

  • Methodology: A useful idea with a practical implementation: video-level preferences are used to train an image-conditioning encoder without explicit labels for physical properties, and the top-down staged reinforcement learning strategy is effective.
  • Evaluation: Comprehensive experiments covering proxy physical tasks, related-task transfer, and diverse open-world scenarios demonstrate strong physical consistency over existing models such as PhysGen and PISA, supported by informative ablations on training stages, trainable components, encoder architectures, and preference criteria.
  • Presentation: Clear and easy to follow.

In response to the reviewers' valuable suggestions, we have conducted additional experiments and clarified our methodology and contributions. The major questions we addressed are summarized below:

  • Budget-matched ablations for PhysEncoder (Reviewer MMiX). The concern was that the variants without PhysEncoder received one DPO round while our final model received two. We add two ablations that match both the total DPO steps and the number of preference pairs: one never uses PhysEncoder, and the other keeps PhysEncoder frozen and spends the two DPO rounds on the generator. Our final model is better than both on all three average metrics of the proxy task and on open-world scenes, it reaches a physical commonsense score of 0.40, compared with 0.35 and 0.36 for the two variants. This demonstrate that the PhysEncoder provides a meaningful performance improvement.

  • Frozen-representation baseline for the necessity of the trainable physical head (Reviewer NdoZ). We added a baseline that directly uses the frozen pretrained representation as the physical condition under the same three-stage schedule. After SFT alone it brings no stable gain. After the full three stages it brings a partial gain, but our model outperforms it on every metric of both evaluation splits (e.g., the average CD is 5.6% lower, and the IoU on the Unseen split improves from 0.136 to 0.145). The training-free representation probes agree: the adapted feature responds about 10 times more to a change of the physical support state than to a replacement of the background, while this ratio is about 2.5 for the frozen pretrained feature.

  • Justification of Flow-DPO and comparison with Flow-GRPO (Reviewer NdoZ). Online objectives such as Flow-GRPO need a reward for every sampled video, which is not available for open-world physical plausibility due to the lack of useful and automatic reward, while Flow-DPO can be trained on a fixed set of human preference pairs. On the proxy task, where an automatic reward exists, we implemented Flow-GRPO under the same number of training steps. Flow-DPO gives lower trajectory errors (average L2 and CD lower by 6.5% and 9.1%), with comparable IoU.

We have addressed all their concerns through additional experiments and clarifications, and sincerely appreciate the reviewers' constructive feedback, which has helped us strengthen our work.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.