- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes P2Flow, a preference post-training method for flow-based video generation models. Instead of using DPO-style ranking losses, P2Flow perturbs intermediate states along the winner-to-loser latent direction and trains the model with a standard flow-matching MSE loss to predict corrective velocities pointing back to the preferred sample. The authors also introduce a 40.8K video preference dataset annotated via human–VLM consensus and show that P2Flow outperforms SFT and prior DPO baselines on two large video backbones.
Critical strengths
-
The method is lightweight and easy to integrate into existing training pipelines, as it reduces preference learning to a standard flow-matching MSE loss requiring only one forward pass per step and no reference or reward models in memory.
-
The paper contributes a well-curated 40.8K video preference dataset built with semantic balancing across 20K prompt clusters, candidate videos from 8 modern generators, and strict consensus filtering between Gemini-3 and three human annotators.
Critical weaknesses
- Formulating preference optimization purely as an MSE velocity regression anchored to the preferred sample z_w acts more like data-augmented behavior cloning than true preference alignment. First, it explicitly breaks global rectified-flow trajectory consistency (as shown in Equation 9, where the tangent of the constructed state path does not match the target velocity), which may distort the learned vector field. Second, because the objective only regresses toward fixed offline winners, it lacks exploration to discover better generations beyond the training data. Third, the method is largely heuristic and lacks rigorous theoretical justification connecting this local MSE regression to formal preference optimization or reward maximization.
- Since candidate videos for a given prompt are generated independently with different initial noise, the preferred sample z_w and rejected sample z_l often have completely different spatial layouts, camera motions, and object structures. Treating their linear subtraction in VAE latent space as a local preference-violating direction is geometrically questionable; perturbing states toward a structurally misaligned video and forcing the model to regress back to the preferred sample via L2 loss can push intermediate states off-manifold and induce target-averaging artifacts.
Questions for authors
- Since the constructed velocity target breaks global trajectory consistency (Equation 9), does P2Flow increase trajectory curvature and degrade generation quality under few-step ODE sampling (e.g., 5 to 15 steps) compared to the pretrained model and DPO baselines?
- Because preferred and rejected videos come from different models with misaligned spatial layouts, could the authors directly visualize the decoded images of the intermediate weighted features (z_t^pref) across different timesteps? This would clarify whether combining structurally different latents produces meaningful features or severe ghosting artifacts.
- Is the proposed method specifically tailored to video generation? Can it be directly applied to general flow-matching models such as image generation?