- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CorrFlow adds a learned refiner to a flow-matching model to correct its velocity predictions. The refiner uses the base model’s predictions and intermediate features, and is motivated by a connection between clean-sample prediction errors and velocity residuals. The paper provides theoretical analyses and evaluates the method on image generation, text-to-video generation, and image-to-video generation.
Critical strengths
-
The paper is generally well written and easy to follow. The motivation and proposed method are clearly explained, and the appendix provides detailed training and evaluation settings that help clarify the implementation.
-
The qualitative video results are promising. The examples presented for both text-to-video and image-to-video generation look visually convincing, and the comparisons with the baselines suggest improvements in object consistency and the plausibility of the depicted interactions.
Critical weaknesses
-
Limited methodological contribution. The method learns velocity residuals from the base model's clean prediction and reused hidden states, then applies these corrections during sampling. This is a coherent design, but I find the methodological advance incremental. The theoretical analyses provide conditional support for refinement, while offering, in my view, limited additional insight into its practical advantages. The modest ImageNet gains in Table 7 also make the overall contribution less compelling to me.
-
Unclear benefit relative to the additional cost. My main concern is whether the improvements justify the extra computation. Reusing the base model's prediction and intermediate features avoids additional base-model evaluations, but the method still requires an extra refiner evaluation at active sampling steps. The image comparisons with equal training epochs and the small-scale video ablation are useful, but they do not establish an empirical advantage at a comparable computational budget. Although the appendix reports absolute training times, it does not quantify the incremental training cost or end-to-end inference overhead relative to the corresponding base model alone.
Questions for authors
-
What is the main methodological insight gained by predicting corrections from the clean-sample estimate and reused hidden states? Which theoretical or experimental results best demonstrate this insight?
-
Could you report the incremental training cost and end-to-end generation time relative to the corresponding base model alone, under the same hardware and settings? An empirical comparison with continued base-model training at comparable training cost would also help assess whether the quality improvements justify the additional computation.
4 comments
[Q2 & W2] Matched-Compute Performance and Inference Efficiency
We agree that equal training epochs alone do not establish computational efficiency. First, we now quantify the incremental training cost. Including forward, backward and gradient-checkpoint recomputation, the base DiT costs 1.2324 PFLOPs per training example, while the refiner adds approximately 0.181 PFLOPs (+14.7%). Across five epochs on 66,572 clips, this increases adaptation computation from 410.2 to 470.4 EFLOPs, an additional 60.2 EFLOPs.
Second, we have added a full-scale Wan2.1-1.3B-FT-Long baseline that fine-tunes the same pretrained base model on our complete training dataset with the same base objective and optimization settings. FT-Long allocates the refiner’s training budget to 3,054 additional base-model updates, extending training from 20,805 to 23,859 steps and matching CorrFlow’s total adaptation cost of approximately 470.4 EFLOPs. The results are below, with higher values indicating better performance on all metrics.
| Benchmark | Metric | Wan2.1-1.3B | FT-Long | Ours |
|---|---|---|---|---|
| VideoPhy2 ALL | Physical Commonsense | 57.8 | 61.9 | 65.7 |
| Semantic Adherence | 30.0 | 28.0 | 31.3 | |
| Joint | 24.8 | 24.2 | 27.0 | |
| FinePhyEval | Object | 0.84 | 0.88 | 0.90 |
| Action | 0.49 | 0.50 | 0.55 | |
| Physics | 0.36 | 0.39 | 0.44 | |
| Overall | 0.54 | 0.56 | 0.60 | |
| VBench | Subject Consistency | 91.83 | 96.42 | 96.68 |
| Temporal Flickering | 99.17 | 99.61 | 99.81 | |
| Motion Smoothness | 96.51 | 98.79 | 98.85 | |
| Aesthetic Quality | 57.24 | 59.25 | 61.65 | |
| Imaging Quality | 59.14 | 66.96 | 68.66 | |
| Object Class | 76.09 | 76.58 | 82.81 | |
| Spatial Relationship | 62.37 | 72.38 | 77.88 | |
| Temporal Style | 22.23 | 23.43 | 24.05 | |
| Quality Score | 79.81 | 82.83 | 84.69 | |
| Overall Score | 76.93 | 79.48 | 81.86 |
At matched training computation, CorrFlow exceeds FT-Long on every reported metric, including gains of 2.8 percentage points in VideoPhy2 Joint, 0.04 in FinePhyEval Overall and 2.38 points in VBench Overall. Longer base-model training improves visual quality but does not improve VideoPhy2 Joint over the pretrained baseline. These results support allocating the additional training budget to residual refinement.
Third, we have added per-video computation, generation latency and peak GPU memory comparisons for both fp32 and automatic mixed-precision (autocast) refiners, keeping base-model precision and sampling settings fixed. For text-to-video generation at (frames × height × width), CorrFlow adds approximately 6% computation and 5% latency with autocast, compared with 53% latency overhead in fp32. Relative costs below are normalized to Wan2.1-1.3B.
| T2V method | PFLOPs/video | Relative compute | Latency (s) | Relative latency | Peak memory (GiB) |
|---|---|---|---|---|---|
| Wan2.1-1.3B | 28.9 | 1.00× | 89.3 ± 2.4 | 1.00× | 15.4 |
| CorrFlow-1.3B (fp32 refiner) | 30.7 | 1.06× | 136.3 ± 0.2 | 1.53× | 17.8 |
| CorrFlow-1.3B (autocast refiner) | 30.7 | 1.06× | 94.2 ± 0.2 | 1.05× | 17.6 |
| DreamWorld-1.3B | 70.9 | 2.45× | 238.2 ± 0.5 | 2.67× | 21.2 |
| VPT-1.3B | 56.9 | 1.97× | 193.4 ± 0.1 | 2.17× | 19.9 |
| VideoREPA-5B | 88.4 | 3.06× | 294.0 ± 1.1 | 3.29× | 25.1 |
| PhysRAG-5B | 10.8 | 0.37× | 36.1 ± 0.1 | 0.40× | 34.5 |
| Wan2.1-14B | 168.1 | 5.82× | 475.2 ± 0.1 | 5.32× | 46.6 |
CorrFlow requires less computation, lower latency and less peak memory than DreamWorld, VPT, VideoREPA and Wan2.1-14B. PhysRAG is faster and uses fewer FLOPs but requires more memory.
For image-to-video generation at , CorrFlow adds approximately 7% computation and 8% latency with autocast, compared with 59% latency overhead in fp32. The autocast variant has lower latency and peak memory than all compared I2V adaptation methods. Relative costs below are normalized to Wan2.2-5B.
| I2V method | PFLOPs/video | Relative compute | Latency (s) | Relative latency | Peak memory (GiB) |
|---|---|---|---|---|---|
| Wan2.2-5B | 30.6 | 1.00× | 83.6 ± 2.4 | 1.00× | 25.9 |
| CorrFlow-5B (fp32 refiner) | 32.6 | 1.07× | 132.7 ± 0.1 | 1.59× | 29.2 |
| CorrFlow-5B (autocast refiner) | 32.6 | 1.07× | 90.0 ± 0.2 | 1.08× | 29.2 |
| WMReward-5B (guidance only) | 62.7 | 2.05× | 230.7 ± 0.7 | 2.76× | 115.2 |
| PhysRAG-5B | 30.8 | 1.01× | 98.5 ± 0.0 | 1.18× | 48.9 |
| VLIPP-5B (30 steps) | 223.4 | 7.30× | 713.2 ± 0.1 | 8.53× | 45.4 |
These results demonstrate improved quality at matched training computation, with approximately 5–8% additional generation latency when using an autocast refiner.
Dear Reviewer,
Thank you for your valuable feedback. In response to your concerns, we have provided further clarifications and additional experimental analyses in our rebuttal.
We would greatly appreciate it if you could revisit our responses and let us know whether they have adequately addressed your concerns. If your concerns have been resolved, we sincerely hope you would consider raising your rating.
Once again, we sincerely thank you for your time, effort, and thoughtful review of our work.
Best,
Authors
Dear Reviewer,
With about 20 hours remaining before the discussion period closes, we would greatly appreciate it if you could revisit our rebuttal and additional experimental results. If your concerns have been adequately addressed, we sincerely hope you would consider raising your rating.
Thank you again for your time and thoughtful consideration.
Best,
Authors
We thank the reviewer for the constructive review.
[W1] Methodological Novelty and Broad Validation
Firstly, we respectfully disagree that CorrFlow's contribution is incremental. To our knowledge, prior work has not introduced a dedicated refiner that jointly uses a flow model's clean prediction and hidden states from the same forward pass to learn velocity residuals and correct the velocity within the sampling loop. The clean-prediction error identity provides principled supervision. CorrFlow enables the generator to correct its own predictions at each sampling step without additional base-model evaluations.
Secondly, our revised theory explains why this mechanism helps. It identifies learnable residual directions near base-model stationarity, establishes finite-data advantages over direct velocity prediction, and extends these guarantees to moving checkpoints. Our nonlinear AdamW analysis provides sufficient conditions for outperforming continued full-base training at matched compute, while the endpoint analysis connects training risk to generated distributions. An explicit nonlinear construction using our architecture and AdamW verifies these conditions. Experimentally, residual prediction achieves a Quality Score of 0.8029 versus 0.6956 for direct prediction, and CorrFlow outperforms capacity- and compute-matched baselines on all six mean metrics.
Thirdly, we added ImageNet-256 experiments on the original RAE using the RAEv2 protocol. Each Off/On pair shares the same adapted base checkpoint, initial noise and class labels. Enabling the refiner improves all five metrics across both training budgets, with and without AutoGuidance (AG). FID and representation distances decrease while IS increases, demonstrating consistent improvements.
These image improvements are complemented by larger gains in video generation, where coherent motion, object interactions and physical consistency pose greater challenges, demonstrating CorrFlow's value beyond single-image generation.
[Q1] Theoretical and Empirical Evidence for Feature Reuse
Our central insight is that a model can improve its prediction by revisiting both its current answer and the intermediate representations that produced it. Under rectified flow, x^0−x0=σr links clean-prediction error to velocity residual r. The clean estimate provides a prediction to review, while reused hidden states offer contextual features for estimating corrections without additional base-model evaluations.
Theoretical evidence: Our frozen-feature theorem constructs an example where a one-parameter readout of reused features exactly recovers the conditional residual, while raw-input affine predictors retain positive excess risk. Revised finite-data and moving-checkpoint theorems quantify statistical, optimization and feature-drift penalties.
Experimental evidence: Removing text conditioning while retaining hidden-state cross-attention achieves 0.7958, above the no-refiner baseline (0.7894). Removing cross-attention, eliminating hidden-states, reduces the score to 0.7819. Replacing nonlinear hidden-state fusion with weighted averaging lowers it to 0.7914. These comparisons support the value of reused representations and cross-attention.
Related work demonstrates diffusion model's hidden-state feature reuse: Gen4U[1] uses diffusion features for understanding, Next Forcing[2] fuses multilayer features for future prediction, DiT4DiT[3] conditions an action DiT on video features, and VPP[4] uses SVD features for action diffusion. CorrFlow uniquely applies this principle to correcting the generator's own velocity within the sampling loop.
[1] Gen4U: Unifying Video Generation and Understanding via Diffusion, arXiv 2026.
[2] Next Forcing: Causal World Modeling with Multi-Chunk Prediction, NeurIPS 2026.
[3] DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control, arXiv 2026.
[4] Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations, ICML 2025.