acceptodds
Withdrawn from ICLR 2027

Divergence-Suppressing Couplings for Rectified Flow

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The promise of Rectified Flow rests on producing self-generated couplings whose trajectories are straight, or nearly so. In practice, trajectories generated by the base flow model can bend and intertwine, and the resulting coupling inherits this distortion. In this paper, we identify that such trajectory entanglement is often associated with regions of nonzero divergence in the learned velocity field, where local expansion or contraction distorts trajectories and steers particles away from their ideal endpoints. We then propose divergence-suppressing couplings for Rectified Flow, an offline correction that attenuate the divergent component of the learned velocity during coupling generation. The correction is paid only once per coupling pair and amortized over training, so deployment runs plain Euler at identical wall-clock cost to standard Rectified Flow. Empirically, this offline modification yields consistent improvements on 2D synthetic benchmarks and on image generation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Flow Matching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NXNEOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a modification to Rectified Flow. During Training, before the t_stop, in each step some points are sampled in the neighborhood of x, the one with Hutchinson-Div-Estimation score is adopted to replace x. The other parts of the algorithm are the same of Rectified Flow. Experiment shows that the method can achieve better results compared with the original Rectified Flow.

Critical strengths

The method is easy to plug in, with minimal modification to the original Rectified Flow framework.

Critical weaknesses

The experiment scale is limited and some necessary baselines are missing. The authors only conduct experiments on small-res Dataset like CIFAR-10 and CelebA-64.A very relevant baseline "Improving the Training of Rectified Flows" [Lee et.al 2024] which improves the FID of Rectified Flow is not mentioned or compared with.

The motivation for using divergence metric to select neighbors is unclear. Divergence measures local volume change, not straightness. There is little theoretic support for adopting divergence in the paper.

Questions for authors

What is the FID of the cached endpoints x_1 (original and DS)? This would indicate whether the improvements come from a better pairing or better-quality endpoint images.

Do DS couplings show measurably straighter trajectories or lower transport cost?

Reviewer_xMNoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DS-RectFlow, a divergence-guided offline coupling-generation strategy for Rectified Flow. The main idea is to improve the source–target pairs used for reflow by modifying the integration process of a pretrained velocity field. Specifically, the method searches for nearby states with lower estimated absolute divergence and continues integration from the selected states. The resulting coupling pairs are then used to train a standard Rectified Flow model with the original CFM objective. Since the divergence-based correction is only applied during offline pair generation, the final model requires no additional computation at inference time.

The authors evaluate the method on Checkerboard, GMM, CIFAR-10, and CelebA-64. The results show improvements over vanilla Rectified Flow across multiple inference budgets. For example, on CIFAR-10, the single-step FID decreases from 17.276 to 12.272 with Euler-generated couplings and from 9.388 to 8.939 with RK45-generated couplings. While these improvements are encouraging, the proposed approach appears closely related to existing divergence-based sampling methods, particularly FDS [1]. The main distinction lies in applying divergence-guided refinement during offline reflow coupling generation rather than inference. Consequently, stronger comparisons and more targeted ablations are needed to establish the independent contribution of the proposed method.

Critical strengths
  1. The paper explores a practical direction for improving Rectified Flow by modifying offline coupling generation without changing the training objective, model architecture, or inference procedure.

  2. The proposed method is straightforward to implement and can potentially be integrated into existing Rectified Flow pipelines without additional inference overhead.

  3. The experiments demonstrate improvements on both synthetic benchmarks and image-generation datasets, including results with Euler and RK45 coupling-generation solvers.

Critical weaknesses
  1. The methodological novelty is limited compared with existing divergence-based sampling approaches. In particular, FDS [1] already introduces a closely related procedure that samples nearby candidate states, evaluates their divergence, and continues integration from a selected state. The main difference in this paper is applying this mechanism during offline coupling generation, followed by standard reflow [2]. Although FDS is discussed in the related work, the paper does not experimentally establish whether the proposed approach offers benefits beyond directly combining FDS-style sampling with standard reflow. A comparison with FDS-guided coupling generation followed by standard reflow is essential to clarify the contribution.

  2. The divergence-based selection criterion is insufficiently justified and lacks important ablations. FDS [1] motivates minimizing signed divergence through its connection to the conditional flow-matching residual, whereas this paper minimizes absolute divergence. These criteria are not equivalent, particularly in regions of negative divergence where contraction may be necessary for distribution transport. Moreover, the image implementation directly replaces the integration state with a perturbed candidate, making it unclear whether the improvements arise from divergence-based selection or generic stochastic perturbations. Comparisons with signed-divergence minimization, random candidate selection, and velocity-only correction are needed.

  3. The baseline comparisons are insufficient, particularly considering the additional offline computational cost. The paper primarily compares against vanilla Rectified Flow, while related approaches such as FDM [3], Multisample Flow Matching [4], and OT-CFM [5] also address aspects of flow quality or coupling construction. Although these methods operate at different stages, relevant comparisons under matched computational budgets would help establish the practical value of the proposed approach. Furthermore, DS-RectFlow requires multiple forward evaluations and vector–Jacobian products during offline integration, but the paper does not sufficiently compare this cost against simply increasing solver accuracy or performing additional reflow rounds. This is particularly important because the improvements become substantially smaller with RK45-generated couplings.

  4. The paper does not clearly distinguish improved coupling quality from improved endpoint distributions. The proposed correction changes the integration trajectory and may therefore modify both the source–target assignments and the marginal distribution of generated endpoints. Consequently, the downstream FID improvement does not necessarily demonstrate that the constructed couplings are geometrically better. The authors should directly evaluate the endpoint distributions before reflow training and measure whether the resulting couplings actually improve trajectory straightness or reduce conditional velocity ambiguity. The current divergence–compression analysis does not sufficiently establish these claims.

  5. The experimental evaluation is limited to relatively small-scale image-generation settings. CIFAR-10 and CelebA-64 with NCSN++ backbones do not adequately demonstrate the applicability of the method to modern Rectified Flow models. Experiments with flow-transformer architectures or higher-resolution image generation would strengthen the generality claim. In addition, the paper lacks qualitative image-generation comparisons, making it difficult to assess the perceptual effects of the reported FID improvements.

Ref:

[1] Training-Free Refinement of Flow Matching with Divergence-based Sampling.

[2] Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow (ICLR)

[3] Improving Flow Matching by Aligning Flow Divergence.

[4] Multisample Flow Matching: Straightening Flows with Minibatch Couplings. (ICML)

[5] Improving and Generalizing Flow-Based Generative Models with Minibatch Optimal Transport. (TMLR)

Questions for authors

In rebuttal, I want to see:

  1. Can the authors compare DS-RectFlow with FDS [1] applied during offline coupling generation, followed by standard reflow? This comparison seems particularly important for determining whether the contribution extends beyond relocating an existing divergence-guided sampling mechanism to the reflow pipeline.

  2. Why does the proposed method minimize absolute divergence instead of signed divergence as motivated in FDS [1]? Could the authors provide controlled comparisons among signed-divergence selection, absolute-divergence selection, random-neighbor selection, and velocity-only correction?

  3. How does the FID of the corrected endpoint distribution compare with that of standard Euler- or RK45-generated endpoints before reflow training? Can the authors also report direct measures of trajectory straightness, coupling ambiguity, and transport cost to support the coupling-quality explanation?

  4. How does the performance compare under matched total offline computation budgets, including comparisons with more accurate numerical integration, additional reflow rounds, and relevant coupling-construction methods [3–5]?

  5. Can the authors provide qualitative image comparisons and evaluate the approach on a modern flow-transformer architecture or a higher-resolution generation benchmark?

Reviewer_EaaxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DS-RectFlow, a method to improve Rectified Flow by modifying the offline coupling generation process. During reflow, the authors search for nearby states with lower estimated absolute divergence and use them to generate new source-target pairs. The resulting model is trained with the standard flow-matching objective, without additional inference cost. Experiments on synthetic datasets, CIFAR-10, and CelebA-64 show improvements over standard Rectified Flow.

Critical strengths

The method is simple and does not require modifying the training objective or inference procedure.

The paper reports consistent FID improvements on CIFAR-10 and CelebA-64 across different inference budgets.

The experiments include both Euler and RK45 coupling generation, with results averaged over three training runs.

Critical weaknesses

Limited novelty: The proposed divergence-based state selection is closely related to Flow Divergence Sampler (Cha et al., 2026). The main difference is applying the correction during offline coupling generation rather than inference. The authors should better demonstrate the benefits of this distinction.

Missing critical ablations: Comparisons with random neighborhood perturbations, alternative selection criteria, and compute-matched coupling generation would help establish whether the improvements come specifically from divergence suppression.

Limited empirical scope: The experiments are restricted to low-dimensional synthetic datasets and relatively small image-generation benchmarks. The method's effectiveness on larger-scale generative models remains unclear.

Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.