acceptodds
Under review as a conference paper at ICLR 2027

Flipping the Flow: Promoting Diversity in Flow Models via Velocity Inversion

Abstract

Flow-matching models such as FLUX achieve strong generative quality but often concentrate samples around a few modes, limiting diversity. Existing approaches address this through batch-interaction, optimization-based, or guidance-based methods. Batch-interaction and optimization-based methods improve diversity but require joint sample generation and often additional computation, precluding independent sampling. Guidance-based methods incur little computational overhead but typically rely on text-embedding perturbation or classifier-free guidance (CFG) modulation. In this work, we analyze how scaling the velocity field affects differential entropy, showing that negating a contractive velocity field increases entropy and drives trajectories toward low-density regions. Based on this insight, we discover that a surprisingly simple mechanism—reversing the velocity field during early sampling steps—serves as a remarkably effective tool for mode coverage. Our training-free method requires no model modification, text-embedding access, or CFG variation, enabling application across standard and distilled models, spanning image (FLUX, Z-Image Turbo), video (LTX), and 3D generation (TRELLIS). On GenEval, our approach exceeds the diversity range of noise-initialization methods, rivals batch-interaction methods without joint sampling, and outperforms guidance-based methods under worst-case quality.

Primary area:
generative models
Keywords:
flow matching, improve diversity

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PbLsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a training-free method for improving diversity in flow-matching models by reversing the velocity field during early sampling steps. The paper designs a ease-out schedule for early exploration and gradually restores the original velocity for later refinement. The method is motivated by the relationship between velocity divergence and differential entropy and requires no additional network evaluations or interactions between samples. Experiments show promising diversity-quality trade-offs, including evaluations using minimum quality per prompt. Additional experiments demonstrate its effectiveness on distilled image models, video generation, 3D generation, and unconditional image generation.

Critical strengths
  1. The proposed intervention method is simple and practical. It modifies only the predicted velocity, preserves independent sampling, and introduces negligible additional computation.
  2. The evaluation on image generation is comprehensive, covering multiple baseline families and several diversity and quality metrics. Reporting minimum quality and mean quality is particularly useful for checking whether diversity gains come from a few poor samples.
  3. The experiments provide promising evidence that velocity inversion changes meaningful image structure. The ablation studies and evaluation complement the qualitative results and support the proposed method.
Critical weaknesses
  1. The theoretical analysis does not fully justify the claimed improvement in mode coverage. In Equation (10), the assumption that the expected divergence under the corrected sampling distribution is negative, may not remain valid throughout the entire sampling trajectory. And, an increase in differential entropy does not necessarily imply improved mode coverage, as it could instead come from greater dispersion within existing modes.
  2. The ablation studies are insufficient to clearly isolate the contribution of velocity inversion. The proposed approach simultaneously modifies the direction and magnitude of the velocity and the cumulative displacement along the trajectory. Conducting more rigorous comparative experiments, e.g. controlling for perturbation intensity or output quality, would help determine whether velocity inversion itself is the key factor driving the observed performance improvements.
  3. The evaluation mainly relies on feature-based diversity metrics, which cannot clearly distinguish the recovery of rare semantic modes from variations in viewpoint, composition, or appearance. More explicit evaluations of semantic mode coverage on real-world conditional generation tasks would strengthen the results.
Questions for authors
  1. Can the authors report divergence statistics across prompts under the modified sampling trajectories, including how often the negative-divergence assumption fails during the velocity inversion interval?
  2. Can the authors compare the proposed method against non-negative time-dependent velocity schedules with comparable perturbation strengths or output quality?
  3. How sensitive is the proposed method to the number of sampling steps? Does the diversity–quality trade-off remain consistent when varying the sampling budget?
3 comments
AuthorsOpenReview ↗

Response to Reviewer PbLs: W1, Q1

W1 / Q1. Does the negative-divergence assumption hold along the trajectory? Divergence statistics across prompts and failure rate in the inversion interval.

In our experiment, E[∇⋅vθ]<0\mathbb E[\nabla\cdot v_\theta] < 0 at every step, and ∇⋅vθ<0\nabla\cdot v_\theta < 0 in all 1,000 / 2,000 / 2,500 inverted (trajectory, step) pairs for η = 1.5 / 2 / 2.5. The experiment estimates the divergence of the unscaled velocity with Hutchinson’s estimator (4 Gaussian probes per step) on 500 trajectories (50 T2I-CompBench validation prompts × 10 seeds; FLUX.1-dev, 20 steps). The entropy estimate rises at every inverted step and falls once αk>0\alpha_k > 0, while for Base it decreases monotonically (Table R3).

Table R3. Per-step statistics over the 500 trajectories, mean ± SE over prompts: (a) E[∇⋅v]\mathbb E[\nabla\cdot v] of the unscaled velocity; (b) entropy change Hk−H0H_k - H_0 (nats). Bold = inverted steps (αk<0\alpha_k < 0).

(a) E[∇⋅v]\mathbb E[\nabla\cdot v], where v=vθ(xt,t)v = v_\theta(x_t, t) is the model’s velocity before our scaling, not αkv\alpha_k v.

kkBaseOurs, η = 1.5Ours, η = 2.0Ours, η = 2.5
0−65,506 ± 50−65,506 ± 50−65,506 ± 50−65,506 ± 50
1−65,335 ± 52−65,277 ± 47−65,223 ± 46−65,157 ± 46
2−66,026 ± 47−65,680 ± 48−65,491 ± 52−65,325 ± 52
3−66,780 ± 42−66,205 ± 49−65,920 ± 54−65,663 ± 55
4−66,942 ± 41−66,280 ± 50−65,926 ± 55−65,563 ± 55
5−67,745 ± 37−67,060 ± 55−66,465 ± 60−65,878 ± 59
6−68,024 ± 36−67,391 ± 57−66,607 ± 61−65,855 ± 60
7−68,566 ± 38−68,067 ± 63−67,083 ± 63−66,184 ± 65
8−69,197 ± 41−68,821 ± 63−67,833 ± 65−66,846 ± 65
9−70,399 ± 47−70,237 ± 66−69,519 ± 61−68,609 ± 62
10−71,404 ± 50−70,906 ± 69−70,160 ± 58−69,141 ± 60
11−72,571 ± 54−71,845 ± 75−71,169 ± 60−70,190 ± 57
12−74,254 ± 57−73,338 ± 79−72,719 ± 61−71,823 ± 56
13−76,707 ± 53−74,700 ± 52−73,445 ± 52−71,411 ± 92
14−79,775 ± 61−77,158 ± 57−75,688 ± 68−73,411 ± 104
15−84,200 ± 83−80,885 ± 71−79,159 ± 89−76,606 ± 130
16−90,880 ± 124−86,484 ± 98−84,375 ± 122−81,354 ± 166
17−102,340 ± 224−95,756 ± 168−93,035 ± 200−89,081 ± 259
18−128,882 ± 604−113,278 ± 470−108,339 ± 516−101,392 ± 531
19−309,737 ± 10,412−95,300 ± 3,481−80,271 ± 2,730−63,084 ± 1,687

(b) Hk−H0H_k - H_0 (nats)

kkBaseOurs, η = 1.5Ours, η = 2.0Ours, η = 2.5
0−361 ± 0.3+180 ± 0.1+361 ± 0.3+541 ± 0.4
1−758 ± 0.6+266 ± 0.2+607 ± 0.4+948 ± 0.7
2−1,204 ± 0.9+259 ± 0.2+745 ± 0.5+1,230 ± 0.8
3−1,707 ± 1.1+158 ± 0.2+777 ± 0.5+1,393 ± 0.9
4−2,274 ± 1.4−41 ± 0.2+699 ± 0.5+1,436 ± 0.9
5−2,924 ± 1.7−347 ± 0.4+507 ± 0.4+1,355 ± 0.9
6−3,668 ± 1.9−771 ± 0.8+190 ± 0.5+1,144 ± 0.8
7−4,532 ± 2.3−1,334 ± 1.3−267 ± 0.8+788 ± 0.8
8−5,547 ± 2.7−2,063 ± 1.9−893 ± 1.3+262 ± 0.9
9−6,764 ± 3.4−3,003 ± 2.7−1,734 ± 2.0−478 ± 1.4
10−8,242 ± 4.3−4,203 ± 3.8−2,832 ± 2.8−1,473 ± 2.2
11−10,071 ± 5.5−5,745 ± 5.3−4,272 ± 3.9−2,806 ± 3.2
12−12,400 ± 7.2−7,770 ± 7.4−6,188 ± 5.3−4,609 ± 4.4
13−15,477 ± 8.8−10,476 ± 8.8−8,753 ± 6.6−7,009 ± 6.7
14−19,714 ± 11−14,251 ± 11−12,351 ± 8.9−10,397 ± 11
15−25,914 ± 17−19,827 ± 15−17,684 ± 14−15,438 ± 19
16−35,813 ± 29−28,760 ± 23−26,241 ± 25−23,536 ± 34
17−53,982 ± 66−45,049 ± 47−41,836 ± 56−38,248 ± 76
18−97,908 ± 267−82,348 ± 195−77,091 ± 217−70,852 ± 241
19−106,938 ± 552−85,050 ± 285−79,346 ± 287−72,607 ± 281
AuthorsOpenReview ↗

Response to Reviewer PbLs: W1, W2, Q2, Q3

W1 / W2 / Q2. The theoretical analysis does not fully justify the claimed improvement in mode coverages; non-negative schedules at matched strength or quality.

We agree that increased entropy alone does not imply diversity, as adding noise also increases entropy. We also acknowledge that our theory does not fully explain mode coverage. A guarantee would require characterizing how the learned velocity field partitions the noise space into modes, which is beyond current analysis. We therefore rely on the empirical evidence below.

To investigate the contribution of velocity inversion, we compare against the following schedules:

  • Our exponential function but with η = 1: αk=1−ηe−xk/σ\alpha_k = 1 - ηe^{-x_k/\sigma}, which never goes below 0.
  • Ramps: αk=g(min⁡(1,xk/w))\alpha_k = g(\min(1, x_k/w)), with
    • linear: g(u)=ug(u) = u
    • cosine: g(u)=(1−cos⁡πu)/2g(u) = (1 - \cos \pi u)/2
    • exponential: g(u)=(e5u−1)/(e5−1)g(u) = (e^{5u} - 1)/(e^5 - 1)

For each η of our main sweep (1.5, 2.0 and 2.5; Sec. 5), we set σ or w so that each schedule’s perturbation strength, measured as the average of αk\alpha_k over all steps, matches ours. All runs use FLUX.1-dev, 20 steps, 10 images per prompt and the same starting noise, on 50 GenEval test prompts.

Table R4. All rows within a block have the same perturbation strength (mean α, column 2). All metrics are higher-is-better; ▼ / ▲ = statistically significantly lower / higher than ours in the same block (two-sided paired t-test over prompts, p < 0.05).

MethodMean αCLIP ↑HPSv2 ↑DINO ↑DreamSim ↑
Matched strength: mean α = 0.61 (ours at η = 1.5)
Ours, η = 1.50.61128.0928.97.705.425
Our function, η = 1, σ = 0.420.61028.66▲28.74▼.687▼.396▼
Linear ramp, w = 0.770.60928.1129.07▲.698.409▼
Cosine ramp, w = 0.770.60927.9929.00.704.413▼
Exponential ramp, w = 0.480.60827.76▼28.81▼.721▲.443▲
Matched strength: mean α = 0.48 (ours at η = 2.0)
Ours, η = 2.00.48127.1028.18.753.503
Our function, η = 1, σ = 0.660.48027.66▲27.79▼.706▼.375▼
Linear ramp, w = 1.040.48127.2228.03.725▼.426▼
Cosine ramp, w = 1.040.48126.6627.99▼.743.472▼
Exponential ramp, w = 0.640.48524.82▼27.17▼.777▲.542▲
Matched strength: mean α = 0.35 (ours at η = 2.5)
Ours, η = 2.50.35124.7426.90.786.555
Our function, η = 1, σ = 1.060.35024.8925.62▼.695▼.266▼
Linear ramp, w = 1.420.35224.8025.78▼.717▼.308▼
Cosine ramp, w = 1.340.35124.4725.57▼.750▼.401▼
Exponential ramp, w = 0.810.35420.09▼23.54▼.736▼.475▼

At η = 1.5 and 2.0, ours is more diverse on DreamSim than every schedule except the exponential ramp, which gains diversity only by losing quality: at η = 2.0 its CLIP drops to 24.82 (ours 27.10), consistent with corrupted images. At η = 2.5, ours is more diverse than every schedule on DINO and DreamSim and has higher HPSv2.

Q3 Is the trade-off consistent across budgets? Sampling step sensitivity?

The diversity–quality trade-off is consistent across budgets in terms of rank order: at every N total number of sampling steps, a larger η gives higher diversity (DINO and DreamSim: η = 1.5 < 2.0 < 2.5) and lower CLIP (η = 1.5 > 2.0 > 2.5); only the values shift slightly. Setup: we reran Base and ours (η = 1.5, 2, 2.5) with N = 15, 20, 25 and 30 steps on 50 validation prompts. Note that the schedule is defined on the normalized step index xk=k/(N−1)x_k = k/(N-1), so the inverted window ends at nearly the same noise level for every N.

Table R4. Mean ± SE over prompts. Quality: CLIP; diversity: DINO, DreamSim.

N = 15N = 20N = 25N = 30
(Quality) CLIP ↑Base26.88±.3726.82±.3726.77±.3926.66±.39
η=1.526.84±.3726.90±.3727.05±.3627.08±.37
η=2.026.02±.3526.30±.3626.36±.3726.49±.38
η=2.524.66±.3225.02±.3325.13±.3625.07±.34
(Diversity) DINO ↑Base.639±.009.642±.009.642±.009.645±.009
η=1.5.729±.007.727±.007.729±.007.729±.007
η=2.0.759±.006.754±.006.754±.006.755±.006
η=2.5.776±.004.770±.004.770±.004.770±.004
(Diversity) DreamSim ↑Base.284±.013.284±.013.286±.013.289±.013
η=1.5.425±.013.416±.013.415±.013.413±.013
η=2.0.511±.010.494±.010.492±.010.489±.010
η=2.5.553±.007.541±.007.538±.007.535±.007
AuthorsOpenReview ↗

Response to Reviewer PbLs: W3

W3. Feature-based diversity metrics cannot separate rare semantic modes; more explicit evaluation of semantic mode coverage is required.

We thank the reviewer for the suggestion.

Setup. To measure whether a method reaches different semantic modes, we follow GRADE [Rassin et al., 2024] by using a VLM to detect predefined modes in each image and measuring their spread with the normalized entropy Ĥ.

We use 50 underspecified prompts specifying various objects / scenes (e.g., “a photo of a vase”, “a photo of a fox”, …) and generate 10 images for each prompt. To evaluate images from a given prompt (e.g., "..frog"), we first predefine five kinds of attributes (e.g., "color", "action", "kind"), each with 10 possible values, including or “none of these / cannot tell”. These attributes and their values are different for each prompt and are designed by Claude Opus 5.5. During evaluation, we use Qwen2-VL-7B to classify each image into one of the 10 values for each attribute. We measure mode coverage as the normalized entropy Ĥ of these classes, and report CLIP score (text alignment) and HPSv2 (image quality).

We compare against representatives of each baseline type: IG, CNO, SPELL, with matched quality: For each baseline, we tuned their hyperparameters using a separate validation set (also 50 prompts x 10 images) and picked the ones whose CLIP and HPSv2 scores are at most 1% below those of Base (one-sided paired t-test, p < 0.05).

Table R5. Mean ± SE over prompts. ▼ / ▲ = statistically significantly lower / higher than ours. All marks: two-sided paired t-test over prompts, p < 0.05.

MethodCLIP ↑HPSv2 ↑Ĥ ↑
Base, cfg 3.524.64 ± 0.2428.70 ± 0.120.242 ± 0.014 ▼
IG [0.01, 0.99]24.59 ± 0.2628.70 ± 0.120.267 ± 0.013 ▼
CNO, γ = 124.62 ± 0.2728.65 ± 0.120.268 ± 0.016 ▼
SPELL, r = 15024.53 ± 0.2628.61 ± 0.140.248 ± 0.015 ▼
Ours, η = 1.2524.58 ± 0.2628.70 ± 0.130.330 ± 0.014

Summary. Ours has a statistically significantly higher Ĥ than every other baselines. Note that the test CLIP and HPSv2 scores of all baselines remain comparable when they were tuned for matched quality on the validation set.

Qualitatively, examples of rare semantic modes only our method produced include:

  • Object categories: a saxophone or a trumpet for “a photo of a musical instrument” (normal: guitar, piano, violin);
  • Species: a dragonfly for “a photo of an insect” (normal: butterfly, bee, ladybug), and a horned viper for “a photo of a snake” (normal: python, rattlesnake, garter snake);
  • Dishes: waffles for “a photo of a breakfast” (normal: eggs and bacon, pancakes, cereal with milk);
  • Subjects: a mythical creature for “a photo of a sculpture” (normal: human figure, bust or head, abstract shape).
Reviewer_geCjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Training-free diversity for flow samplers: scale the velocity by α(t) = 1 − η·exp(−t/σ), negative in early steps (velocity inversion), → 1 later. Motivated by dH/dt = E[∇·v]: negating a contractive field raises entropy. Needs no CFG branch or text access, so it works on distilled models. Evaluated on FLUX.1-dev/GenEval (553 × 10) vs. 10 baselines with quality cut-offs and a per-prompt minimum-quality metric; also Z-Image Turbo, LTX video, TRELLIS 3D, LSUN P/R/D/C.

Critical strengths
  1. One-line, single-pass, model-agnostic; applicable where CFG-based methods cannot run.

  2. Careful evaluation: 10 baselines, quality cut-offs, min-quality metric exposing outlier-inflated diversity; losses (3D TVA, video) reported.

Critical weaknesses
  1. Mechanism not isolated; entropy argument does not support “reaches different modes.” Fig. 1 shows non-negative scaling and noise injection reach the same entropy without diversity, so entropy is not the distinguishing factor. Alternative explanation untested: near t≈0, v ≈ E[x₁|x] − x, so a reversed step inflates the noise norm and pushes away from the conditional mean while the timestep input no longer matches the latent. Missing controls: initial-noise temperature k>1, time-shifted start, and a proper SDE/churn sampler (Fig. 13 only tests uncompensated noise). Appendix C’s v = −∇log p(x|y) is incorrect for flow matching.

  2. Main results are curves only; some baselines appear non-functional. No table for Fig. 2, so Pareto claims cannot be verified. SPELL r=110/125/150 in Table 1a are identical to four digits (repellency never triggers); the significant gain over CNO is DINO only. LSUN P/R/D/C: 2,000 samples, no error bars.

Questions for authors
  1. Add noise-temperature, time-shift and SDE-sampler controls at matched quality.

  2. Provide a numeric table for Fig. 2; state the number of trajectories for the Hutchinson estimates.

  3. Why are the three SPELL settings identical?

2 comments
AuthorsOpenReview ↗

Response to Reviewer geCj

W1 / Q1. Mechanism not isolated; entropy argument does not support "reaches different modes." / Add noise-temperature, time-shift and SDE-sampler controls at matched quality.

We agree that increased entropy alone does not imply diversity. What distinguishes the three strategies in Fig. 1 is the direction of the update. Given that the learned velocity at xtx_t points along the line through xtx_t and its posterior mean x^1=E[x1∣xt]\hat x_1 = \mathbb E[x_1 \mid x_t] (see our reply to ZLZG's Q1):

  • Isotropic noise is nearly orthogonal to this direction in high dimension, so it moves the sample off the flow line (App. C.1).
  • Non-negative scaling (α≥0\alpha \ge 0) stays on the flow line but only moves toward x^1\hat x_1, more slowly. The outcome can change only through the drift of x^1\hat x_1 as the timestep advances while the sample lags. This gives limited diversity, and raising entropy further produces high-frequency noise before meaningful diversity appears (Sec. 5.1).
  • Ours (α<0\alpha < 0) also stays on the flow line but additionally moves away from x^1\hat x_1, undoing part of the sample’s commitment to its current posterior mean, so later steps can settle on a different mode.

We ran the suggested controls (FLUX.1-dev, 20 steps, 10 seeds per prompt, same initial noise), tuned on 50 T2I-CompBench validation prompts and tested on 50 GenEval prompts. For each control, we sweep its strength parameter from a value that is non-inferior to Base, meaning its CLIP Score and HPSv2 are shown to be at most 1% below those of Base (one-sided paired t-test over prompts, p < 0.05), until a value that fails this criterion.

  • Noise temperature: initial noise z∼N(0,k2I)z \sim \mathcal N(0, k^2 I), k∈{1.05,1.1,1.25,1.5,2}k \in \{1.05, 1.1, 1.25, 1.5, 2\}. In our truncated table below, we show k = 1.05 and 1.1, whose validation CLIP is below and above ours; k = 1.25 already fails the bar (HPSv2 statistically significantly lower than Base).
  • Time-shifted start: the latent is frozen for the first m∈{1,…,10}m \in \{1, \dots, 10\} steps while the time input advances. Shown: m = 4 and m = 5, the latter with statistically significantly lower CLIP than ours.
  • SDE / churn sampler: at each inverted step of ours (αk<0\alpha_k < 0, except the first), fresh Gaussian noise moves the sample back onto the marginal at t′=max⁡(0,tk+αkΔt)t' = \max(0, t_k + \alpha_k \Delta t); the network is evaluated at t′t', and an Euler step goes to tk+1t_{k+1} (no velocity scaling). η∈{1.5,2.0,2.5}\eta \in \{1.5, 2.0, 2.5\} (Sec. 5), our window. Shown: the most diverse setting meeting the bar, η = 1.5.

Table R2. Mean ± SE over prompts. ▼/▲: statistically significantly lower/higher than ours (two-sided paired t-test, p < 0.05).

MethodCLIP ↑HPSv2 ↑DINO ↑DreamSim ↑
Base28.54±.5129.26±.13▲.571±.016▼.302±.017▼
Temp. k=1.0528.68±.4829.28±.14.545±.017▼.289±.015▼
Temp. k=1.128.71±.5029.11±.15.533±.017▼.283±.015▼
Temp. k=1.2528.38±.5027.52±.18▼.643±.014▼.387±.012
Time-shift m=428.16±.5029.18±.13.661±.013▼.369±.016▼
Time-shift m=527.95±.51▼29.15±.13.677±.011.388±.017
SDE η=1.528.71±.5129.21±.13.570±.015▼.298±.016▼
Ours28.35±.4929.14±.13.680±.012.394±.016

Ours is statistically significantly more diverse than every setting on DINO, except time-shift m = 5, which has statistically significantly lower CLIP than ours.

W2 / Q3. Why are the three SPELL settings identical? SPELL's repellency, ReLU$(r - d)$, is zero unless the predicted clean samples are closer than rr. On Z-Image (50 prompts × 10 seeds), r=110r = 110 and 125125 change no image, r=150r = 150 changes 5 of 500, and r=175r = 175 and 200200 change 34 and 106. The identical rows thus lie below SPELL's working range, and at r=200r = 200 its quality already drops below Base.

W2 / Q2. Outperform CNO only on DINO for Z-image; LSUN error bars; #Hutchinson trajectories. We will tone down the claim with respect to CNO. Our two main advantages over CNO are cost and independent sampling: 1). Ours uses exactly the base sampler’s 8 evaluations per image, whereas CNO’s noise optimization adds 3–5 more. 2). CNO jointly optimizes a fixed-size batch of initial noises per prompt, which becomes harder at scale (> 2,000 samples), and drawing additional samples after optimization is not supported. Ours generates each image separately, so any number of samples can be drawn, one at a time or in parallel.

LSUN error bars (mean ± SE):

Precision ↑Recall ↑Density ↑Coverage ↑
Base.595 ± .011.459 ± .017.885 ± .038.785 ± .015
Ours.621 ± .011.465 ± .018.954 ± .039.823 ± .014

Hutchinson: Fig. 1 used one prompt, 10 seeds and one probe per step. We recomputed it on 500 trajectories (50 prompts × 10 seeds, 4 probes per step): the entropy estimate rises at every inverted step and falls afterwards, while for Base it decreases monotonically (PbLs response, W1/Q1).

AuthorsOpenReview ↗

Numeric table for Fig. 2

MethodSettingCLIP ↑Pick ↑HPS ↑min CLIPmin Pickmin HPSDINO ↑DreamSim ↑Vendi ↑MSS ↓
BaseCFG 3.532.5623.1329.2529.8422.3128.290.5670.2992.570.316
CFG 3.032.4923.2129.3629.8422.2828.210.5790.2992.590.300
CFG 2.032.1723.0828.9029.0822.0327.710.6260.3472.820.233
CFG 1.531.7522.6728.2828.1721.4226.960.6660.4103.090.177
CFG 1.029.9921.5226.8825.2619.9725.120.7220.5153.610.112
IG-Marg.[.01,.99]31.2223.0528.9327.5021.8127.560.6760.3732.920.211
[.10,.90]28.7221.9827.6922.6519.7625.800.7670.4933.530.151
[.20,.80]19.0418.3624.8013.3816.4923.430.8470.5944.260.100
IG[.01,.99]32.0223.2129.2728.9122.1528.030.6290.3292.730.243
[.10,.90]31.5222.9028.8227.9921.5827.390.6670.3802.960.205
[.20,.80]30.6422.3128.0726.3020.6626.300.7060.4463.320.160
[.30,.70]29.9621.9027.5625.2020.1525.670.7240.4773.500.138
[.40,.60]29.4921.5927.0724.5019.8725.180.7290.4943.620.126
CADSs = 0.0131.3122.8328.6727.3721.1126.840.6970.4123.000.182
s = 0.131.2422.7228.6127.1920.8326.600.7040.4323.060.171
s = 0.230.7122.5328.3623.6819.7024.730.7180.4743.240.158
s = 0.329.8822.1428.0018.5417.8022.190.7500.5433.530.131
s = 0.429.8722.0127.8118.7217.4921.190.7540.5573.590.124
s = 0.827.5821.2226.5118.7317.7120.370.7790.6724.100.110
OG+STPs = 132.4122.8329.0129.1921.7127.640.6430.3822.850.244
s = 1.2532.3822.7128.9328.9321.5427.370.6370.3842.870.253
s = 1.532.3222.5728.8028.6921.3427.140.6330.3912.910.257
s = 1.7532.3022.9029.0129.0321.7427.670.6510.3832.850.232
s = 232.1222.9428.9528.6821.7827.650.6630.3882.880.216
s = 330.9521.8327.8026.2620.3725.440.6330.4363.080.250
OG+DTPs = 125.0020.2325.8817.2417.5620.730.8080.6624.300.087
s = 224.7119.8224.7517.0417.5521.210.7200.6214.120.145
s = 323.4319.3723.7115.9417.4222.490.6590.5623.600.195
SGIλ = 0.932.5223.1428.6329.0621.5327.480.6660.3762.850.216
λ = 20.032.5022.7828.3928.4121.1227.270.6660.3762.890.192
CNOγ = 0.8532.5123.0729.3529.6122.0828.010.5790.3262.640.292
γ = 1.032.5123.0729.3529.6622.0828.020.5910.3372.660.273
γ = 100.032.4823.0829.3629.6022.0928.040.5840.3282.650.286
NegToMeα = 0.932.3123.0129.4929.3221.8828.100.6630.3822.850.226
α = 2.031.8822.5728.9028.2421.1927.310.6820.4323.110.193
α = 3.031.4822.2928.5227.3620.7326.670.6930.4553.260.173
α = 4.031.0822.0728.3126.5320.3426.150.7050.4713.390.160
SPELLr = 15032.5023.2129.4529.8022.3228.290.5640.2902.550.320
r = 17532.3723.0929.3329.4121.9927.950.5770.3132.650.299
r = 20031.1122.4428.4826.4020.6726.070.6180.3933.070.251
r = 22528.6221.3826.9222.9719.3523.990.6520.4633.500.219
r = 25026.5920.5825.6521.2418.7322.970.6460.4643.550.238
r = 27525.4420.0524.8020.9818.5622.610.6150.4203.390.287
Oursη = 1.2532.2222.9729.1328.8021.8127.860.6720.3822.830.200
η = 1.532.0222.7828.9528.3521.4727.590.7000.4182.970.178
η = 1.7531.7222.4528.6527.7320.9927.170.7240.4613.130.159
η = 2.031.1421.9728.2026.5820.2326.390.7470.5073.370.140
η = 2.2530.3121.4027.6424.8219.3725.370.7670.5423.630.127
η = 2.529.2420.7927.0123.1118.6724.400.7790.5613.850.123
Reviewer_ZLZGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors present a training free approach to improve the diversity of flow matching models at inference. The proposed method is simple: flipping the direction of the flow velocity in some early timesteps, justified by a useful identity between differential entropy and the velocity divergence. The authors show quality-diversity tradeoff w.r.t. existing diversity methods (noise-optimization based, batch repulsion-based, guidance-based, etc.). Through experiments over various tasks, like image, video and 3D generation, the authors demonstrate the efficacy of their method, which also comes at negligible overhead to compute.

Critical strengths

Critical Strengths

CS1. The authors propose an effective inference-time approach for improving diversity in flow matching models, based on an entropy identity. The method is fairly simple to implement and has practically no computational overhead. As such, from a utility perspective, the method can be useful to the community.

CS2. The authors provide a wide array of experimental results over different models, tasks, and datasets. The breadth of the experiments make me more confident in the practical utility of the method.

This is not a critical strength, but I also appreciate the way the authors organize the various “schools of thought” in improving diversity in diffusion/flow matching models in their Related Work section.

Critical weaknesses

CS1. Underxplained Effect on Score and Conditional Density

I think the paper needs significantly more exposition on what actually happens to the associated score function and the underlying densities if the velocity is multiplied with a negative constant. The authors mention in L852 that:

By setting α(t)<0\alpha(t) < 0, the flow follows −∇log⁡p(x∣y)-\nabla \log p(x \mid y), repelling samples from high-density regions of the conditional distribution.

But I believe this interpretation can be incorrect. I will try to explain with an example:

The exact identity between the velocity and the score is vt(x)=atx+bt∇log⁡p(x∣y)v_t(x) = a_t x + b_t \nabla \log p(x \mid y) (given by Lemma 1 in Zheng et al. 2022). The flipped velocity is then −vt(x)=−atx−bt∇log⁡p(x∣y)-v_t(x) = -a_t x - b_t \nabla \log p(x \mid y).

We can look at a simple example in 1D: consider X0∼N(0,1),X1∼N(5,σ2)X_0 \sim \mathcal{N}(0,1), \quad X_1 \sim \mathcal{N}(5,\sigma^2). Using the linear path Xt=(1−t)X0+tX1,X_t = (1-t)X_0 + tX_1, the exact flow-matching velocity at t=0t=0 is v0(x)=E[X1−X0∣X0=x]=5−x.v_0(x) = \mathbb{E}[X_1-X_0 \mid X_0=x] = 5-x.

However, if we look at the score, it is ∇xlog⁡p0(x)=−x.\nabla_x \log p_0(x) = -x.

Now consider a sample x=1x=1. The negative velocity is −v0(1)=−4,-v_0(1) = -4, while the neg score is −∇xlog⁡p0(1)=+1.-\nabla_x \log p_0(1) = +1.

The negative velocity -4 points in the negative direction from 1, toward x=0x=0, which is actually the high-density region of the initial Gaussian N(0,1)\mathcal{N}(0,1). This is opposite to the actual negative score, which correctly points to +1 (away from the center 0), the low density region. Therefore, the negative velocity does not necessarily align with the negative score function, and does not universally repel samples from high-density regions.

Given that the negative multiplier is actually being applied to the CFG guided velocity, more unusual effects can occur. Please refer to Q1, where I elaborate on this further.

CS2. Critical Weakness 2

The authors mention that they work with standard and distilled models (L24, contribution 3), where the standard model is FLUX.1-dev. However, FLUX.1-dev is actually a distilled model as well, specifically a guidance distilled model from FLUX.1-Pro. That is, FLUX.1-dev actually does not do true classifier free guidance ((1−w)⋅vt(x)+w⋅vt(x∣y)(1-w) \cdot v_t(x) + w \cdot v_t(x \mid y)). Instead, through guidance distillation, it is parameterized to directly estimate the guided velocity in the form vt(x∣y,w)v_t(x | y, w).

In their experiments, do the authors have any regular models that are doing true CFG?


Zheng et al. 2022, "Guided Flows for Generative Modeling and Decision Making".

Questions for authors

Question 1. Could the authors elucidate what exactly happens to the conditional and unconditional densities under classifier free guidance?

Consider the guided velocity v~t(x∣y)=(1−w)⋅vt(x)+w⋅vt(x∣y)\tilde{v}_t(x \mid y) = (1-w) \cdot v_t(x) + w \cdot v_t(x \mid y), with the associated density p~t(x∣y)∝pt(x)1−w⋅pt(x∣y)w\tilde{p}_t(x \mid y) \propto p_t(x)^{1-w}\cdot p_t(x \mid y)^w, a sort of geometric average between the conditional and unconditional densities.

The closest related method, opposite guidance (OG), has the velocity v~tOG(x∣y)=(1+w)⋅vt(x)−w⋅vt(x∣y)\tilde{v}^{OG}_t(x \mid y) = (1 + w) \cdot v_t(x) - w \cdot v_t(x \mid y). By using Lemma 1 from Zheng et al., we can derive v~tOG(x∣y)=atx+bt∇log⁡[pt(x)1+w⋅pt(x∣y)−w]\tilde{v}^{OG}_t(x \mid y) = a_t x + b_t \nabla \log \big[p_t(x)^{1+w}\cdot p_t(x \mid y)^{-w} \big], and subsequently say p~tOG(x∣y)∝pt(x)1+w⋅pt(x∣y)−w\tilde{p}^{OG}_t(x \mid y) \propto p_t(x)^{1+w}\cdot p_t(x \mid y)^{-w}. This has a nice interpretation of reweighting the conditional and unconditional densities.

For flip the flow (which I am referring to as FF), let's assume α(t)=−1\alpha(t) = -1 for simplicity.

The FF velocity is v~tFF(x∣y)=−v~t(x∣y)=(w−1)⋅vt(x)−w⋅vt(x∣y)\tilde{v}^{FF}_t(x \mid y) = -\tilde{v}_t(x \mid y) = (w - 1) \cdot v_t(x) - w \cdot v_t(x \mid y). Using Lemma 1, we can obtain v~tFF(x∣y)=vtFF=−atx+bt∇xlog⁡[pt(x)w−1pt(x∣y)−w]\tilde{v}^{FF}_t(x \mid y) = v_t^{FF} = -a_t x + b_t \nabla_x \log \left[p_t(x)^{w-1}p_t(x\mid y)^{-w}\right]. Since the drift −atx-a_t x is negative, it is not as straightforward to derive/interpret p~tFF\tilde{p}_t^{FF}. That is, we cannot interpret the effect of FF as a simple reweighting as in OG.

It would be helpful if the authors continued a theoretical analysis like this, and explained how the effect of FF on the cond/uncond densities is different from that of OG.

Question 2. Is the proposed approach marginal preserving, or does it preserve the target distribution?

Having inspected the method, it seems to me the method is not marginal-preserving along the probability path, nor does it conserve the actual target distribution. The authors touch the drift problem in appendix C2, but as I note in W1, the argument made by the authors is not entirely correct (unless they convince me otherwise over the rebuttal.)

Note that, the method does not necessarily need to be marginal-preserving or target distribution conserving to be useful. We only want to identify these properties in a clear manner.

2 comments
AuthorsOpenReview ↗

Response to Reviewer ZLZG (1/2): CS1, Q1, Q2

CS1 / Q1. Effect on score and conditional density; conditional and unconditional densities under CFG; difference from OG.

The reviewer is right. The correct interpretation should be that FF moves the sample away from its posterior mean x^1=E[x1∣xt]\hat x_1 = \mathbb E[x_1 \mid x_t]. We have corrected L852 and App. C.2 accordingly.

Proof: for xt=(1−t)x0+tx1x_t = (1-t)x_0 + t x_1, the model's velocity is v~t=E[x1−x0∣xt]=x^1−x^0\tilde v_t = \mathbb E[x_1 - x_0 \mid x_t] = \hat x_1 - \hat x_0, with x^i=E[xi∣xt]\hat x_i = \mathbb E[x_i \mid x_t]. Taking E[ ⋅∣xt]\mathbb E[\,\cdot \mid x_t] of the path identity gives xt=(1−t)x^0+tx^1x_t = (1-t)\hat x_0 + t\hat x_1, hence v~t=(x^1−xt)/(1−t)\tilde v_t = (\hat x_1 - x_t)/(1-t): an Euler step of size αΔt\alpha\Delta t moves along the line through xtx_t and x^1\hat x_1, toward xtx_t for α>0\alpha > 0 and away for α<0\alpha < 0.

On CFG, we also thank the reviewer for the insightful analysis, which we extend below. The FF velocity v~tFF=−v~t\tilde v^{FF}_t = -\tilde v_t has drift −atx-a_t x, which, as the reviewer notes, prevents a direct reading as a reweighting. However, this drift can be absorbed into a Gaussian factor: −atx=atx+bt∇log⁡Gt-a_t x = a_t x + b_t \nabla \log G_t with Gt(x)∝exp⁡ ⁣(−(at/bt)∥x∥2)G_t(x) \propto \exp\!\big(-(a_t/b_t)\lVert x \rVert^2\big), which is a valid Gaussian since at=1/ta_t = 1/t and bt=(1−t)/tb_t = (1-t)/t are positive for our linear path for t∈(0,1)t \in (0, 1). This yields

v~tFF=atx+bt∇log⁡qtFF,qtFF∝Gt(x) pt(x)w−1 pt(x∣y)−w=Gt(x) (pt(x)1−w pt(x∣y)w)−1∝Gt(x)p~t(x∣y).\tilde v^{FF}_t = a_t x + b_t \nabla \log q^{FF}_t, \qquad q^{FF}_t \propto G_t(x)\, p_t(x)^{w-1}\, p_t(x \mid y)^{-w} = G_t(x)\, \big(p_t(x)^{1-w}\, p_t(x \mid y)^{w}\big)^{-1} \propto \frac{G_t(x)}{\tilde p_t(x \mid y)}.

So under this view, FF involves the same reweighting of the conditional and unconditional densities as CFG, but its associated (unnormalized) density is the reciprocal of this reweighted density multiplied by a Gaussian factor. Equivalently, it negates the guided log-density: log⁡qtFF=−log⁡p~t(x∣y)+log⁡Gt+const\log q^{FF}_t = -\log \tilde p_t(x \mid y) + \log G_t + \text{const}.

We can also write qtFFq^{FF}_t in terms of the OG density: qtFF∝Gt p~tOG(x∣y)/pt(x)2q^{FF}_t \propto G_t\, \tilde p^{OG}_t(x \mid y) / p_t(x)^2. Both down-weight the conditional, but FF additionally down-weights high-likelihood regions of the unconditional distribution. Intuitively, this additional term favors lower-likelihood regions, which aligns with the higher diversity we observe for FF than OG.

Q2. Is the proposed approach marginal preserving, or does it preserve the target distribution? Our method neither preserves marginal probability path or target distribution. While α(t)≠1\alpha(t)\neq1 the sampler leaves the model's probability path, so the marginals differ from ptp_t from the first inverted step; as α(t)\alpha(t) returns to 1 it follows the ODE from a displaced state, so the endpoints differ from the model's target. Like CFG, FF does not aim to sample from the target distribution exactly, but offers a way to trade a small amount of fidelity for diversity.

AuthorsOpenReview ↗

Response to Reviewer ZLZG (2/2): CS2

CS2. Do the authors have any regular models that are doing true CFG?

Yes, we have TRELLIS, a non-distilled model that uses true CFG. Our LSUN Bedroom model in App. H is a standard unconditional flow-matching model, which needs no CFG, but is also a non-distilled model.

To further test generalization to a non-distilled text-to-image model with true CFG, we test on Stable Diffusion 3.5 Medium (20 steps, 512 px, 10 seeds)

Setup. All hyperparameters are tuned on 50 T2I-CompBench validation prompts and applied unchanged to the 553 GenEval prompts. For the reduced-CFG baseline, we sweep the guidance scale over {4.5, 3.5, 2.5, 2.0, 1.5, 1.0} and pick the one with the highest diversity (DINO,DreamSim) that is statistically non-inferior to the default 4.5, i.e., whose CLIP Score and PickScore are at most 1% below those of CFG 4.5 (one-sided paired test over prompts, p < 0.05). That scale is 3.5. Our settings (σ = 0.05, η = 1.25), also tuned on the validation set, pass the same test.

Table R1. SD3.5 Medium; mean ± SE over prompts. CLIP, PickScore, HPSv2: quality; DINO, DreamSim: diversity. ▼ / ▲ = Base statistically significantly lower / higher than Ours (two-sided paired t-test over prompts, p < 0.05).

CLIP ↑PickScore ↑HPSv2 ↑DINO ↑DreamSim ↑
Validation (50)
Base, CFG 4.527.87 ± .3822.93 ± .1329.92 ± .170.668 ± .008 ▼0.380 ± .014 ▼
Base, CFG 3.527.80 ± .3822.84 ± .13 ▼29.64 ± .17 ▼0.684 ± .008 ▼0.410 ± .014
Ours (σ = 0.05, η = 1.25)27.91 ± .4022.91 ± .1329.90 ± .180.708 ± .0060.416 ± .013
GenEval (553)
Base, CFG 4.528.63 ± .14 ▲22.59 ± .04 ▲28.95 ± .040.627 ± .003 ▼0.374 ± .004 ▼
Base, CFG 3.528.36 ± .1422.51 ± .0428.68 ± .04 ▼0.641 ± .003 ▼0.398 ± .005 ▼
Ours (σ = 0.05, η = 1.25)28.36 ± .1422.52 ± .0428.95 ± .040.664 ± .0030.407 ± .004

Result on GenEval. CFG 4.5 scores higher than our method on CLIP Score and PickScore, but lower on diversity (DINO and DreamSim). Against CFG 3.5, which meets the same quality bar as ours (non-inferior to CFG 4.5 on validation), ours is more diverse and has higher HPSv2. All stated differences are statistically significant (p < 0.05).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.