acceptodds
Under review as a conference paper at ICLR 2027

JIVE: Jacobian-Informed Volume Expansion for Diverse Generative Sampling

18 views

Abstract

Generative models often suffer from mode collapse and limited sample diversity. While prior works attempt to mitigate this by jointly generating a batch of samples and repelling the trajectories apart, these heuristics do not explicitly maximize the diversity of the resulting endpoints. We introduce JIVE, a training-free framework that enhances generative diversity by injecting velocity perturbations aligned with the leading right singular subspace of the generator's endpoint Jacobian. By leveraging this local geometric structure, JIVE provably maximizes endpoint diversity while preserving sample quality. To maintain practical efficiency, we compute these perturbation directions via matrix-free iterations rooted in classical numerical linear algebra, requiring only a small computational overhead. Across different benchmarks, JIVE boosts both pixel and feature-level diversity in few-step and one-step generation.

Primary area:
generative models
Keywords:
Diffusion Model, Diversity, Flow matching

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VB4bOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes JIVE, a training-free method for improving sample diversity in flow-based generative models. The main idea is to perturb the sampling trajectory along directions that induce large changes in the predicted endpoint, identified using the leading right singular subspace of the endpoint Jacobian. The authors also provide a theoretical analysis connecting these perturbations to increased pairwise distances between generated samples, and introduce efficient approximations to avoid explicitly computing the Jacobian. Experiments on FLUX and SD3.5 show improvements in sample diversity, including few-step and one-step generation, with additional results on inversion-based generation.

Overall, I have a positive impression of this paper. The presentation is clear, the proposed approach seems reasonable, and the experimental results are encouraging. I therefore lean towards acceptance. This topic is somewhat outside my main research area, and I am not sufficiently familiar with the relevant literature to make a confident assessment of the paper's novelty relative to prior work. My recommendation should therefore be considered with relatively low confidence.

Critical strengths
  1. The paper is well written and easy to follow. The motivation for using the endpoint Jacobian to guide perturbations is clearly explained, and the toy example helps illustrate why this approach can be useful compared with existing diversity guidance methods.

  2. The experimental evaluation is fairly comprehensive. The authors consider different generative models and sampling settings, and report both diversity and quality metrics. I also appreciate the analysis of computational overhead and the inclusion of low-cost approximations, which make the proposed method more practical.

Critical weaknesses

The proposed method introduces additional computation during sampling, particularly when estimating the leading singular directions. Although the authors provide cheaper approximations and report their computational costs, this overhead may still be relevant for practical applications where inference speed is important.

I do not have other major concerns based on my reading.

Questions for authors

I have a few questions for the authors:

  1. Perturbation timing. The experiments mainly apply JIVE at the initial sampling step (t=0t=0). Is there a particular reason for this choice? Would applying the perturbation at intermediate steps lead to different results?

  2. Perturbation magnitude. The paper uses a heuristic to determine the perturbation norm. How sensitive is the method to this choice? Does the same heuristic work well across different generative models?

  3. JVP vs. JᵀJ. The JᵀJ variant generally achieves better diversity-quality tradeoffs, while JVP is computationally cheaper. Could the authors provide more guidance on when each variant would be preferable in practice?

5 comments
AuthorsOpenReview ↗

[1/4] Rebuttal by Authors

We thank reviewer VB4b for the helpful feedback. We are glad that the reviewer found the paper easy to follow and the explanation of using endpoints Jacobians clear. We address the reviewer's questions below.

CW1: Computational overhead of JIVE.

Thanks reviewer for highting this. We acknowledge that JIVE incurs a computational overhead. However, unlike methods that require interventions throughout the sampling process, JIVE applies a single perturbation at the beginning of sampling, with no further interventions required during subsequent denoising steps. Hence, JIVE only needs to compute a perturbation once.

To evaluate the practical computational cost, we measure the wall-clock cost of generating images with JIVE (JVP) with one direction and one iteration and OSCAR on FLUX.1-schnell at 512×512 with 4 denoising steps. Both methods generate a batch of 16 images, and the per-image time is the batch time divided by 16, averaged over 5 runs:

MethodPer imagePer batch
JIVE (JVP)2.171 ± 0.007 s34.74 s
OSCAR2.615 ± 0.004 s41.85 s

Thus, while JIVE introduces overhead compared with unmodified sampling, its computational cost remains competitive with existing diversity-enhancement methods.

AuthorsOpenReview ↗

[2/4] Rebuttal by Authors

Q1. Is there a particular reason for applying mainly at the initial sampling step rather than a later one? Would applying the perturbation at intermediate steps lead to different results?

Thank you for raising this question. We inject at the first step since we reasoned that injecting sooner would cause a greater overall change to the trajectory. This is supported by the table below where we compare injecting at the first step and at intermediate steps. Injecting at the first step raised diversity (Vendi-DINO) the most, but also had greater losses in terms of quality (HPSv2 and Pickscore.) Overall, injecting sooner seems to move JIVE further away from the baseline on the diversity-fidelity trade-off curve.

We test on a multi-step model (Flux.1 dev) below.

FLUX.1-dev (28 step, 512×512512 \times 512) GenEval at 4 images per prompt. Values are mean ±\pm se

JIVE (JTJJ^TJ, norm 8), injected at timestep K

Injection stepVendi DINOVendi pixelCLIP scoreHPSv2PickScoreGenEval
Baseline2.054 ± 0.0241.516 ± 0.01332.49 ± 0.1430.41 ± 0.1223.40 ± 0.040.651
K = 02.220 ± 0.0251.600 ± 0.01332.49 ± 0.1429.54 ± 0.1123.19 ± 0.040.639
K = 32.138 ± 0.0251.572 ± 0.01432.47 ± 0.1430.16 ± 0.1123.29 ± 0.040.646
K = 82.088 ± 0.0241.523 ± 0.01332.44 ± 0.1430.34 ± 0.1223.34 ± 0.040.643
K = 142.074 ± 0.0241.517 ± 0.01332.45 ± 0.1430.35 ± 0.1223.35 ± 0.040.652
AuthorsOpenReview ↗

[3/4] Rebuttal by Authors

Q2: How sensitive is the method to the choice of perturbation norm? Does the same heuristic work well across different generative models?

We thank the reviewer for this question. The heuristic in Sec. 2.2 gives a model-specific upper bound on the norm, which keeps the perturbed latent near the typical norm of a Gaussian sample. Thus, our proposed heuristic provides a practical upper bound on the perturbation norm, rather than a recommended or optimal value.

The choice of heuristic norm depends only on the latent dimension (about 26 for FLUX.1-schnell and 38 for SD3.5-Turbo in Fig 5). Within this range, the norm acts as a control on the diversity–quality tradeoff. To assess sensitivity, we swept the norm for both variants on both models in Fig. 5: JTJJ^TJ over {4, 6, 8, 12, 19} on FLUX.1-schnell and {8, 12, 16, 24} on SD3.5-Turbo, and JVP over {8,12,19,26,35,45} and {19,26,35,45}, spanning roughly 0.15–1.7× the respective bounds. Across these ranges, diversity and quality change smoothly and monotonically with the smaller norm than herustic norm. However, as norm approaches the herustic norm, quality would become more sensitive and thus degrade more quickly. The same behavior holds on both models when norms are expressed relative to each model’s bound, which indicates that the heuristic transfers. As noted in our limitations, the heuristic provides a practical upper bound rather than an optimal perturbation norm. In practice, we recommend choosing a smaller norm within this bound, as Fig. 5 shows that smaller perturbations generally achieve more favorable diversity–quality trade-offs.

Reviewer_Q7NnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes JIVE, a training-free framework for improving diversity in flow-matching models. The main idea is to perturb sampling latents along directions that have a large effect on the predicted endpoint. The authors use the leading right singular subspace of the endpoint Jacobian and show that this choice maximizes the expected increase in pairwise squared distance under a local linear approximation. They also develop cheaper approximations based on Jacobian-vector products. Experiments on FLUX.1-schnell and SD3.5-Large-Turbo show improvements in feature and pixel-level diversity in few-step and one-step generation.

Critical strengths
  1. The idea of selecting perturbations based on their effect on the final endpoint is interesting and well motivated by the toy example. The method also works independently on each sample, without requiring an external feature or reward model.
  2. The theoretical analysis connects local endpoint perturbations to the expected pairwise distance between generated samples.
  3. The experiments cover both one-step and few-step generation across two model families. The low-cost variants achieve promising diversity-quality trade-offs, especially on SD3.5-Large-Turbo. The experiments with CADS and RF-Inversion also show that the method can be combined with other sampling approaches.
Critical weaknesses
  1. The theoretical guarantee is somehow limited. Theorem 1 only considers the squared Euclidean distance between linearized outputs of the one-step endpoint predictor instead of the actual multi-step endpoints or their semantic diversity. Moreover, the cheaper JVP approximation produces directions in the left singular basis, while the proposed objective requires right singular directions. It remains unclear how accurate these approximations are at the perturbation norms used in the experiments.
  2. The diversity improvements come with a noticeable loss in image quality. In Table 2, JIVE with the Jacobian-transpose product improves F-Vendi from 3.23 to 4.40, but reduces HPSv2 from 29.66 to 26.87 and CLIP-IQA from 0.671 to 0.612. The authors provide diversity-quality trade-off curves for the low-cost variants, but not for the main PartiPrompts experiments in Table 2. Without these curves, it is difficult to tell whether JIVE offers a better trade-off at comparable image quality.
  3. The computational cost of the main experimental configuration is not reported. Table 1 only measures a single direction-estimation iteration, whereas the results in Table 2 use a four-dimensional subspace with ten iterations. Even a single product takes 4.24 times as long as a forward pass and increases peak memory from 22.3 to 35.4 GB. It would be useful to know whether the diversity gains justify this cost.
Questions for authors
  1. How accurately does the one-step endpoint predictor approximate the actual multi-step sampler? Can the authors measure the linearization error and actual endpoint displacement at the perturbation norms used in the experiments?
  2. Can the authors provide diversity-quality trade-off curves for the main PartiPrompts experiments?
  3. What are the total sampling time and peak memory usage for the four-direction, ten-iteration configuration?
  4. Can the authors include a comparison with STRIDE, which is already discussed in the related work, or explain why a direct comparison is not feasible?
  5. The main experiments apply the perturbation at (t=0). Have the authors considered applying JIVE at different sampling steps? It would be interesting to see whether the perturbation timing affects the diversity-quality trade-off.
7 comments
AuthorsOpenReview ↗

[1/6] Rebuttal by Authors

We thank reviewer Q7Nn for their constructive feedback, and we're happy that they thought the motivation of selecting perturbations based on endpoint change was interesting. We answer the reviewer's questions below.

CW1- Part (A): Theorem 1 is limited and uses linearized endpoints, along with a one-step predictor instead of the full trajectory

Thank you for the good question. As stated in Section 2.3, the scope of Theorem 1 is about linearized endpoint of the one-step predictor. Although Theorem 1 uses a simplified setting, our experimental results in Table 2 and 3 show that the predicted increase in linearized endpoint diversity carries over to the full trajectory and in feature space.

Additionally, the setting of Theorem 1 allows JIVE to be computationally efficient as we would not need the extra function evaluations that the full trajectory requires. This approximation requires only a single velocity evaluation, whereas computing the exact endpoint would require multiple denoising steps and substantially greater computational and memory overhead. More importantly, the purpose of this approximation is not to precisely reconstruct the full trajectory, but to efficiently identify local perturbation directions that are likely to produce large changes in the final generated output.

CW1-Part (B): The JVP estimate finds directions in the left singular basis instead of the right singular basis.

We measured the alignment of the JVP direction and the leading right singular basis directly in Table 5 of Appendix A.8. We found that, on Flux.1 Schnell, the JVP estimate produced a direction with significant alignment with the leading right singular basis. We measure alignment with the amount of energy the JVP estimate had in the right singular basis. One JVP direction had approximately 7% of its energy in the subspace of top 4 right singular vectors. In contrast, random directions analytically only had 0.0061% of their energy in the same subspace. So, one Jacobian vector product (JVP) with a random vector moved a significant amount of that vector's energy into the top right singular subspace.

AuthorsOpenReview ↗

[2/6] Rebuttal by Authors

Question 1: For each result, we used 32 GenEval prompts at 4 images each. Results are reported as the median of each measurement.

Part (A): How accurately does the one-step predictor approximate the multi-step sampler?

We measured the Cosine similarity between the one-step predictor and the multi-step sampler's endpoints. On both 4-step Flux.1 Schnell and 4-step Turbo, the median cosine similarity was 0.94, meaning the direction found by the one-step predictor was faithful to the full trajectory's endpoint.

Part (B): Can you measure the linearization error?

We evaluate how accurately the local linear approximation predicts the changes induced by JIVE perturbations. Specifically, we compare the actual output displacement after applying a JIVE perturbation with the displacement predicted by the local linear approximation.

We report two metrics: (1) relative linearization error, which measures the difference between the actual and predicted displacements, and (2) cosine similarity, which measures how well their directions align. Lower linearization error and higher cosine similarity indicate a more accurate local linear approximation.

ModelLinearization errorCosine
FLUX.1-schnell (‖δ‖ = 8)1.040.77
SD3.5-Large-Turbo (‖δ‖ = 16)0.640.89

The actual displacement direction aligns with the linear displacement direction (cosine 0.77-0.89). The linearization error is high, and was due to the linear model overpredicting the actual displacement's magnitude.

Part (C): What was the actual endpoint displacement of the trajectory at the norms used in the experiments?

We directly measure the endpoint displacement ∥ΔS∥\|ΔS\| of the full trajectory after injecting with JIVE. We tested on our one-iteration J⊤JJ^\top J variant.

The table shows that the JIVE direction has significantly greater endpoint displacement (5.4-9.2 times larger) than a random direction, supporting JIVE's motivation of selecting directions with large endpoint change.

FLUX.1-schnell, 1 step (‖δ‖ = 8)SD3.5-Turbo, 4 steps (|δ| = 16)
‖ΔS‖, JIVE direction339535
‖ΔS‖, random direction6258
JIVE ÷ random5.4×9.2×
AuthorsOpenReview ↗

[3/6] Rebuttal by Authors

CW2/Q2 Can you provide diversity-fidelity trade-off curve on Partiprompts? :

The large scale of the Partiprompts experiments makes it difficult for us to provide a trade-off curve in that setting within the limited time as

  • PartiPrompts contains 1,632 prompts, compared with 553 prompts in GenEval.
  • our evaluation generates 16 images per prompt, compared with four for GenEval.
  • Sweeping different hyper-parameters in this large experiment.

However, Figure 5 also provides a trade-off curve on a smaller dataset on the low-cost setting, which gives us a reference for the operating point in Table 2.

As shown in Fig. 5, JIVE achieves favorable diversity–quality trade-offs compared with existing baselines even under this inexpensive approximation, outperforming them across several evaluation metrics. Since the setting in PartiPrompt experiment provides a more accurate approximation of the leading singular subspace, we expect it to achieve similarly favorable trade-offs.

We also wish to emphasize that the diversity improvement is substantial relative to the observed quality degradation. As the reviewer correctly points out, JIVE increases F-Vendi from 3.23 to 4.40, while reducing HPSv2 from 29.66 to 26.87 and CLIP-IQA from 0.671 to 0.612. In relative terms, this corresponds to a 36.22% improvement in F-Vendi, accompanied by a 9.41% decrease in HPSv2 and an 8.79% decrease in CLIP-IQA. Although these metrics measure different aspects of generation and their percentage changes are not directly comparable, the results highlight that JIVE achieves a substantial diversity gain with comparatively smaller relative changes in the reported quality metrics. Furthermore, the perturbation magnitude can be adjusted to control the balance between diversity and fidelity, allowing users to prioritize their desired trade-off.

Reviewer_VEppOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors tackle the problem of improving the sample diversity of diffusion or flow matching models at inference time. The authors first identify a recent category of methods which they term as “batch-repulsion” techniques; these methods encourage repulsion in the trajectories of a batch of samples, which yields diversity. The authors argue that these methods only implicitly encourage diversity. Through their proposed method, JIVE, the authors claim to maximize diversity in an explicit manner by sampling perturbations from the Jacobian of the estimated endpoint. JIVE is experimentally evaluated for text-to-image generation using FLUX schnell and SD3.5 Turbo, and it shows notable improvements over baselines.

Critical strengths

Critical Strengths

CS1. The paper proposes a novel Jacobian-guided approach to promote diversity, in a training-free manner. In general, training-free approaches are useful to the community, because they can be utilized to improve already trained monolithic models.

CS2. The authors come up with useful approximations to avoid explicit Jacobian calculation, keeping the method computationally bounded. Computational efficiency improves the utility of the approach.

Critical weaknesses

Critical Weakness 1. Fig. 3 suffers from selection bias, and the motivation in Sec. 2.2 is somewhat misguided, conflating two kinds of diversity.

The diversity problem can arise from (i) the nature of IID sampling, which is unaware of previously explored modes, or (ii) an overly concentrated or sharp generative distribution. Non-IID repulsion methods explicitly address the former, while IID methods such as CADS pursue the latter.

In Fig. 3 the authors show that when sampling a group of K=8 points, JIVE discovers a large (the max K) number of modes, while the base flow or the repulsion baseline only discovers a few modes. This figure motivates the entire section 2.2., and I believe it is highly misleading. Similar toy experiments can be found in Particle Guidance (Corso et al. 2024) [Appendix C, Fig. 7], or OSCAR (Wu et al. 2026) [Fig. 2], where they show high mode coverage. For a 2D toy distribution, I don’t see why it is difficult for repulsion-based methods to simply push the points apart.

As such, fig. 3 likely suffers from selection bias, and should be backed up by statistical results.

The authors can do as follows:

  • Given a uniform Gaussian mixture with N=16 equiprobable modes (like Fig. 3), sample (i) K=8, (ii) K=16 points, over ~1000 random trials.
  • Compare (i) avg. modes discovered, (ii) the frequency of discovering the maximal K modes (like JIVE does in Fig. 3c).

Since all the N modes are equally likely, a perfect/analytic generative model theoretically already has maximal diversity. IID samples from this analytic model will converge to equal density for every mode.

But if we were to repeatedly sample groups of size K (e.g. K=8, or K=16) in an IID manner, the expected number of discovered modes is actually N * (1 - (1 - 1/N)^K) . (If the authors are interested in the derivation, the special case of K=N is derived in Proposition 2 of Particle Guidance (Corso et al. 2024)).

That is, if the authors were to carry out the toy experiment suggested above, we can already predict the outcome: 6.45/8 modes for (K=8) and 10.30/16 modes for (K=16). Given that the underlying model is analytic, and both the base and JIVE are IID, I am fairly confident that both the base analytic flow and JIVE empirically achieve highly similar number of modes, close to the theoretical outcome.

It should be fairly easy to come up with a random seed in Fig. 3, where the base flow actually finds more modes than JIVE out of 8 or 16. This is because JIVE is IID; each point actually has no information about what has been already explored. A pair of consecutive points can converge to the same mode with probability 1 / N.

Repulsion methods aim to achieve sample-efficient diversity. If we are sampling some K points from N ≥ K modes, these methods aim to maximize the expected modes discovered in K. JIVE and other IID methods (like CADS) improve diversity from a more distributional perspective. As such, in section 2.2, given the uniform mixture and the analytic flow, JIVE actually has little to no advantage over the base flow model in terms of mode discovery, and the motivation/figure is misleading.


Critical Weakness 2. CADS inconsistencies, single step CADS

The authors first compare against CADS (Sadat et al. 2024) and later combine JIVES with CADS. Could they clearly explain exactly how they extended CADS with their settings? Sadat et al. define parameters τ1,τ2∈[0,1]\tau_1, \tau_2 \in [0, 1]; they are intermediate timestep thresholds. But in Tab. 2, L294, the authors mention τ2=1.2\tau_2 = 1.2 (which exceeds 1) without any mentions of a τ1\tau_1 throughout the paper.

Further, I am not sure how it is possible at all to apply CADS for 1 step generation, as done by the authors in Table 3. As the authors mention in L283, L465, CADS injects noise into the text condition and gradually reduces/anneals the noise away over time. When there is just 1 sampling step, then CADS is either applied, or not applied at all.

If CADS is applied at all, then it means the authors are generating the image from a noisy text conditioning vector, which is a very strange thing to do (it’s like the model never sees the actual text condition given by the user, but some random perturbation of it). I see no point of doing this experiment, and question whether the generated images remain prompt-faithful. I believe I didn’t see any qualitative examples of this setting either.

Questions for authors

Q1. Arbitrary timesteps. In Sec. 2.2, the authors assume xt∼N(0,Id)x_t \sim \mathcal{N}(0, I_d). This assumption only holds at x0x_0 for t=0t=0 (assuming standard flow matching convention); it does not apply to a general tt. This, of course, matches with L169 where the authors say they only do 1 perturbation in practice.

Given this, my initial impression was that the method involves taking a norm-bounded step on the surface of the Gaussian hypersphere, relying on a generator that is good at 1 or few step generation capability. As such, the main results are primarily on step-distilled models like Flux Schnell, and SD3.5 Turbo.

But in Fig. 3 experiment, the perturbation seems to be applied at step 100/800. Why is this the case? Also in RF-inversion experiment, what exact step is the perturbation inserted?

In general, the performance of JIVE for non-step-distilled model remains unclear, there are no comparisons with other methods for regular models. Further, it would be good to know how the performance of JIVE is related to the quality of the endpoint estimate.

Q2. Maximizing diversity, preserving quality. The authors claim throughout the paper that the proposed method maximizes diversity. However, the diversity gain through JIVE always lags behind the gain from CADS, in terms of both quality and diversity in Table 2. Do the authors have an explanation for this?

Q3. Discussion on results. One unusual thing about the results, is that the repulsion baseline (OSCAR) appears to be nearly identical to the unmodified sampler. This strongly contradicts the results published by Wu et al. 2026, where they show their method achieves better performance than CADS (which outperforms JIVE in this paper). Why is this the case? It is critically important to investigate and clarify this inconsistency. Further, given the ambiguity of this baseline, the authors should optionally add an additional repulsion baseline (such as Particle Guidance) if they want to establish their claim of being better than repulsion-based methods.

Q4. How does JIVE "provably" preserve quality? Is there any other quality regularization other than the norm constraint? Further, the quality can be affected even if the norm is maintained, as we can observe from Table 2; JIVE seems to have a consistent drop in quality compared to baselines. On a related note, is JIVE marginal-preserving?

I am willing to raise my score if the authors can convincingly address the weaknesses and questions.


Baseline suggestion:

As described in Q1, the practical implementation of JIVE for Schnell and SD3.5 Turbo amounts to an optimization on the initial Gaussian latent. As such, I believe it is more appropriate to compare JIVE to other related methods that involve optimizing a diversity objective at the noise space, such as, DivIn (Li et al. 2026) and Divgen (Harrington et al. 2026), the latter of which is mentioned in the related work.

References: Li et al. 2026, "Initialization is Half the Battle: Diverse Initialization for Mitigating Memorization in Diffusion Models", ICML 2026 spotlight.

Other referred papers cited in the main text by authors.

11 comments
AuthorsOpenReview ↗

[1/7] Rebuttal by Authors

We thank reviewer VEpp for their efforts and time in evaluating the paper. We are glad that the reviewer finds the method useful due to being training-free and computationally efficient. We address the reviewer’s comments below.

CW1: Fig. 3 suffers from selection bias, the motivation in Sec. 2.2 is somewhat misguided, conflating two kinds of diversity

A: Thank you for raising this question. This gives us the opportunity to clarify these different diversity approaches. Repulsion methods achieve diversity by enforcing sampling trajectories to avoid each other, so they do not hit the same target location. For target modes with low probability, repulsion methods cannot guarantee to hit them even with more samples. Distributional methods (including JIVE) modify the generation distribution implicitly. JIVE essentially “flattens” the target distribution to ensures even modes with low probability can have a decent chance to be covered. This is indeed justified theoretically. Our Theorem 1 shows that JIVE increases the total variance (Tr(Σ\Sigma)) of the generation distribution. This has the effect of increasing the expected squared distance between pairs of points, thereby increasing diversity by flattening the distribution.

The toy example in Fig. 3 in Section 2.1 demonstrated JIVE’s diversity, but did not fully demonstrate the advantage of JIVE over repulsion methods (OSCAR). We agree the performance of OSCAR depends on the selected initial seeds, and thank the reviewer for pointing this out.

We hereby provide an additional example, i.e., an asymmetric version of the same setting. See Fig. 7 in the updated manuscript’s appendix. We assign one mode (bottom center) with higher weight than the others. The distribution is over-concentrated at the special mode, whereas the support is still the 16 modes. We note that in this case, naive flow matching will shoot almost all samples to the special mode. Repulsion method (OSCAR) mitigates the issue for a bit. But samples are still mostly concentrated at the special mode. JIVE flattened the distribution so other modes get better chance to be covered. In Fig 8, we show quantitatively that JIVE has much higher coverage compared with baselines.

Experimental setup. We place 16 Gaussian modes (σ = 0.08) on a circle of radius 2. We set one mode to have weight 200 and each of the others to have weight 1, so about 93% of the probability mass sits on a single mode. The distribution is as follows:

pmodel(y)=200N(y;c12,0.082I)+∑m≠12N(y;cm,0.082I)215.p_{\mathrm{model}}(y)=\frac{200\mathcal N(y;c_{12},0.08^2I)+\sum_{m\neq12}\mathcal N(y;c_m,0.08^2I)}{215}.

Initial particles are sampled IID from N(0,0.252I)\mathcal N(0,0.25^2I). We run 1,000 paired trials in which all methods share the same initial noise, and every method uses 800 Euler integration steps. JIVE and OSCAR are given the same cumulative perturbation budget per particle of 0.5.

As shown in the table, JIVE not only improves mode coverage (6.800 vs 2.103 average occupied modes) but also increases pairwise distance (6.932 vs 1.190 pairwise squared L2), consistent with the theoretical analysis in Theorem 1. These results support our central motivation that by exploiting the generator's local Jacobian structure, JIVE can alleviate low mode coverage without relying on explicit in-batch repulsion. In particular, when flow-matching trajectories tend to concentrate in a limited number of modes, JIVE enables samples to reach a broader range of modes. We have added the additional example to Appendix A.1.

KMethodOccupied modesSquared L2L_2Heavy-mode assignment
8FM1.544±0.0221.544\pm0.0221.244±0.0571.244\pm0.05793.08%±0.28%93.08\%\pm0.28\%
8In-batch repulsion3.258±0.0293.258\pm0.0293.600±0.0433.600\pm0.04366.15%±0.45%66.15\%\pm0.45\%
8JIVE4.208±0.0404.208\pm0.0406.764±0.0756.764\pm0.07555.36%±0.56%55.36\%\pm0.56\%
16FM2.103±0.0302.103\pm0.0301.190±0.0371.190\pm0.03792.91%±0.20%92.91\%\pm0.20\%
16In-batch repulsion4.902±0.0364.902\pm0.0363.926±0.0283.926\pm0.02861.16%±0.43%61.16\%\pm0.43\%
16JIVE6.800±0.0496.800\pm0.0496.932±0.0506.932\pm0.05054.31%±0.39%54.31\%\pm0.39\%
AuthorsOpenReview ↗

[2/7] Rebuttal by Authors

CW2-Part (A): Why is CADS’ τ2\tau_2 parameter above 1?

In short, CADS’ default τ2\tau_2 range of [0,1][0,1] caused the generated images to not follow the prompt. We raised τ2\tau_2 to >1 to fix this issue, so that CADS would function well in a distilled setting. Below, we further clarify two points: (1) why choosing τ2>1\tau_2>1 is mathematically valid, and (2) why we use τ2=1.2\tau_2=1.2 instead of the default CADS setting.

(1) How to extend CADS such that τ2>1\tau_2>1

Although the original formulation assumes τ2∈[0,1]\tau_2 \in [0,1], setting τ2>1\tau_2 > 1 is still well-defined. Regarding τ1\tau_1, we used CADS’ default value in stable diffusion of 0.6.

The original schedule is

\begin{cases} 1, & t\le\tau_1, \\\\ \dfrac{\tau_2-t}{\tau_2-\tau_1}, & \tau_1<t<\tau_2, \\\\ 0, & t\ge\tau_2. \end{cases}$$ By setting $\tau_2=1.2$, the schedule becomes $$\gamma(t)= \begin{cases} 1, & t\le\tau_1, \\\\ \dfrac{1.2-t}{1.2-\tau_1}, & \tau_1<t\le1. \end{cases}$$ **(2) Why setting $\tau_2=1.2$:** **The choice to raise $\tau_2>1$ is because at CADS’ default range of $\tau_2 \in [0,1]$, the generated images completely deviate from the prompt**. This is reflected by the fact that the **CLIP score drops significantly at CADS’ parameter value of $\tau_2=0.9$**, which was the value used by CADS in their Stable Diffusion experiments. This is because a parameter value of $\tau_2=0.9$ makes the first flow matching step completely unconditional (i.e. the conditioning vector is a pure noise vector instead of a text embedding). In a 4- or 1-step model, each step takes up a significant part of the inference trajectory. A single unconditional step makes it so that the intermediate trajectory deviates significantly from the conditional distribution, hence the outputs are not prompt-aligned. By raising $\tau_2$, the first step’s conditioning vector is a mixture of text signal and noise, instead of purely unconditional. We have included a table of CADS at $\tau_2=0.9$. **CADS on 4-step Flux.1 Schnell at $\tau_2=0.9$, GenEval at 4 images per prompt** | Method | CLIP score | Vendi DINO | Vendi pixel | PickScore | HPS | GenEval | | --- | --- | --- | --- | --- | --- | --- | | Baseline | 32.88 | 2.039 | 1.707 | 23.26 | 30.56 | 0.695 | | CADS(τ₂ = 0.9) | 18.29 | 3.850 | 1.892 | 17.81 | 12.18 | 0.014 | ---- **CW2-Part (B): Single Step CADS** Raising $\tau_2$ allowed us to use CADS in a 1-step setting. As shown in Table 3, the CLIP score of CADS was still comparable to unmodified flow matching, suggesting that the images were still prompt-faithful. We have included a qualitative comparison in the Fig 9.
AuthorsOpenReview ↗

[3/7] Rebuttal by Authors

Q1-part (A): Timesteps of JIVE’s perturbation

We would like to clarify that JIVE works both when injecting its perturbation at a later timestep, and when applied to a multi-step model. This is shown in the experiment below where we apply the one-iteration JtJ variant of JIVE to a a multi-step model (Flux.1 dev, 28 inference steps). JIVE is able to increase the Vendi DINO score from 2.054 ± 0.024 to 2.220 ± 0.025. Additionally, injecting JIVE at a later timestep also increases diversity, as shown by the table for values of K>1. We apply JIVE �0��0� with one subspace iteration and a perturbation norm of 8, injecting the perturbation at timestep K.

FLUX.1-dev (28 step, 512×512512\times512) GenEval at 4 images per prompt. Values are mean ±\pm se.

Injection stepVendi DINOVendi pixelCLIP scoreHPSv2PickScoreGenEval
Baseline2.054 ± 0.0241.516 ± 0.01332.49 ± 0.1430.41 ± 0.1223.40 ± 0.040.651
K = 02.220 ± 0.0251.600 ± 0.01332.49 ± 0.1429.54 ± 0.1123.19 ± 0.040.639
K = 32.138 ± 0.0251.572 ± 0.01432.47 ± 0.1430.16 ± 0.1123.29 ± 0.040.646
K = 82.088 ± 0.0241.523 ± 0.01332.44 ± 0.1430.34 ± 0.1223.34 ± 0.040.643
K = 142.074 ± 0.0241.517 ± 0.01332.45 ± 0.1430.35 ± 0.1223.35 ± 0.040.652

Q1-part (B): Fig. 3 Intervention Time.

In Fig. 3, we intervened at time t=100 instead of t=0 due to properties of the closed-form vector field we were using. At the initial timestep, the optimal closed form solution of a flow matching model points towards the mean of the target distribution. Hence, the endpoint of every point is the mean. As such, the Jacobian matrix is the 0 matrix, hence the singular vectors aren’t meaningful. We note in the case of an actual learned flow matching model, it is unrealistic for this degenerate case to occur.

Q1-part (C): RF Inversion. In RF-inversion, the intervention with JIVE is still applied at the first denoising step t=0.

Q1-part (D): Comparison on multi-step models. As the reviewer requested, we included a comparison of JIVE to other baselines on Flux.1 dev, a multi-step model. We included the table in Question 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Official Comment by Authors

We thank all three reviewers for their thoughtful feedback and the Area Chair for their time and service. Across the reviews, we are encouraged by the positive reception of JIVE's practical capability to improve the diversity of generative models in a training-free manner. We are happy that the reviewers found the idea of selecting perturbations based on endpoint change intuitive, and the experimental results convincing on the applicability of JIVE to different settings.

In response to the questions raised during the discussion, we have incorporated the following additional experiments, which will be reflected in the revised manuscript:

  • We have included additional experiments of JIVE in a multi-step setting and using JIVE at different intermediate sampling steps
  • We have added comparisons to additional baselines such as DivIn, DivGen, Particle Guidance, and STRIDE
  • We have included an additional toy experiment that demonstrates JIVE’s spread/variance increasing property
  • We have added wall-clock runtime and peak memory benchmarks for the 4-direction, 10-iteration Jt⊤JtJ_t^\top J_t variant of JIVE.

We appreciate everyone's engagement and remain happy to address any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.