acceptodds
Under review as a conference paper at ICLR 2027

Probability-Flow Distillation: Distribution Matching in Parameter Space

Abstract

Score distillation methods use pretrained diffusion models as priors for optimizing parameters through differentiable forward models, most notably in text-to-3D generation. Yet the distribution they induce over those parameters is not well understood. Observing that existing distillation methods reduce to one of three—Score Distillation Sampling (SDS), Score Distillation via Inversion (SDI), and Variational Score Distillation (VSD)—we extend the particle variational inference view of VSD to the other two. We show that SDS collapses onto the modes of the target, while SDI converges to a contracted version of it, and explain why SDI needs a negative classifier-free guidance scale. Next, we observe that the DDIM posterior mean equals a single Euler step of the probability-flow ODE (PF-ODE). Replacing this step in SDI with a full reverse solve makes the target a fixed point, but it requires solving two concatenated PF-ODEs. Dropping a Jacobian from the resulting gradient gives Probability-Flow Distillation (PFD), which requires solving only the forward PF-ODE. Experiments on synthetic targets, the CelebA dataset, and text-to-3D generation support our analysis and demonstrate the practical effectiveness of PFD.

Primary area:
generative models
Keywords:
score distillation, diffusion models, particle variational inference, distribution matching, text-to-3D generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZY2COpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Probability-Flow Distillation (PFD). PFD first moves particles using probability-flow ODE of the current particle distribution, rather than just adding Gaussian noise on it. Then, PFD leverages the differnece between the target score and the particle score for optimization, which follows VSD. Main experiments cover image generation on CelebA (10K samples) and text-to-3D (only 22 samples).

Critical strengths
  1. This paper studies an interesting problem. PFD tries to calculate the score differences along a determinstic trajectory, without the randomness of adding a gaussian noise.

  2. Experiments are reasonably controled. PFD and VSD are both tested under same time quote and times of score evaluation. The text-to-3D experiment is also conducted under the same prior model, representation, and optimizer. CelebA's experiment covers 4 seeds and 10K samples.

Critical weaknesses
  1. My biggest concern is that the performance gains over VSD is very slight. In table 1 the quality metirics are basically the same, and PFD's FID is even worse. In table 2, the ImageReward improves but the CLIP score is also basically the same. And both methods takes 2 hours which is significantly slow. It's also very hard to tell the difference in figure 2 & 3 (except figure 2's right subfigure, which clearly shows the determinstic nature of PFD). I believe that PFD makes the distillation more determinatic, but why neither the quality nor efficiency seems to be benefited from this change.

  2. My second-biggest concern is that the 3D generation quality comparison is conducted on only 22 samples, making these experiments less convencing compared to CelebA's experiment on image generation.

  3. The paper's "mode-seeking" claim for prior methods is questionable. Methmatically, equation 13 can be considered as optimizing the "mean" across different noise levels, but it's methmatically different from the "mode" of clean distribution p0p_0. For example, there are many samples around -1 and +1, so there are 2 modes, but the global optimization direction will be pointed to 0, which is clearly not a mode. This issue may make the methmatical logics of PFD less convencing.

Questions for authors

See the weaknesses please.

3 comments

We thank the reviewer for their time and thoughtful comments, which helped us state PFD's advantage over VSD more precisely here. We address each weakness below.


1. Performance gain over VSD.


PFD and VSD differ only in how xt\boldsymbol{x}_t is computed from x0\boldsymbol{x}_0. They share the same fixed point, so given enough steps the FIDs of both reach similar values, which is why they differ only slightly in Table 1. What the deterministic coupling (x0,xt)(\boldsymbol{x}_0, \boldsymbol{x}_t) buys over the stochastic one is a faster rate of convergence. Theorem 5 proves this for general targets in the small-time regime. Extending it beyond this is difficult for general targets and we leave it to future work, but Prop. 6 does so for isotropic Gaussians, where Fig. 2 also confirms it empirically. To show that this also holds beyond the Gaussian case, we now report the comparison for CelebA generation in the table below, which gives FID every 20 iterations (lower is better). FID is computed on 5k samples against 5k real images, so it is not directly comparable to Table 1. For reference, the FID between two disjoint sets of 5k real images is 5.7. PFD reaches each FID level in fewer iterations than VSD at the same per-step cost. For example, PFD reaches an FID of 46.4 at iteration 120, a level VSD reaches only at about iteration 240, that is, with about half the compute. We have added this result, with a plot of FID against iteration, to Sec. 5.2 of the revised manuscript (Fig. 5).

IterationVSDPFDVSDPFD
0465.1465.116073.237.7
20452.9441.118064.535.7
40375.4340.520057.434.0
60356.5314.722050.632.7
80277.9172.724044.631.6
100166.266.926039.829.5
120113.146.428036.029.5
14087.539.430033.527.2

2. Scale of the text-to-3D evaluation.


We acknowledge that 22 prompts is a relatively small evaluation, and we now state this as a limitation in the conclusion. A single run takes about 2 GPU-hours (1 for SDS), so evaluating four methods on 22 prompts took about 154 GPU-hours. Since only one 3D representation fits in GPU memory at a time, these runs had to be done sequentially, which set the scale of the evaluation within our compute budget.


3. Where SDS converges.


We agree that ''modes'' is imprecise here, and we used the term only to follow prior work, where SDS is often associated with ''mode-seeking'' and ''mode collapse''. Eq. 13 does average over noise levels, but in the reviewer's example the particles do not all go to 0. Those starting near +1+1 end at a single point on the positive side, slightly pulled towards 0, and those near −1-1 at its mirror image, so the particle distribution collapses into two spikes with no spread. These spikes lie close to the modes of p0p_0 when the modes are far apart. Whether two modes merge depends on the distance between them relative to the noise level, and since distances typically grow with the dimension, such merging is rare in high dimensions. We refer the reviewer to the 20-D MoG in Fig. 3 (bottom row, panel b), where SDS collapses the particles onto single points close to the modes of p0p_0. To be technically precise, the revision replaces "collapses onto the modes of p0p_0" with "collapses onto point masses near the modes of p0p_0". Either way the spread of p0p_0 is lost, which is the failure of SDS that Sec. 3.1 points out. PFD does not have this problem, since its fixed point is p0p_0 itself.


We hope these clarifications and the new convergence results address the reviewer’s concerns, and we would be glad to discuss any remaining questions.

Dear Reviewer ZY2C,

We would be grateful to know whether our response above and the revised manuscript address your concerns. If there are further experiments that could be completed within the remaining discussion period, it would be very helpful to hear about them now, so that we can report the results before it ends.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer ZY2C,

As the author–reviewer discussion period is closing soon, we would be very grateful if you could share any further concerns you may have, so that we can respond promptly before the deadline.

Reviewer_85VtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies score distillation methods that optimize the parameters of a differentiable generator with a pretrained diffusion prior. Viewing SDS, SDI and VSD as particle dynamics, it argues that SDS collapses onto the modes of the target while SDI converges to a contracted distribution, with closed-form results for isotropic Gaussian targets. It then proposes Probability-Flow Distillation (PFD), which evaluates the VSD score difference along the forward PF-ODE of the particle distribution instead of at a Gaussian perturbation. Experiments on synthetic targets, CelebA and text-to-3D report that PFD matches or exceeds VSD.

Critical strengths

Strength 1. The paper characterizes what each method converges to, which goes beyond prior unified views such as the ParVI view of VSD and the bridge view of SDS-Bridge. In particular, the result that SDI converges to a contracted version of the target, and the explanation of why SDI needs a negative CFG scale, are new and clarify the over-smoothed outputs of SDI.

Strength 2. Theorem 4 gives an explicit expression for the residual of the dual bridge, and PFD turns it into a practical estimator. PFD keeps the target as a fixed point and gives deterministic gradients, up to the randomness of ω, at the same per-step cost as VSD.

Critical weaknesses

Weakness 1. The framing is less novel than presented, and PFD and the title’s parameter-space claim are not supported by the theory.

The ParVI view of VSD and the reduction of SDS to it are from ProlificDreamer. SDS-Bridge (McAllister et al., 2024) already describes each SDS step as a first-order approximation of a dual diffusion bridge and treats SDI as reducing that error. It also implements the full dual bridge, inversion followed by sampling (its Sec. 5), and reports that this lowers FID in all three 2D settings of its Table 3. The paper nevertheless presents Observation 3 and the dual bridge as its own and mentions SDS-Bridge only in Appendix G. There it describes SDS-Bridge as merely alluding to the dual bridge and avoiding it in practice on the grounds of cost. PFD is then obtained by dropping a Jacobian for convenience, and it is not shown to be the gradient flow of any functional. VSD’s velocity is the Wasserstein gradient of its objective, while PFD is a different field that only shares the fixed point. Descent on general targets over the full noise range is not established, and the bias from the dropped term is not analyzed. The speed results hold either only on a small time range [0, η], while experiments use a wide noise range (Thm. 5), or only for isotropic Gaussians (Prop. 6). Both also compare rates at the same weighting and step size. All formal results also assume that g is the identity and the score of q is exact. The dynamics in parameter space, where ∂x₀/∂θ acts as a preconditioner and p₀ may not be reachable, are never studied, despite the title. In practice PFD uses a learned score both for the score difference and to compute the coupling, with one Euler step between 8 to 32 quadrature nodes.

Weakness 2. The experiments do not show distribution matching in parameter space or a clear advantage over VSD.

Quantitative distributional results use only the identity g (synthetic targets and CelebA generation). Among multi-particle settings, non-identity g (symmetric generation, inpainting and vector graphics) appears only in qualitative figures. Text-to-3D, the only quantitative case with a non-identity g, uses a single particle, so the setting the title refers to is never evaluated. On CelebA, PFD has a slightly worse FID than VSD (16.92 vs. 16.61) and the same CMMD. Both also beat the teacher’s own Euler and Heun samples (25.54 and 24.43), which suggests that the metric does not measure matching p₀, and the paper does not explain this. The claim that PFD converges faster is tested only on the one-dimensional Gaussian with closed-form scores. In text-to-3D, SDS uses a different backbone (DeepFloyd IF) and CFG 7.5 instead of DreamFusion’s 100. Despite the reasons given in Appendices G and H.3, this confounds the comparison. Evaluation uses 22 prompts with no seeds or variance reported. CLIP R-precision ties VSD and PFD on two of three backbones, leaving ImageReward (0.680 vs. 0.891) as the only clear difference. Two closely related PF-ODE-based methods are neither cited nor compared: DreamFlow (Lee et al., 2024), which SDS-Bridge itself names as a way to reduce the first-order error, and Consistent Flow Distillation (Yan et al., 2025). ISM, CSD, NFSD and SDS-Bridge are discussed only in Appendix G and are not compared experimentally either.

References

  • Wang et al., ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, NeurIPS 2023.
  • McAllister et al., Rethinking Score Distillation as a Bridge Between Image Distributions, NeurIPS 2024.
  • Lee et al., DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow, ICLR 2024.
  • Yan et al., Consistent Flow Distillation for Text-to-3D Generation, ICLR 2025.
Questions for authors

Please see weaknesses.

7 comments

We thank the reviewer for their detailed comments and respond to each point below, in two comments.


1. Position w.r.t. SDS-Bridge.

SDS-Bridge describes SDS as a first-order approximation of a Schrodinger bridge. Observation 3 is a different statement, and an exact one. Tweedie's posterior mean coincides with a single Euler step of the PF-ODE, so SDI's second step is exactly one Euler step of the reverse solve of the dual bridge, not an approximation of it, which motivates the dual bridge. We thank the reviewer for pointing out the full-path results in SDS-Bridge, which were not in the preprint on the authors' official project page that we had referred to. Even so, its authors note the difficulty of implementing the full path for distillation tasks, and their main algorithm remains the one described in Appendix G, as our appendix intended to convey. We now cite the full-path variant in Section 3.3. Theorem 4, which characterizes the dual-bridge gradient exactly and leads to PFD, is not in SDS-Bridge.


2. Gradient flow, the dropped term and convergence on general targets.

We agree that PFD is not a gradient flow and do not claim otherwise. VSD is the Wasserstein gradient flow of Et[w(t)DKL(qt,τ∥pt)]\mathbb{E}_t[w(t) D_{\mathrm{KL}}(q_{t,\tau} \Vert p_t)], an averaged surrogate, and not of DKL(q0,τ∥p0)D_{\mathrm{KL}}(q_{0,\tau} \Vert p_0) in Eq. (1). VSD therefore has no descent guarantee on the latter either. We instead justify PFD through three results that we prove. Its fixed point is p0p_0. From a common q0,τq_{0,\tau}, it decreases this KL at least as fast as VSD on [0,η][0,\eta] (Theorem 5). For isotropic Gaussians, it converges globally and faster than both VSD and the dual bridge (Proposition 6). On CelebA, it also converges faster than VSD in FID (point 5 and Fig. 5). The dropped Jacobian multiplies the score difference ∇log⁡qt,τ−∇log⁡pt\nabla\log q_{t,\tau}-\nabla\log p_t, which vanishes when q0,τ=p0q_{0,\tau}=p_0, so dropping it leaves the fixed point unchanged and introduces no stationary bias. Sec. 4 now states this explicitly. Dropping it is also not just a convenience, as it makes convergence faster and removes the O(N2)O(N^2) cost of the dual bridge (L339-341). We acknowledge the restriction to the small-time regime as a limitation (L529-530), and extending the result over the full noise range may require stricter assumptions.


3. Identity g\boldsymbol{g}, parameter-space dynamics and exact scores.

The identity g\boldsymbol{g} serves mainly to simplify notation, and all of our analysis except Theorem 5 holds for a general g\boldsymbol{g}. Eq. (1) defines the target π⋆\pi^\star in parameter space as the distribution whose pushforward is as close as possible to p0p_0, which does not require p0p_0 to be reachable. Distribution matching in parameter space refers to driving πτ\pi_\tau, the distribution of the parameters θ\boldsymbol{\theta} that every method updates through Eq. (2), towards π⋆\pi^\star. Since π⋆\pi^\star is defined through its pushforward, our results are stated for it, and they carry over to the parameters θ\boldsymbol{\theta} for a general g\boldsymbol{g} as follows. Under SDS, particles do not interact, so each θ\boldsymbol{\theta} converges on its own to a point and πτ\pi_\tau collapses for any g\boldsymbol{g} (L193-194). Under SDI, the estimator does not vanish at p0p_0 in general. Under VSD and PFD, the estimators vanish at any π\pi with g♯(π⊗ξ)=p0\boldsymbol{g}_{\sharp}(\pi\otimes\xi)=p_0, so such π\pi are stationary for any g\boldsymbol{g}. Theorem 4 holds for any g\boldsymbol{g}, Propositions 1, 2 and 6 concern Gaussian targets, where g\boldsymbol{g} is the identity by construction, and Proposition 7 does not involve g\boldsymbol{g}. The one exception is Theorem 5, which assumes the identity g\boldsymbol{g} so that the comparison isolates the effect of the estimator itself, without the preconditioner ∂x0/∂θ\partial \boldsymbol{x}_0/\partial\boldsymbol{\theta} that a general g\boldsymbol{g} would add to both methods. Theorem 5 now states this assumption explicitly.

Two questions lie outside our claims, namely where the dynamics end when p0p_0 is not reachable, and the effect of a learned score. Neither is straightforward. The first depends on how g\boldsymbol{g} restricts the reachable distributions, and accounting for the score error is difficult even for standard diffusion sampling and harder still in score distillation. To our knowledge, no prior analysis of score distillation, including that of VSD, addresses either. Though important, they are beyond the scope of this work, and the introduction now states this. In our experiments, CelebA and text-to-3D use learned scores, and symmetric generation, inpainting, vector graphics and text-to-3D use a non-identity g\boldsymbol{g}. In all of them the methods behave as our analysis predicts, which suggests that neither effect changes our conclusions.


We continue with points 4 to 7 in the next comment.

4. Quantitative results with a non-identity g\boldsymbol{g}.

Distributional metrics require ground-truth samples from the target, which are available only when g\boldsymbol{g} is the identity. For a non-identity g\boldsymbol{g}, such as symmetric generation, inpainting or vector graphics, no ground-truth distribution exists. Symmetrized CelebA faces, for instance, do not form the intended target. Because of asymmetries in pose and camera angle, many of them are not plausible faces, unlike the samples in Fig. 9. Generating such samples is the purpose of distillation, so for these settings we provide qualitative results (Appendix I). We would also like to point out that single-particle text-to-3D reflects distribution matching with a non-identity g\boldsymbol{g}, as even with one particle q0,τq_{0,\tau} is the non-degenerate distribution of renders over views (detailed in Section 5.3). In this sense, Table 2 serves as an indirect distributional measure.


5. CelebA FID and convergence.

PFD and VSD share the same fixed point, so given enough steps their sample quality becomes similar, which is why they differ only slightly in FID (16.92 vs. 16.61) and not at all in CMMD. The advantage of PFD, in which xt\boldsymbol{x}_t is obtained deterministically from x0\boldsymbol{x}_0, lies instead in how quickly it approaches the fixed point. The submission shows this only for the Gaussian case, so we have now tracked FID during optimization on CelebA (Fig. 5 and Sec. 5.2 of the revised manuscript; full table in point 1 of our response to Reviewer ZY2C). Both methods start from the same 5k particles and are evaluated against 5k real images every 20 iterations, at the same cost per iteration. Because of the smaller sample size, these FIDs are not directly comparable with the 10k FIDs of Table 1. PFD stays ahead throughout, with FIDs of 46.4 vs. 113.1 at iteration 120, 34.0 vs. 57.4 at iteration 200, and 27.2 vs. 33.5 at iteration 300, against a real-vs-real floor of 5.7. VSD needs about 240 iterations to reach the FID that PFD reaches at 120.

As for the samplers, Euler and Heun verify that the pretrained model is reasonably trained, but they do not bound the attainable quality (Heun, though second-order and using twice the NFEs, improves on Euler by only 1.1 FID). A sampler starts from a Gaussian, which only approximates the marginal of the data at the largest noise level, and transports it along a fixed-length trajectory of the learned PF-ODE, so its output depends on this mismatch and on the score errors accumulated along that trajectory. Distillation involves no prior, and its trajectory has no fixed length (L63-65). Its fixed point is where the scores of the particle distribution match the learned score, so the results in Table 1 are consistent with this.


6. SDS backbone and number of prompts.

We agree that the SDS row of Table 2 is not directly comparable to the others. It is there to illustrate the collapse predicted by Proposition 1, not as a baseline. The comparison of interest, PFD against VSD, uses the same backbone, CFG scale, representation and optimizer. A matched SDS row is not possible. At CFG 100, the guidance term dominates and SDS approximates CSD, itself an approximation of VSD (Appendix G, Eq. 60). With Stable Diffusion at CFG 7.5, it collapses to empty space (Appendix H.3). The conclusion now states the use of 22 prompts with a single seed as a limitation. Its size was set by our computational budget, since every prompt requires a separate text-to-3D optimization for each method, and these runs cannot be batched on a single GPU.


7. Related work.

We thank the reviewer for pointing out DreamFlow and Consistent Flow Distillation (CFD). Both are now cited and discussed in Appendix G, together with the related FSD and Consistent3D raised by Reviewer wXFy, and in more detail in our response to them. In summary, DreamFlow uses VSD's estimator along a predetermined decreasing noise schedule, and CFD uses SDI's estimator with xt\boldsymbol{x}_t obtained from x0\boldsymbol{x}_0 by adding a fixed noise rather than by DDIM inversion. As for ISM, CSD, NFSD and SDS-Bridge, we kept their discussion in Appendix G to leave main-text space for the analysis. As stated in the introduction (L55-56), they reduce to one of the three estimators we study. We therefore compare the underlying estimators at matched per-step cost, since differences in representation, optimization, schedule, regularization, or how \boldsymbol{\epsilon}_\boldsymbol{\psi} is approximated are not the subject of this work.


We hope these responses, together with the revisions now in the manuscript, address the reviewer's concerns, and we would welcome any further questions.

Thank you for the response. Points 1, 3, 5, 6 and 7 are addressed, and the new CelebA convergence result is helpful. Points 2 and 4 remain.

Point 2. VSD's objective E[w(t)KL(qt∥pt)]\mathbb{E}[w(t) \mathrm{KL}(q_{t} \Vert p_{t})], averaged over tt, is zero only when q0=p0q_{0} = p_{0}, so VSD decreases a functional whose unique minimizer is the target. PFD has no such functional. That the dropped term vanishes at q=pq = p only shows that p0p_{0} is a fixed point, not that it is unique or attractive for general targets.

Point 4. A non-identity gg can be evaluated quantitatively on simple synthetic data, e.g., a Gaussian π∗\pi^{\ast} with a simple nonlinear gg and target p0=g♯π∗p_{0} = g_{\sharp}\pi^{\ast}, where distances to p0p_{0} or π∗\pi^{\ast} can be measured directly. CLIP R-precision and ImageReward score individual images and do not capture diversity or coverage, so they are hard to read as distributional measures.

Reviewer_wXFyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies a family of score distillation methods: score distillation sampling (SDS) and score distillation via inversion (SDI) and variational score distillation (VSD). More specifically authors identify the distribution that the samples x0 follow for SDS and SDI. In the case of SDS it leads to mode collapse and in the case of SDI it leads to reduced variance (and thus diversity). This work then introduces an updated variant of SDI - probability-flow distillation (PFD), which is obtained by replacing posterior mean with a full reverse ODE solve and then dropping the Jacobian term from the gradient update. Method is validated on a collection of synthetic examples, CelebA dataset, as well as text-to-3D generation.

Critical strengths
  • Work bridges a gap between several existing score distillation methods in a unified particle variational inference formulation. There have been multiple attempts on understanding the mode collapsing behavior of SDS (VSD, ESD, etc), but this work does provide some new insights. For example, it provides a more formal explanation of the negative CFG scale in SDI. This is consequential because it explains some of the observed difference when these methods were applied in practice.
  • Authors provide a theoretical justification of the faster convergence of PFD compared to VSD (under specific assumptions). Empirically proposed method is competitive with a strong baseline (VSD). This suggests that the method is a practical lower-variance alternative to VSD.
Critical weaknesses
  • There is potentially a missing comparison / discussion to existing methods that use a similar technique: Consistent 3D, Consistent Flow Distillation (and FSD). They also replace the random Gaussian noise with PF-ODE trajectories. It seems like these methods at least have to be cited and discussed in detail. Without this, I am more inclined towards reject.
Questions for authors
  • Could authors discuss relationship between PFD and similar methods (CFD/FSD/Consistent 3D) which also rely on variants of deterministic ODE trajectories? Is there a significant difference and a benefit of your proposed formulation over those methods?
3 comments

Position w.r.t. FSD, CFD, Consistent3D and DreamFlow.


We thank the reviewer for their time and for pointing out these works, which we had missed. A clarification first: none of them replaces the random Gaussian noise with a PF-ODE. They invoke the PF-ODE to interpret the optimization trajectory, as SDI does, through the observation that an SDS update closely resembles a DDIM (PF-ODE) step. Their shared idea is to draw the noise once per particle (and, in 3D, per view) rather than resample it at every step, xt=x0+σtϵ~\boldsymbol{x}_t=\boldsymbol{x}_0+\sigma_t\tilde{\boldsymbol{\epsilon}} with ϵ~\tilde{\boldsymbol{\epsilon}} fixed, which makes the map from x0\boldsymbol{x}_0 to xt\boldsymbol{x}_t deterministic. This places them in the SDI family of our grouping (Appendix G), i.e., methods that obtain xt\boldsymbol{x}_t from x0\boldsymbol{x}_0 deterministically, whether by DDIM inversion, by fixed-point iteration, or by adding a fixed noise.

Flow Score Distillation (FSD) [1] and Consistent Flow Distillation (CFD) [2] share the same gradient estimator (Eq. 15 in [1], Eq. 9 in [2]); CFD builds on FSD with additional components for better 3D optimization. This estimator is equivalent to SDI's, except that xt\boldsymbol{x}_t is obtained by adding a fixed noise instead of by DDIM inversion. SDI's own ablation covers this choice (Sec. 6.2, Fig. 9), and its authors found DDIM inversion to work better than adding a fixed noise. Section 3.2 suggests why, since DDIM inversion approximates the forward PF-ODE of q0,τq_{0,\tau} and so ties xt\boldsymbol{x}_t to the current particle distribution, whereas adding a fixed noise does so only in a weaker sense. Consistent3D's [3] estimator is closer to ISM's (Appendix G), as both take the difference of \boldsymbol{\epsilon}_\boldsymbol{\phi} at two adjacent noise levels; but ISM obtains xt\boldsymbol{x}_t by DDIM inversion, as SDI does, while Consistent3D adds a fixed noise, as FSD and CFD do. On an isotropic Gaussian target, all three recover the correct mean but, in general, not the correct variance, as does every method in the SDI family. PFD, in contrast, has p0p_0 as its fixed point and recovers both the mean and the variance.

Following Reviewer 85Vt, we also note DreamFlow [4], which approximates the PF-ODE of the Schrodinger bridge from the parameterized image distribution to the natural image distribution. The resulting update, however, is VSD's gradient estimator (the difference between the pretrained score and the score of the current distribution, evaluated at the same freshly drawn noise), applied along a predetermined, decreasing noise schedule. It therefore remains stochastic in the noise, and the lower gradient variance it reports comes mainly from replacing VSD's randomly drawn timesteps with this schedule.

MethodHow xt\boldsymbol{x}_t is obtained from x0\boldsymbol{x}_0Models q0,τq_{0,\tau}Familyp0p_0 a fixed point
SDSfresh noisenoSDSno (collapse)
SDIDDIM inversionapproximatelySDIno
FSD, CFDfixed noisenoSDIno
Consistent3Dfixed noisenoSDI (as ISM)no
VSDfresh noiseyesVSDyes
DreamFlowfresh noise, decreasing scheduleyesVSDyes
PFD (ours)forward PF-ODE of q0,τq_{0,\tau}yes—yes

We have added this discussion to Appendix G of the revised manuscript (marked in blue). FSD, CFD, Consistent3D and DreamFlow are now cited, discussed, and placed in the grouping of Fig. 7. We hope this addresses your concern and are happy to discuss any further questions.

References


[1] R. Yan, K. Wu, K. Ma. Flow Score Distillation for Diverse Text-to-3D Generation. arXiv:2405.10988, 2024.

[2] R. Yan, Y. Chen, X. Wang. Consistent Flow Distillation for Text-to-3D Generation. ICLR 2025.

[3] Z. Wu, P. Zhou, X. Yi, X. Yuan, H. Zhang. Consistent3D: Towards Consistent High-Fidelity Text-to-3D Generation with Deterministic Sampling Prior. CVPR 2024.

[4] K. Lee, K. Sohn, J. Shin. DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow. ICLR 2024.

AuthorsOpenReview ↗

Revised manuscript uploaded

Dear Reviewer wXFy,

We have uploaded the revised manuscript. Appendix G now cites and discusses FSD, CFD, Consistent3D and DreamFlow, and Fig. 7 places them in our grouping of distillation methods. A summary of all revisions is in our general response.

Since the discussion of these works was the main concern in your review, we would be grateful to know whether it is now addressed, or whether any further clarification would help.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer wXFy,

As the author–reviewer discussion period is closing soon, we would be very grateful if you could share any further concerns you may have, so that we can respond promptly before the deadline.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common Response

We thank the reviewers and the AC for the time and effort spent on our submission. As noted by Reviewers 85Vt and wXFy, the analysis offers a new theoretical view of SDI and an explanation of its negative CFG scale. It also identifies a limitation shared by every method that uses a similar gradient estimator, namely that the target is not a fixed point. On the theoretical side, Theorem 4, highlighted by Reviewer 85Vt, characterizes the dual diffusion bridge exactly. On the practical side, PFD is an alternative to VSD at the same cost per step, with provably faster convergence for isotropic Gaussians and empirically faster convergence in FID on CelebA.

We also acknowledge the limitations. Theorem 5 is restricted to the small-time regime and identity g\boldsymbol{g}, yet it still shows why the way xt\boldsymbol{x}_t is obtained from x0\boldsymbol{x}_0 matters. By Proposition 7, the leading-order difference between the two couplings is the generator of the Langevin dynamics of q0,τq_{0,\tau}. Since this generator is negative semi-definite, the gap in Theorem 5 is a non-negative squared norm. The text-to-3D evaluation uses a relatively small set of prompts, but its results agree with the analysis. Finally, our analysis does not cover targets not reachable through g\boldsymbol{g} or errors in learned scores. Both are limitations shared with prior analyses of score distillation, including that of VSD, and a rigorous treatment of either is beyond the scope of this work.

The revised manuscript is uploaded, with changes marked in blue. We summarize them below by reviewer. Whenever we revise the manuscript, we also update the figure, table and line references in our responses to the reviewers to match it.

Reviewer wXFy

  • Appendix G now cites and discusses FSD, CFD, Consistent3D and DreamFlow, and Fig. 7 places them in the grouping.

Reviewer 85Vt

  • Sec. 3.3 credits the full-path variant of SDS-Bridge, and its description in Appendix G is corrected.
  • Sec. 4 states that the dropped Jacobian leaves the fixed point unchanged and introduces no stationary bias.
  • Sec. 4 (Comparison with VSD) states that, unlike VSD, PFD has no functional with p0p_0 as its unique minimizer, and that the uniqueness and attraction of its fixed point are shown globally for isotropic Gaussians and, to leading order, for small times with identity g\boldsymbol{g}.
  • Theorem 5 states the identity-g\boldsymbol{g} assumption. The conclusion lists it as a limitation, together with the 22-prompt text-to-3D evaluation.
  • Sec. 5.1 adds a synthetic experiment with a nonlinear g\boldsymbol{g} (new Fig. 4), reporting MMD to the target in both data and parameter space.
  • Sec. 5.2 adds a convergence comparison of PFD and VSD on CelebA (new Fig. 5), extending the isotropic Gaussian comparison of Fig. 2.
  • The introduction states that exact scores are assumed, as in prior work, and places a rigorous treatment of learned scores and of targets not reachable through g\boldsymbol{g} outside the scope of this work.
  • The abstract now refers to the distribution induced over the generated data, which is what the analysis characterizes.
  • DreamFlow and CFD are cited and discussed (Appendix G).

Reviewer ZY2C

  • New Fig. 5 and text in Sec. 5.2 track FID during optimization on CelebA, where PFD reaches a given FID in about half the iterations of VSD.
  • "Collapses onto the modes" is replaced throughout by "point masses near the modes".
  • The conclusion states the 22-prompt, single-seed evaluation as a limitation.

We are happy to answer any further questions. We would be grateful if the reviewers could share their responses when convenient, so that any remaining concerns can be clarified within the discussion period.

Area_Chair_BufBOpenReview ↗

discussion reminder

Dear reviewer:

At first, thank you for being a reviewer for ICLR 2027, and thanks for your contributions so far.

We are half way through the discussion period. Please check if the authors' response have addressed your concerns, any remaining/extended concerns, and update your final rating with justification as early as possible.

Thank you for your contributions again

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.