acceptodds
Under review as a conference paper at ICLR 2027

Wasserstein Convergence of ODE-Based Samplers in Decentralized Diffusion Model via Velocity Field Decomposition

Abstract

Diffusion models have achieved impressive empirical success, accompanied by substantial progress in understanding their sampling convergence. Motivated by privacy and scalability, decentralized diffusion architectures replace a single global model with local experts and a learned routing mechanism. Random expert switching introduces fluctuations beyond ordinary ODE discretization error, requiring an analysis of how local approximation errors and routing randomness jointly affect generation. We study the sampling convergence of decentralized diffusion models for finite-mixture distributions under linear flow interpolation, using a single-expert-per-step Euler sampler with normalized, state-dependent routing. Under compact support, bounded learned data predictions, and spatial and temporal regularity of the learned composite field, we establish a non-asymptotic Wasserstein-2 () guarantee based on time-integrated population errors. We measure expert error under the respective component marginals and router error under the mixture marginals. At a fixed cutoff , the -step sampler admits a upper bound of order relative to the exact cutoff marginal. The key is to combine posterior-weighted error decomposition with cancellation of the experts' shared linear drift, allowing ordinary router error to enter the bound linearly without an additional spatial weight. Our analysis separates deterministic integration error from stochastic routing fluctuations and explicitly accounts for data-end truncation, connecting local population approximation accuracy to distributional guarantees for decentralized sampling.

Primary area:
generative models
Keywords:
Convergence analysis, diffusion models, ODE, Wasserstein Distance

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_5cMKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies the convergence of probability-flow ODE sampling in decentralized diffusion/flow-matching models. The data distribution is modeled as a finite mixture, and the global velocity field is represented as a posterior-routing-weighted superposition of component expert velocities. At sampling time, each step randomly selects one expert according to learned normalized routing probabilities and performs an Euler update. The sampling error therefore combines expert approximation, routing approximation, Euler discretization, stochastic expert selection, and data-end truncation.

The main contribution is, the paper provide a first non-asymptotic Wasserstein-2 convergence analysis for stochastic-routing ODE sampler. By coupling the true reverse flow, the learned composite flow, its deterministic Euler approximation, and the decentralized sampler, the authors separate the different error sources and show that normalized routing cancels the shared drift, allowing the routing error to enter the final bound linearly through an ordinary L2L^2 error. The resulting bound to the cutoff marginal consists of expert and routing approximation errors plus an N−1/2N^{-1/2} finite-step term; comparing with the data distribution adds a truncation term. The constants explicitly depend on the data support radius, dimension, number of mixture components, cutoff time, and learned-field regularity constants. The paper also discusses limitations, including strong assumptions, the absence of a joint vanishing-cutoff rate, and the lack of a finite-sample statistical guarantee.

Critical strengths

1.Addresses a clear theoretical gap in a relevant setting. The paper provides the first rigorous non-asymptotic W2W_2 convergence analysis for decentralized probability-flow ODE sampling with state-dependent stochastic expert routing. This is a meaningful contribution relative to prior SDE-based convergence analyses and to empirical decentralized diffusion frameworks that lack a theoretical foundation.

2.Clean error decomposition with a key technical observation and complete proof. The analysis separates expert approximation, routing approximation, Euler discretization, stochastic expert selection, and data-end truncation. The observation that normalized routing cancels the shared drift—allowing the router error to enter the W2W_2 bound linearly through an ordinary L2L^2 error is elegant and technically useful. The proof is complete, the constants are explicit, and the limitations are candidly discussed.

Critical weaknesses

Weakness 1: The bound is not a joint vanishing-cutoff rate, and its quantitative content under the dependence on δ\delta is not illustrated.

The authors explicitly acknowledge in Theorem 4.5 that “the theorem does not assert a joint rate as the cutoff and approximation errors tend to zero,” and in Section 6 that coefficients contain inverse powers of δ\delta and that the bound “does not yield a joint vanishing-cutoff rate without additional control of these dependencies.” This is a candid limitation. However, the issue is not merely that a joint rate is absent; the displayed bound mixes a truncation term δR2+d\delta\sqrt{R^2+d} with error coefficients such as AaA_a and ASA_S that scale like δ−1\delta^{-1}. Without any illustrative balancing, it remains unclear whether the bound is quantitatively informative in a non-asymptotic regime, even for fixed cutoff. The paper states the limitation but does not help the reader interpret the trade-off.

Weakness 2: The core regularity and boundedness assumptions are strong and are not shown to be satisfiable by an explicit construction, nor connected to finite-sample training.

Assumptions 4.2–4.4require bounded learned data predictions ∥ml,θ∥≤R\|m_{l,\theta}\|\le R, normalized routing, global spatial and temporal Lipschitz regularity of the learned composite field, and population time-integrated L2L^2 approximation errors. Section 6 explicitly acknowledges that “bounded data predictions and global regularity of the learned composite field are explicit assumptions and require justification for a given implementation,” and that “the analysis does not supply a statistical guarantee connecting finite-sample training to these population errors.” The authors are transparent about this. But the main theorem’s guarantee rests on assumptions that are not automatic for standard neural parameterizations, and the paper does not indicate whether they can be enforced without altering the learned field in a way that preserves the normalized routing structure.

Questions for authors

Question 1: Could the authors provide a concrete illustrative calculation? For example, for fixed NN and fixed ϵv,ϵa\epsilon_v,\epsilon_a, what order of δ\delta minimizes the displayed bound, and what total rate in NN would result? If such balancing is not meaningful under the paper’s scope, could the authors explain why, and give guidance on how a practitioner should choose δ\delta for a given sampling budget NN when reading Theorem 4.5?

Question 2: Could the authors give at least one explicit construction?For example, output clipping or spectral normalization combined with routing renormalization—that provably satisfies Assumptions 4.2–4.3 while preserving the normalized routing? If such a construction is not available, could the authors identify which step of the proof fails under only local Lipschitz regularity, so that readers can understand precisely where the gap between the theory and practical networks lies?

4 comments

Thank you for the careful reading and constructive feedback!

W1 (Joint rate and quantitative content of the δ\delta-dependence). “The bound is not a joint vanishing-cutoff rate, and its quantitative content under the dependence on δ\delta is not illustrated.”

Response. Thank you for pointing this out. The original bound makes the dependence on δ\delta explicit; we now use this dependence to explain the trade-off between truncation and the remaining error sources.

Write D=R2+dD=\sqrt{R^2+d} and η=ϵv+ϵa+N−1/2\eta=\epsilon_v+\epsilon_a+N^{-1/2}. Substituting the explicit coefficients into the main bound gives

W2(qN,δθ,p0)≤Dδ+K(δ)η,W_2(q_{N,\delta}^{\theta},p_0)\le D\delta+K(\delta)\eta,

where K(δ)K(\delta) is stated explicitly in Corollary A.12. The term DδD\delta is the cost of stopping at t=δt=\delta, while K(δ)K(\delta) collects the dependence of flow stability, routing approximation, and discretization on the cutoff. Thus a smaller δ\delta reduces truncation but may amplify the other errors through K(δ)K(\delta).

This expression also gives a direct sufficient condition for joint convergence. As η→0\eta\to0, any choice δη→0\delta_\eta\to0 satisfying

K(δη)η⟶0K(\delta_\eta)\eta\longrightarrow0

implies

W2(qN,δηθ,p0)⟶0.W_2(q_{N,\delta_\eta}^{\theta},p_0)\longrightarrow0.

Hence, once the growth of the learned regularity constants with δ\delta is controlled, the main theorem yields joint convergence to the data distribution. A concrete polynomial rate and its associated cutoff choice are given in Q1.

Revision. We have added Corollary A.12 and its complete proof to the appendix, including the general joint-convergence condition and two concrete rates. The paragraph following Theorem 4.5 and the “Cutoff and stability” paragraph in Section 6 now summarize the cutoff trade-off and the corresponding choices of δ\delta.

W2 (Satisfiability of the assumptions). “The core regularity and boundedness assumptions are strong and are not shown to be satisfiable by an explicit construction ...”

Response. Thank you for raising these points. We clarify below how the assumptions relate to the structure of the true field, the training objectives, and the sampling analysis.

Under compact support, the true component data predictions are bounded by RR, and the true routing weights are nonnegative and sum to one. The true composite field also satisfies spatial and temporal bounds of the same form as Assumption 4.3, with explicit constants in Lemma A.3. This provides structural motivation for considering analogous regularity conditions on the learned field.

Spatial regularity controls the comparison between the true and learned flows and the propagation of existing errors in the discrete sampler. Temporal regularity controls the error from freezing the velocity field within each Euler step. The analysis does not require the learned field's derivatives to approximate those of the true field.

The expert errors ϵv\epsilon_v and routing error ϵa\epsilon_a correspond to population L2L^2 errors for the component flow-matching and routing objectives, respectively. They therefore connect directly to the training objectives and preserve independent expert training. Assuming population L2L^2 errors is a common interface in sampling-convergence analyses, separating the statistical training problem from the convergence analysis of the sampler. Our theorem takes these population errors as inputs and characterizes their propagation through sampling. Converting finite-sample training losses into population-error guarantees requires additional statistical and optimization analysis.

Revision. After Assumption 4.3, we now explain the distinct roles of learned-field regularity and population L2L^2 errors in the proof. In the “Population errors and training” paragraph of Section 6, we further specify the relationship between expert population L2L^2 errors, flow-matching excess risk, and independent training, and distinguish the sampling guarantee from finite-sample statistical analysis.

Q1 (Choice of δ\delta and its dependence on NN). “For fixed NN and fixed ϵv,ϵa\epsilon_v,\epsilon_a, what order of δ\delta minimizes the displayed bound ...”

Response. Corollary A.12(c) gives an explicit cutoff calculation under the following additional conditions on the composite predictor: mˉθ(x,⋅)\bar m_\theta(x,\cdot) is absolutely continuous, and

\qquad \|\partial_t\bar m_\theta(x,t)\| \le\frac{B(1+\|x\|)}{t^3},$$ where $A,B$ are uniform across the network family and cutoff choices. Take population-error budgets $\epsilon_v,\epsilon_a$ that are valid across the cutoff choices being compared. Let $$\xi=\epsilon_a+N^{-1/2}, \qquad C_S=e^A\max\{R\sqrt L,2R\}.$$ The corollary gives $$W_2(q_{N,\delta}^{\theta},p_0) \le e^A\epsilon_v+D\delta+\frac{C_S\xi}{\delta} +\frac{C_E}{N\delta^3},$$ where $D=\sqrt{R^2+d}$ and $C_E$ is explicit in the corollary. Both $C_S$ and $C_E$ are independent of $N$ and $\delta$. The four terms represent expert approximation, truncation, routing approximation together with stochastic routing, and deterministic Euler discretization, respectively. For fixed remaining constants, minimizing this upper bound gives $\delta_*\asymp\sqrt\xi$. Indeed, differentiation yields the stationarity equation $$D-\frac{C_S\xi}{\delta_*^2} -\frac{3C_E}{N\delta_*^4}=0,$$ which directly determines the optimal cutoff for fixed $N,\epsilon_v,\epsilon_a$. For a simpler explicit rate, choose $\delta=\sqrt\xi$ when $0<\xi<1$. Then $$W_2(q_{N,\sqrt\xi}^{\theta},p_0) \le e^A\epsilon_v+(D+C_S+C_E)\sqrt\xi \lesssim\epsilon_v+\sqrt{\epsilon_a}+N^{-1/4}.$$ In particular, $\epsilon_v=O(N^{-1/4})$ and $\epsilon_a=O(N^{-1/2})$ yield an $O(N^{-1/4})$ total rate. Corollary A.12(c) contains the explicit bound, cutoff choice, and complete proof, and Section 6 summarizes the resulting polynomial rate.

Q2 (Parameterization and the role of global regularity). “Could the authors give at least one explicit construction ... If such a construction is not available, could the authors identify which step of the proof fails under only local Lipschitz regularity ...”

Response. For the boundedness and normalization conditions in Assumption 4.2, let zl,θ(x,t)z_{l,\theta}(x,t) denote the raw output of expert ll, which need not be bounded, and define

\frac{Rz_{l,\theta}(x,t)} {\sqrt{R^2+\|z_{l,\theta}(x,t)\|^2}}.$$ This map is smooth in the raw output and satisfies $$\|m_{l,\theta}(x,t)\|^2 =\frac{R^2\|z_{l,\theta}(x,t)\|^2} {R^2+\|z_{l,\theta}(x,t)\|^2}<R^2,$$ so $\|m_{l,\theta}(x,t)\|<R$ without requiring the raw output to be bounded. The routing weights can still be parameterized by $a_\theta(x,t)=\operatorname{softmax}(g_\theta(x,t))$, giving $a_{l,\theta}\ge0$ and $\sum_l a_{l,\theta}=1$ while preserving the normalized-routing structure. This construction addresses Assumption 4.2. For Assumption 4.3, we follow the alternative in the question and identify below precisely where the proof uses global regularity. The spatial and temporal regularity conditions on the composite field provide the stability controls used in the sampling analysis. Spectral-norm constraints, together with Lipschitz activations, can be used to control spatial Lipschitz constants; temporal regularity additionally requires suitable control of the architecture's temporal dependence. Regarding local Lipschitz regularity, write $f$ for the learned reverse composite field, $X,Z$ for the true and learned continuous flows, $\widehat Y$ for deterministic composite-field Euler, and $Y$ for the stochastic-routing sampler. The global spatial Lipschitz condition is used in the following three estimates: 1. **Comparison of the true and learned flows.** The proof uses $$\|f(X,\tau)-f(Z,\tau)\|_2\le\Lambda\|X-Z\|_2.$$ Grönwall's inequality then converts the population approximation error, evaluated at the same position, into a terminal flow error with stability factor $e^{\Lambda T}$. 2. **Mean-square recursion for stochastic expert routing.** With $D_k=Y_k-\widehat Y_k$, controlling the propagation of the existing error through the next update uses $$\left\|D_k+h\{f(Y_k,\tau_k)-f(\widehat Y_k,\tau_k)\}\right\| \le(1+\Lambda h)\|D_k\|.$$ Local spatial Lipschitz regularity alone does not directly provide this uniform recursion coefficient. 3. **Euler discretization.** Spatial Lipschitz regularity is used again to propagate the local errors. Since Euler freezes the field within each step, the estimate also uses the separate temporal condition $$\|f(x,\tau)-f(x,s)\| \le\Gamma(1+\|x\|)|\tau-s|$$ to control velocity changes due to both position and time. Thus, with only local spatial Lipschitz regularity, these estimates do not apply directly. A localized argument would introduce radius-dependent stability constants and tail errors outside the localization region. This adds substantial complexity, particularly in balancing the growth of the local stability constants with the decay of the tail errors. We therefore adopt global spatial Lipschitz regularity as a trade-off between assumption strength and the simplicity and explicitness of the analysis. Establishing guarantees under weaker regularity is a direction for future work.

Thank you for the detailed responses. They address both of my weaknesses.

W1 / Q1. Corollary A.12 is exactly the illustrative calculation I asked for. It makes the cutoff trade-off explicit. It gives the optimal choice of cutoff and states a sufficient condition for joint convergence.

W2 / Q2. The clipped-expert construction answers my concern about Assumption 4.2. My original worry was whether boundedness could be enforced at all without altering the learned field in a way that preserves the routing structure. The construction handles both parts of that worry. Provided the clipping threshold is at least the true bound and the map is 1-Lipschitz, it acts as the identity on the true field and adds no approximation error. The routing weights stay normalised. For Assumption 4.3, you identify three places that use global spatial regularity: the flow comparison, the one-step mean-square recursion and the Euler discretisation. That is a satisfactory answer. The trade-off you describe for a localised argument is also reasonable.

I have updated my soundness score from 2 to 3. My rating stays at 3. I have no further requests.

Reviewer_HNyjOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the convergence of decentralized diffusion and flow-matching models using multiple local experts. It establishes theoretical Wasserstein-2 convergence bounds under specific assumptions and provides supporting image-generation experiments.

Critical strengths

The paper addresses a relevant theoretical problem in decentralized generative modeling. The analysis considers both expert approximation errors and routing errors, and the experiments provide some empirical support.

Critical weaknesses

The experiments do not directly validate the theoretical Wasserstein-2 convergence bound. In particular, the trajectory error analysis measures discrepancy against a numerical DDIM reference rather than the target distribution. A more direct connection between the theoretical results and empirical observations would strengthen the paper.

Questions for authors

Is it possible to provide a more intuitive explanation of the main theoretical findings and discuss their implications for practical decentralized generative models? Additional experiments connecting the theoretical assumptions and bounds to observed performance would strengthen the paper.

1 comment

Thank you for the careful reading and constructive feedback.

W1 (Direct validation of the W2W_2 bound). “The experiments do not directly validate the theoretical Wasserstein-2 convergence bound ...”

Response. We agree that the trajectory experiment does not measure the distance to the target distribution. It does, however, directly test the discretization part of our proof. The proof couples all processes through a common initial noise and bounds the distance between the NN-step sampler YNY_N and the continuous learned flow Z(T)Z(T):

\le\\|Y_N-\widehat Y_N\\|_{L^2}+\\|\widehat Y_N-Z(T)\\|_{L^2} \le\frac{A_S}{\sqrt N}+\frac{A_E}{N}$$ (Propositions A.9 and A.8). The experiment uses the same coupling. The reference is the deterministic scheme $\widehat Y$ with the same experts and router and 1000 steps, so it approximates $Z(T)$; each sample shares its initial noise with its reference; and $E(N)$ is the empirical version of the squared left-hand side. The measured excess decays as $N^{-0.65}$, between the routing rate $N^{-1/2}$ and the Euler rate $N^{-1}$, as the sum of the two terms predicts. The approximation terms $A_v\epsilon_v+A_a\epsilon_a$ cannot be tested on image data, since the true velocity field is unknown. **Q1 (Intuition and practical implications).** *“Is it possible to provide a more intuitive explanation of the main theoretical findings ...”* **Response.** Yes. The bound is a sum of four errors: the experts' training errors, the router's training error, the error of finite steps with random expert selection, and the data-end cutoff. Each enters additively, and the experts can be trained separately on their own data. The least standard term is the $N^{-1/2}$ rate from random routing, which has a simple explanation. At each step the sampler uses one randomly selected expert instead of the router-weighted average of all experts. The resulting local error $h\xi_k$ has size $O(h)=O(1/N)$ and zero mean given the current state. Such errors are uncorrelated, so, ignoring their propagation through later steps, the variance of their sum is the sum of their variances: $$\mathbb E\Big\\|\sum_{k=0}^{N-1}h\xi_k\Big\\|^2 =\sum_{k=0}^{N-1}h^2\\,\mathbb E\\|\xi_k\\|^2 \lesssim N\cdot\frac1{N^2}=\frac1N .$$ Taking the square root gives $N^{-1/2}$. With $\\|\xi_k\\|\le2R/\delta$, this heuristic already gives the constant $2RT/\delta$, which is $A_S$ without the factor $e^{\Lambda T}$. Had these errors a common sign, they would accumulate to $O(1)$; the deterministic Euler errors are $O(1/N^2)$ per step and accumulate to $O(1/N)$. The proof (Proposition A.9) handles two effects that this heuristic ignores. First, the local errors are not independent, because the selection probabilities depend on the current state; we use that they form a martingale difference sequence, so the cross terms still vanish after conditioning. Second, each local error is propagated, and possibly amplified, by the subsequent steps; a discrete Grönwall argument bounds this amplification by $e^{\Lambda T}$. In practice, evaluating one expert per step keeps the per-step cost of a single model, at the price of the rate $N^{-1/2}$ instead of the $N^{-1}$ obtained when all experts are evaluated at every step. **Revision.** We add this explanation after Theorem 4.5 in the main text.
Reviewer_ntwuOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the sampling convergence of decentralized diffusion models, where a single global field is replaced by multiple local experts and a learned stochastic router. The authors analyze the probability-flow ODE in this setting and establish a non-asymptotic W-2 guarantee for a single-expert-per-step Euler sampler, bounding the distance to the cutoff marginal (and, with truncation, to the data distribution) in terms of time-integrated population L2 errors of the experts and the router. The contribution is essentially theoretical, with supporting experiments in the appendix.

Critical strengths

The paper has a few significant strengths overall, which I will outline below.

Strengths:

  1. The paper addresses a relevant and under-analyzed problem, and to my knowledge provides the first non-asymptotic W2 guarantee for decentralized sampling with stochastic routing.
  2. Treating the routing fluctuations as a martingale-difference sequence, rather than a worst-case perturbation, is reasonable and gives a cleaner handle than a direct ODE bound.
  3. The analysis is careful and separates the different error sources (expert approximation, routing, Euler discretization, truncation).
Critical weaknesses

However, despite its strengths, the paper has a few major and minor weaknesses.

Major weaknesses:

  1. The obtained bound is not really usable as a guarantee to the data distribution - the authors themselves admit it does not yield a joint vanishing-cutoff rate, since the constants blow up while the truncation term needs -> 0. For a purely theoretical paper this is a serious limitation.
  2. There is a clear disconnect between the theory and the experiments. The experiments evaluate a different contribution (the derivative-assisted regularization) that is not analyzed in the theory, and the one experiment aimed at the rate measures discrepancy from a numerical reference, which the authors state does not test the bound. So the actual theorem has essentially no empirical support, and no genuinely decentralized deployment is shown.
  3. The assumptions are strong and imposed directly on the learned network, without justification that they hold, and the population errors are not connected to any finite-sample / trainable quantity.
  4. The novelty over the existing ODE-based W2 analyses seems incremental - the main new ingredient is the martingale treatment of the routing noise. The paper should also position/compare its rate against the SDE- and ODE-based results it only cites.

Minor weaknesses:

  1. The main result is only partially self-contained: the constants in Theorem 4.5 appear explicitly only in Eq. (42) in the appendix, and the main body gives only a proof sketch. For a paper whose central contribution is the theorem, I would expect more of it in the main body.
  2. The notation is dense and some objects/operators are used with minimal introduction, which hurts the readability.
Questions for authors

I would like to see especially the responses to the issues mentioned as weaknesses. Besides, I have a few additional questions:

  1. Do you have convergence results in other metrics than W2 (e.g., TV or KL), or is there a fundamental obstruction from the deterministic PF-ODE?
  2. How practically important is the analyzed setting? Could you give concrete real-world scenarios where decentralized diffusion models are actually used, and where your assumptions are expected to hold?
  3. Could the bound be made to vanish jointly as δ→0, at least under stronger assumptions?
7 comments

Thank you for the careful reading and constructive feedback.

W1 (Joint vanishing-cutoff rate). “The obtained bound is not really usable as a guarantee to the data distribution ...”

Response. Corollary A.12 gives joint convergence rates to the data distribution under explicit conditions on learned regularity and stability. The cutoff is chosen as a function of the population approximation errors and the number of sampling steps, using the expanded coefficients of Theorem 4.5.

Let D=R2+dD=\sqrt{R^2+d} and η=ϵv+ϵa+N−1/2\eta=\epsilon_v+\epsilon_a+N^{-1/2}.

  • General form. If the learned regularity constants satisfy Λ≤Λˉ(δ)\Lambda\le\bar\Lambda(\delta) and Γ≤Γˉ(δ)\Gamma\le\bar\Gamma(\delta), then W2(qN,δθ,p0)≤δD+K(δ),ηW_2(q^\theta_{N,\delta},p_0)\le\delta D+K(\delta)\\,\eta, where K(δ)K(\delta) is explicit. Any choice δη→0\delta_\eta\to0 with K(δη)η→0K(\delta_\eta)\eta\to0 gives W2(qN,δηθ,p0)→0W_2(q^\theta_{N,\delta_\eta},p_0)\to0.

  • Learned regularity controlled by true-field upper bounds. Lemma A.3 gives Λ\*δ=δ−1+R2δ−3\Lambda_\*^\delta=\delta^{-1}+R^2\delta^{-3} and Γ\*δ\Gamma_\*^\delta. If the learned constants are at most cc times these bounds, with fixed cc, the choice δη≍[log⁡(1/η)]−1/3\delta_\eta\asymp[\log(1/\eta)]^{-1/3} gives W2(qN,δηθ,p0)≲D,[log⁡(1/η)]−1/3W_2(q^\theta_{N,\delta_\eta},p_0)\lesssim D\\,[\log(1/\eta)]^{-1/3}. When ϵv+ϵa≤N−1/2\epsilon_v+\epsilon_a\le N^{-1/2}, this is (log⁡N)−1/3(\log N)^{-1/3} end to end.

  • Stable composite data prediction. Under the additional conditions Lip⁡(mˉθ(⋅,t))≤1+At\operatorname{Lip}(\bar m_\theta(\cdot,t))\le1+At and ∣∂tmˉθ(x,t)∣≤B(1+∣x∣)/t3\\|\partial_t\bar m_\theta(x,t)\\|\le B(1+\\|x\\|)/t^3, with fixed A,BA,B, the choice δ=ϵa+N−1/2\delta=\sqrt{\epsilon_a+N^{-1/2}} gives W2≲ϵv+ϵa+N−1/4W_2\lesssim\epsilon_v+\sqrt{\epsilon_a}+N^{-1/4}.

The constants are uniform over the network family, and the population error bounds are evaluated at the selected cutoffs. These conditions give quantitative guarantees that vanish as the approximation errors decrease and NN increases.

The logarithmic rate in the second case comes from the uniform Grönwall factor combined with the explicit regularity upper bounds. Benton et al. (2024, Theorem 1) obtain a related factor exp⁡(∫Lt,dt)\exp(\int L_t\\,dt) for deterministic flows. In the third case, the structure of the composite predictor gives a uniform bound on both continuous and discrete error propagation, yielding the polynomial rate. Early stopping for SDE samplers is analyzed in Chen et al. (2023, Section 3.2); endpoint smoothing and approximation error are combined in Benton et al. (2024, Section 3.3). Here the argument includes the additional fluctuations from stochastic expert routing.

Revision. (i) We add Corollary A.12 and its proof to the appendix, together with a remark on the balancing of δ\delta. (ii) We revise the sentence after Theorem 4.5, and the corresponding statements in the Discussion and Conclusion, to state the joint rate and point to the corollary.

W2 (Theory–experiment connection). “There is a clear disconnect between the theory and the experiments ...”

Response. We apologize for the unclear presentation. Table 1 already includes our DDIM+A implementation (“+ MoG-a net”), which evaluates the base posterior-routing sampler studied in our theory. We inadvertently omitted “ours” from this row and will correct the label. We emphasized +DA because it achieves lower FID than +A at every reported sampling budget on both datasets. We did it because we found its performance is better and worth discussion. This improvement is an additional empirical finding and that's why we put it in the appendix; the derivative network is not required by our convergence theorem. We will make these roles explicit and add the missing 'ours' in the second row of Table 1.

Revision. We add the missing ”ours” in the experiment part and clarify the difference between a and da network.

W3 (Assumptions on the learned network). “The assumptions are strong and imposed directly on the learned network ...”

Response. We agree that Assumptions 4.2 and 4.3 are imposed on the learned networks, and we should have explained this choice more clearly in the paper. These assumptions are milder than they may appear, for two reasons. First, they only ask the learned field to satisfy bounds of the same form as those satisfied by the true field (Lemma A.3). Second, the errors in Assumption 4.4 are directly tied to training: the flow-matching loss equals the L2L^2 error of Assumption 4.4 plus a constant independent of the network, so the empirical training loss is an unbiased estimate of it up to that constant. Stating guarantees in terms of such population errors is standard in this literature. We expand on both points below.

Why a condition on the learned field is needed. The sampler follows the learned field, while the L2L^2 errors in Assumption 4.4 control this field only on average under the true marginals. Without any regularity of the learned field, the L2L^2 error can be arbitrarily small while the sampling error is arbitrarily large. Benton et al. (2024) make exactly this point for flow matching. Analyses of deterministic samplers therefore need some derivative-level control of the learned field. One option is accuracy of its Jacobian (Li et al., 2024). The other is a Lipschitz condition on the learned field itself: Benton et al. (2024, Assumption 3) assume the learned velocity field is LtL_t-Lipschitz, and Chen et al. (2023, Assumption 3) assume the learned score is Lipschitz, a condition they single out as the new requirement of the ODE setting. We follow the second route.

What the condition asks for. The true composite field satisfies bounds of exactly the same form, with explicit constants Λ\*δ,Γ\*δ\Lambda_\*^\delta,\Gamma_\*^\delta (Lemma A.3). This is also the justification used by Benton et al. (2024), who choose their Lipschitz assumption because the true field satisfies it. Assumption 4.3 thus asks that the learned field have regularity of the same type as the field it approximates. It concerns only the composite field vθv_\theta, not each expert or the router separately. It requires no accuracy of derivatives: the learned networks enter the error terms only through their L2L^2 errors. Corollary A.12(b) makes this quantitative: if the learned constants are at most a constant multiple of these true-field upper bounds, the bound vanishes jointly as δ→0\delta\to0.

Link to finite-sample training. Stating guarantees in terms of population L2L^2 errors is the standard interface in convergence analyses of diffusion and flow samplers (Chen et al., 2023; Benton et al., 2024; Li et al., 2024). It separates the statistical and optimization error of training from the error of sampling. In our setting this interface is particularly direct. Each expert's term in ϵv2\epsilon_v^2 is its own flow-matching excess risk on its own data, because the flow-matching loss equals the L2L^2 error to the true field plus a constant. It is the quantity each expert minimizes in independent training, and changes in it are tracked by the expert's validation loss. Finite-sample rates for ϵv,ϵa\epsilon_v,\epsilon_a would follow from statistical analyses of score and flow-matching estimation; combining them with our bound is a natural next step, which we state as an open problem.

Revision. We expand the remark after Assumption 4.3 with the necessity argument and the references above, and we rewrite the paragraph on population errors in Section 6: the limitation is kept, but stated together with the role of the assumption and the link to the training objective.

W4 (Novelty and rate comparison). “The novelty over the existing ODE-based W2 analyses seems incremental ...”

Response. We agree that the backbone, a coupling argument combined with Grönwall's inequality, is standard. Our contribution is not a new backbone but a guarantee for a sampler that existing results do not cover: at each step, the decentralized sampler follows a single randomly routed expert, so it is not the discretization of any single ODE. Obtaining a vanishing bound for it requires the following ingredients, none of which are needed in single-field analyses.

  1. Single-field results cannot be applied as they stand. Both Grönwall-based W2W_2 bounds and density-ratio arguments for TV track one deterministic map. Here the map changes randomly from step to step, so the analysis must be carried out for the routed sampler itself.

  2. Worst-case perturbation analysis fails. One could view the sampler as the Euler scheme of the composite field plus a perturbation. The perturbation is of order h=1/Nh=1/N per step, so a worst-case bound accumulates to O(1)O(1) over NN steps and does not vanish. Treating the routing fluctuations as martingale differences, as the reviewer notes in Strength 2, yields the N−1/2N^{-1/2} term instead.

  3. Routing error enters linearly. The routing weights sum to one, so the drift shared by all experts is applied exactly, and only the differences between expert predictions are routed. As a result, the router's L2L^2 error enters the bound linearly, and no accuracy of the router's derivatives is needed.

  4. The error decomposes over experts. The approximation term is a weighted sum of each expert's error under its own component marginal. This matches independent training on local data and is specific to the decentralized setting.

Comparison with existing results. The table below summarizes the closest results.

WorkSamplerMetricCondition on learned fieldMultiple experts
Chen et al. (2023)SDETVL2L^2 score error onlyno
Benton et al. (2023)SDEKLL2L^2 score error onlyno
Chen et al. (2023)ODE + Langevin correctorTVLipschitz scoreno
Li et al. (2024)ODETVJacobian errorno
Benton et al. (2024)ODE, continuous timeW2W_2LtL_t-Lipschitz fieldno
Oursrouted ODE, EulerW2W_2Lipschitz composite fieldyes

For reference, Li et al. (2024) obtain $\mathrm{TV}\lesssim d/T+\sqrt d\,\epsilon_{\mathrm{score}} +d\,\epsilon_{\mathrm{Jacobi}}$ up to logarithmic factors, and Benton et al. (2024) obtain W2≤ϵexp⁡(∫Lt,dt)W_2\le\epsilon\exp(\int L_t\\,dt) for the continuous-time flow. For fixed δ\delta, our bound reads W2≲eΛ(ϵv+ϵa+N−1/2)W_2\lesssim e^{\Lambda}(\epsilon_v+\epsilon_a+N^{-1/2}).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.