acceptodds
Under review as a conference paper at ICLR 2027

Convergence Theory of Decentralized Diffusion Models via Joint-Space Gaussian Cancellation

Abstract

Diffusion probabilistic models (DPMs) have demonstrated success in generative modeling, motivating a growing theory of their convergence. Recent decentralized DPMs combine experts trained on separate data partitions through a learned router, raising an additional question about how routing affects sampling accuracy. We develop Joint-Space Gaussian Cancellation (JGC), a flexible convergence analysis pipeline for standard and decentralized DPMs with DDPM sampling. Our key insight is to analyze the Gaussian reverse bridge jointly with its noisy input, retaining posterior cancellations before bounding the error. An exact Gaussian-channel interpolation expresses the complete finite-step defect through a signed divergence; a posterior-covariance potential then controls its accumulation without losing an extra dimension factor. This yields an explicit total-variation discretization bound on equal log-SNR grids with polynomially moving SNR endpoints, merely under a finite second data moment. Here is the data dimension and the number of denoising steps. Extending the analysis classwise preserves this convergence order, while Gaussian overlap quantifies the additional routing error through expert separation without requiring a probability floor. These two results connect linear-dimensional convergence theory with the additional approximation cost of decentralized routing.

Primary area:
generative models
Keywords:
convergence theory, diffusion models, generative models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_J2UAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Diffusion probabilistic models (DPMs) have achieved significant advances in both theory and applications. However, the sampling errors of decentralized DPMs, particularly the additional errors introduced by routing decisions, remain to be further investigated. In this paper, it propose a Joint-space Gaussian Cancellation (JGC) method to analyze the convergence properties of both standard and decentralized DPMs within the framework of denoising DDPM samplers. Under the assumptions that the data distribution has a finite second moment, a specified log-SNR grid is employed, and a posterior-variance DDPM sampler is used, we establish that the total variation error of standard diffusion models can be controlled by initialization error, discretization error, and denoising network error. Furthermore, we extend this analysis to decentralized DPMs and derive an additional error bound associated with routing decisions.

Critical strengths

(1) The theoretical analysis in this paper is rigorous, and the proofs are clear, providing a detailed characterization of initialization error, discretization error, denoising network error, and routing error. Under the mild assumption that the data distribution has only a finite second moment, the paper establishes an upper bound on the error in total variation distance.

(2) The paper proposes a unified analytical framework that applies not only to standard DPMs but also extends to decentralized DPMs that combine multiple experts through a router, thereby providing explicit convergence guarantees for decentralized DPMs.

(3) The paper is well-written, with a clear and concise structure and a smooth, logical flow of presentation.

Critical weaknesses

(1) Limited practical applicability. Modern DPMs predominantly employ ODE-based sampling methods, including image generation models (e.g., FLUX, SD3.5, and Qwen-Image) and video generation models (e.g., Wan, CogVideoX, and HunyuanVideo). However, the convergence theorems in this paper focus on stochastic reverse sampling using DDPM with posterior variance. The derived error bounds are specific to its Gaussian reverse transitions and do not cover the sampling errors associated with ODE solvers. This limitation restricts the practical relevance of the theoretical results to modern DPMs.

(2) Insufficient experimental validation. The paper does not include any experiments in the main text. Moreover, the experimental results and conclusions presented in the appendix are not sufficiently robust. For instance, on the two-dimensional synthetic dataset, the experimental conclusions differ between 10-step and 50-step sampling, as shown in Table 5. Similarly, on CIFAR-10, the results for standard DDPM and SD-DDPM lead to inconsistent conclusions, as shown in Table 6.

(3) Lack of discussion of related work. The paper does not discuss exposure bias in DPMs, although prior studies on these topics have also investigated errors in sampling distributions. A more comprehensive discussion of these related works would help clarify the novelty and contributions of the proposed theoretical framework.

[1] Elucidating the SNR-t Bias of Diffusion Probabilistic Models, ICLR 2024

[2] On Error Propagation of Diffusion Models, ICLR 2024

[3] Frequency Regulation for Exposure Bias Mitigation in Diffusion Models, ACM MM 2025

Questions for authors

Please see the Weaknesses section.

2 comments
AuthorsOpenReview ↗

Rely to Reviewer J2UA (Block 1)

Dear Reviewer J2UA,

Thank you for your careful review and positive assessment of the analysis. We have clarified the scope, expanded the experimental documentation, and added the related-work discussion you suggested.

W1. Applicability to stochastic versus ODE sampling.

We agree that our theorems address stochastic posterior-variance DDPM sampling and do not cover ODE solvers. The contribution is a proof method that gives a comparable first-order, linear-dimensional rate for standard diffusion and extends to learned routing in decentralized diffusion. We discuss this scope and its relation to existing theory in Appendix P.

Studying stochastic and deterministic sampling separately is common because their transition laws call for different techniques. For example, Chen, Lee and Lu [1] analyze stochastic reverse discretizations, whereas Gao and Zhu [2] study probability-flow ODEs using contraction and synchronous coupling. JGC uses exact Gaussian-channel interpolation and a posterior-covariance potential; Gaussian overlap then quantifies the additional cost of routing between experts.

Our routing bound takes account of expert separation: confusing nearby predictions costs less than confusing distant ones. Appendix O.3 makes this concrete for fixed positive-variance Gaussian classes with exact experts. On a shrinking-noise grid, both routing coefficients remain bounded because the expert predictions approach each other at high SNR. We have summarized this example in the main text. We are also investigating decentralized ODE convergence in separate ongoing work.

W2. Experimental validation and the choice of metrics.

We have clarified the experiments' purpose and expanded their documentation: they compare generation quality, while the proofs and the new analytical example establish and explain the convergence guarantees. The experiments remain in Appendix J, consistent with the paper's theoretical focus, with a clearer pointer from the main text.

MMD and FID provide sample-based evaluations for synthetic and image data. Direct TV between empirical point clouds with disjoint supports equals one; estimating the underlying high-dimensional TV requires additional density or structural information. We therefore use these sample-quality metrics alongside an analytical Gaussian example in which the routing bounds can be calculated explicitly.

Appendix J now includes training and evaluation parameter tables (Tables 9-10), the precise MMD estimator, and the repeated-evaluation procedure. We have also organized the synthetic DDPM evaluation source, models, and documentation into an anonymous supplementary package to submit with the revision. Deployment launchers are excluded for commercial reasons; the research interfaces and parameter settings are documented.

References

[1] Hongrui Chen, Holden Lee, and Jianfeng Lu. "Improved Analysis of Score-based Generative Modeling: User-friendly Bounds under Minimal Smoothness Assumptions." International Conference on Machine Learning, pp. 4735-4763, 2023.

[2] Xuefeng Gao and Lingjiong Zhu. "Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances." International Conference on Artificial Intelligence and Statistics, pp. 1009-1017, 2025.

AuthorsOpenReview ↗

Rely to Reviewer J2UA (Block 2)

W2, continued. Different conclusions at 10/50 steps and across DDPM/SN-DDPM.

The relative ranking of Fixed-class and per-step routing changes with the step count and base sampler, while both strategies can improve on the unconditional baseline. We have revised Appendix J to state this distinction explicitly and explain how the two strategies relate to our theory.

On Circles, the 10-step MMD values are 121.0 (Uncond), 33.44 (Fixed-class), and 71.94 (per-step); at 50 steps they are 12.42, 4.764, and 2.594. Thus Fixed-class is better at 10 steps and per-step routing is better at 50 steps, with both improving on Uncond. The same reversal appears for Moons and Checkerboard. On CIFAR-10 with the linear schedule at 50 steps, Fixed-class/per-step FID is 15.12/13.81 for DDPM and 3.62/4.29 for SN-DDPM.

The two strategies target the same data mixture but use different finite-step processes. Fixed-class keeps the initial label; per-step routing updates the expert choice using the current sample. Appendix G gives the Fixed-class guarantee, and Theorem 3.2 accounts for the additional learned-routing error. These results identify the initialization, discretization, network, and routing budgets relevant to the choice. The tables then show how the overall generation quality varies across the reported settings.

W3. Relation to exposure bias and error propagation.

We have added a concise discussion of all three works in Appendix P, connecting them to our analysis through the common question of how sampling errors accumulate. Thank you for pointing out these connections.

Exposure-bias studies examine how generated trajectories depart from the forward-process marginals. Our comparison lemma connects errors measured under those true marginals to the actual generated endpoint law; JGC controls the finite-step discretization contribution, and the mixture comparison isolates routing error. This connects the distribution mismatch to the network, discretization, and routing terms in our bound.

Li and van der Schaar [3] relate modular and cumulative KL errors and motivate a training regularizer. Yu et al. [4] study sample-SNR/timestep mismatch. Yu and Zhan [5] study frequency-dependent energy discrepancies and a wavelet correction. Appendix P discusses how these approaches relate to our finite-step analysis.

Thank you again for the helpful suggestions.

References

[3] Yangming Li and Mihaela van der Schaar. "On Error Propagation of Diffusion Models." International Conference on Learning Representations, 2024.

[4] Meng Yu, Lei Sun, Jianhao Zeng, Xiangxiang Chu, and Kun Zhan. "Elucidating the SNR-t Bias of Diffusion Probabilistic Models." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 43461-43470, 2026.

[5] Meng Yu and Kun Zhan. "Frequency Regulation for Exposure Bias Mitigation in Diffusion Models." Proceedings of the 33rd ACM International Conference on Multimedia, 2025. doi:10.1145/3746027.3755608.

Reviewer_dVLCOpenReview ↗
Rating
1
Confidence
3(+1)
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proves an O(d log²N / N) total-variation bound on the discretization error of DDPM sampling. It uses the posterior-variance sampler, an equal log-SNR grid, and assumes only a finite second moment. The proof technique, "Joint-Space Gaussian Cancellation," interpolates through exact Gaussian channels and controls the finite-step error with a posterior-covariance potential that telescopes across the grid. The paper then extends the bound to decentralized diffusion models, where a learned router picks among class experts at each step, and bounds the extra routing error using Gaussian-mixture overlap.

Critical strengths

The core argument is elegant and appears correct; I checked the main accumulation proof (Appendix D) in detail. The constants are explicit, the step-size condition is dimension-free, and the log factor improves on Li & Yan's log³N. The paper is also candid about its scope and limitations.

Critical weaknesses
  • Unclear significance and no connection to practice. I am not sure what this theory adds to our understanding of diffusion models in practice.

    • Standard DDPM. The result reproves a known rate for a specific sampler (posterior-variance DDPM) on a specific grid (equal log-SNR spacing) that practitioners rarely use. It says nothing about the samplers, schedules, or step counts that work well in practice, such as DDIM, DPM-Solver, EDM schedules, or 20–50 steps.
    • Decentralized setting. The bounds do not explain or predict anything observable. They do not say when per-step routing beats a fixed-class sampler, how many experts to use, how to partition the data, or how router accuracy should trade off against expert accuracy.

    The authors should explain concretely what new insight or design guidance this analysis provides beyond existing O(d/T) theory.

  • The rate is not new. Li & Yan (ICLR 2025) already prove Õ(d/T) under a weaker assumption (finite first moment), and Jain & Zhang (2025) obtain a comparable result under a finite second moment. The contribution is a new proof with sharper constants, not a new rate, and the framing oversells it.

  • The role of the second moment is unclear. The truncation argument in Appendix D seems to use no moments, so the discretization bound may hold for any data distribution. The authors should say where the assumption is actually needed.

  • The setting is narrow. The result covers only the posterior-variance sampler on an equal log-SNR grid, not common noise schedules.

  • The experiments don't test the theory. They use a shared conditional network rather than separate experts, report MMD/FID rather than TV, and appear to be carried over from an earlier version.

  • The presentation is heavy. The paper runs 73 pages, and the key idea is buried in the appendix.

  • Over-hedged prose. The writing is repetitive and qualifies nearly every claim.

  • Boilerplate disclaimers in Appendix J. It repeatedly says "the retained records do not establish…" and "the retained records do not determine…", which suggests the text was revised around experiments the current authors could not fully document.

Questions for authors
  1. What concrete insight or design guidance does this analysis give practitioners beyond existing O(d/T) theory?
  2. Can the method adapt to intrinsic dimension, since the expected posterior-covariance trace is plausibly of order k for k-dimensional data?
11 comments
AuthorsOpenReview ↗

Rely to Reviewer dVLC (Block 1)

Dear Reviewer dVLC,

Thank you for your careful reading and checking the accumulation proof in Appendix D.

W1-W2 / Q1. What does the paper add beyond existing first-order rates?

Our main contribution is a new finite-step analysis, JGC, that preserves Gaussian posterior cancellation and controls the complete remainder through a covariance potential. It yields an explicit discretization bound and, together with our Gaussian-mixture stability analysis, a convergence guarantee for learned routing in decentralized diffusion.

For an equal log-SNR grid with span H=log⁡(η0/ηN)H=\log(\eta_0/\eta_N), our accumulated marginal-TV discretization bound is DN=d(9H+5)4(eH/N−1),\mathcal D_N=\frac{d(9H+5)}4\bigl(e^{H/N}-1\bigr), provided eH/N−1≤1e^{H/N}-1\le1. Thus H=O(log⁡N)H=O(\log N) gives O(dlog⁡2N/N)O(d\log^2N/N), with a mesh condition independent of dd. The key insight is to accumulate posterior covariance across noise levels before applying dimension bounds. Exact Gaussian-channel interpolation retains the full finite-step error, and the covariance potential controls its higher-order contribution through covariance energy and telescoping boundary terms (Section 4 and Appendix D).

Our second contribution is routing stability that depends on the geometry of the experts being confused. The classwise discretization budget is preserved, while Gaussian overlap and pairwise coupling quantify the additional error from a learned router, including zero predicted weights. This separates within-class uncertainty, network error, and routing geometry in the resulting guarantee.

Li and Yan [1] and Jain and Zhang [2] establish comparable linear-dimension, first-order rates. Our contribution lies in the finite-step mechanism above and the explicit decentralized guarantees it supports. The revised main text and Appendix O explain their implications for the following sampling choices:

  1. Partition choice. A useful partition reduces the uncertainty left within each class. The routed oracle preserves between-class posterior covariance and omits within-class covariance; Appendix L proves that nested refinement decreases the latter and its associated discretization budget. We now explain this mechanism in the main text.
  2. Router versus expert accuracy. Routing accuracy should be assessed together with the separation of the experts being confused. When ΔiDi(x)2\Delta_iD_i(x)^2 is uniformly small, Appendix O.4 shows that squared routing error is weighted by squared expert separation and the same SNR increment used for network error. Appendix O.2 sharpens this further by charging the affected expert pairs. Mistakes between well-separated experts therefore require tighter control.
  3. Fixed-class versus per-step routing. The two choices have different error budgets. The Fixed-class guarantee in Appendix G uses an average classwise initialization discrepancy and classwise network errors. Per-step routing uses the mixture initialization discrepancy and adds a geometry-dependent routing cost. The revised main text presents this comparison directly; the empirical tables show how their relative quality varies with the sampler and step count.
  4. Expert count. Increasing the number of experts changes within-class uncertainty, expert accuracy, and routing geometry. These are the quantities the theory identifies for balancing the choice. The classwise discretization bound itself has no explicit multiplier in the number of experts.
  5. Shrinking stopping noise. For fixed positive-variance Gaussian classes with exact experts, Appendix O.3 proves bounded routing coefficients on η0=N2\eta_0=N^2, ηN=N−2\eta_N=N^{-2}. Expert predictions approach one another at high SNR, compensating for the growing absolute SNR increments. The overlap budget has linear dependence on the maximum root-mean-square routing error, while coupling gives square-root dependence. The calculation uses the changing expert separation at each noise level.

References

[1] Gen Li and Yuling Yan. "O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions." Journal of Machine Learning Research, 26(292):1-55, 2025.

[2] Nishant Jain and Tong Zhang. "A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions." arXiv:2508.16306v1, 2025.

AuthorsOpenReview ↗

Rely to Reviewer dVLC (Block 2)

W1 / W4. Why this sampler and grid rather than ODE solvers or other schedules?

We study posterior-variance stochastic sampling because its Gaussian bridge structure supports a complete finite-step analysis, with equal log-SNR spacing enabling covariance accumulation. We have clarified these roles and added a discussion of stochastic and deterministic convergence theory in Appendix P.

Separate analyses are common: Chen, Lee and Lu [3] study stochastic reverse discretizations, whereas Gao and Zhu [4] study probability-flow ODEs using contraction and synchronous coupling. Their different dynamics require different stability arguments. Our theorem gives an explicit finite-grid guarantee for the specified stochastic sampler; Appendix J separately reports sample quality at practical step counts, including 20-50-step image evaluations. We are pursuing decentralized ODE convergence in separate ongoing work.

W3. Where is the second moment needed?

The second moment supports the explicit initialization and clean-data error estimates in our overall analysis; the oracle discretization bound itself does not require it. We have made this distinction explicit and strengthened the oracle result in Appendix O.1.

Your observation about truncation is correct. The same discretization bound holds for every Borel data law on equal log-SNR grids with positive finite SNR endpoints and the original mesh condition. Table 11 maps the moment assumptions to the individual conclusions, including the moment-sensitive interval refinement and learning-error sufficient conditions. This also clarifies the comparison with Li and Yan's first-moment results [1]: our original assumption was stronger than needed for the JGC discretization component.

Q2. Can JGC adapt to intrinsic dimension?

Intrinsic-dimension adaptation is a natural extension of JGC, and we have added it to the future-work discussion. A general result requires a dedicated analysis of data geometry across the noise interval.

The expected posterior-covariance trace is where dimension enters our proof. A general intrinsic-dimension bound requires geometric assumptions that control this trace throughout the SNR range. We plan to study this extension separately, building on the general-data argument and decentralized routing framework developed here.

References

[1] Gen Li and Yuling Yan. "O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions." Journal of Machine Learning Research, 26(292):1-55, 2025.

[3] Hongrui Chen, Holden Lee, and Jianfeng Lu. "Improved Analysis of Score-based Generative Modeling: User-friendly Bounds under Minimal Smoothness Assumptions." International Conference on Machine Learning, pp. 4735-4763, 2023.

[4] Xuefeng Gao and Lingjiong Zhu. "Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances." International Conference on Artificial Intelligence and Statistics, pp. 1009-1017, 2025.

AuthorsOpenReview ↗

Rely to Reviewer dVLC (Block 3)

W5. Shared experts, MMD/FID, and experiments carried over from earlier work.

The shared conditional network is a valid instance of our mathematical model, and the experiments provide complementary sample-quality comparisons. We have clarified these two roles in Appendix J and documented the existing experiments more precisely.

Each conditional output defines an expert function yθl(x,t)=yθ(x,t,l)y_\theta^l(x,t)=y_\theta(x,t,l). The theorems control these functions and their approximation errors, so parameter sharing is compatible with the analysis. The experiments evaluate this sampling mechanism; separate-site training and deployment concern its system implementation.

MMD and FID evaluate sample quality on synthetic and image data. TV concerns the underlying distributions: directly comparing disjoint empirical point clouds gives TV equal to one, so useful high-dimensional TV estimation requires additional modeling. We establish the convergence bounds by proof and use the Gaussian example to exhibit their routing dependence analytically. The reported numerical results are the existing experiments, now accompanied by clearer protocol and evaluation descriptions.

W6-W7. Length, accessibility of the main idea, and repetitive qualifications.

We have brought the proof intuition and design implications forward, and consolidated the repeated scope qualifications. The main text now explains both the JGC mechanism and the consequences for decentralized sampling.

Section 4 explains posterior cancellation and covariance accumulation. Section 5 now summarizes the Gaussian example, the partition/routing trade-off, and the Fixed-class comparison. Detailed derivations remain in the appendices. Appendix K collects the scope discussion, and Appendix J replaces repetitive disclaimers with concrete experimental descriptions.

W8. Experimental documentation and repeated references to retained records.

We agree that the experimental materials were poorly organized. Appendix J.6 now provides concrete parameter and evaluation details, and we have prepared a supplementary source package for inspection.

Tables 9-10 document the original image-training and FID settings and the synthetic evaluator's parameters. We also give the precise squared-MMD estimator and explain the repetition procedure: the eight evaluation statistics share a generated sample pool. The revised description states exactly how these measurements were computed.

The supplementary package contains the synthetic DDPM evaluation source, model files, data, and documentation, which we will submit with the revision. Deployment launch scripts are excluded for commercial reasons; research interfaces and parameter settings are documented.

Thank you for the detailed feedback. We hope these revisions address your concerns about the contribution and presentation.

Reviewer_ocZdOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Posterior-variance DDPM sampling is analyzed through an exact Gaussian-channel interpolation and a posterior-covariance potential. A logarithmically growing span gives an O(dlog⁡2N/N)O(d\log^2 N/N) discretization term. Categorical routing preserves the classwise bound, while Gaussian-mixture comparisons account for learned routing errors. Positive-noise guarantees in total variation are separated from convergence to clean data in bounded-Lipschitz distance.

Critical strengths
  • Appendix D controls the complete finite-step defect through exact-channel interpolation. Lemma D.3 uses symmetry of the third-moment tensor in the resolvent-potential argument, and Section D.4 accumulates the estimates through overlapping intervals. Together, these steps give linear dimension dependence without a dimension-dependent mesh restriction. No uncontrolled local-expansion remainder is left in the accumulated estimate.
  • Specifying the transition law and target distribution prevents several possible misinterpretations of the rate. Posterior bridge variance differs from other DDPM variance choices, and categorical expert selection differs from averaging predictions. Gaussian examples in Appendix C distinguish conditional from marginal error and show why interval dependence cannot simply be omitted.
  • Routing bounds respond to expert geometry. Perturbing weights among nearby experts can have a small effect on the transition distribution, which a probability-only bound would miss. Lemmas E.2 and E.3 avoid a probability floor, and Proposition E.7 gives quadratic sensitivity to small routing perturbations. A complementary coupling bound covers a different regime.
  • Population accuracy is separated from what training actually guarantees. Appendices F and H distinguish consistency from retaining the oracle rate on moving grids. An averaged continuous-time training loss alone does not certify errors at prescribed sampling nodes. Stating this distinction makes the learned-sampler guarantee interpretable.
Critical weaknesses

Quantitative routing guarantees can be much more conservative than the discretization estimate. Equation (10) uses the maximum expert diameter, so even experts unrelated to a routing perturbation can increase the bound, as Appendix E.5 acknowledges. On the shrinking-noise schedule η0=N2\eta_0=N^2, the largest absolute SNR increment grows as 4Nlog⁡N4N\log N, and Corollary 3.3's linear routing-error specialization can acquire a rapidly growing coefficient. Coupling avoids this exponential coefficient at the cost of weaker small-error dependence. These trade-offs limit how readily the theorem guides partition or router design. A worked example comparing both bounds on the same moving grid would show when each gives an informative guarantee.

Questions for authors
  1. Does Section D.5 require a finite second moment for the oracle marginal-defect estimate itself, or only for other parts of the analysis? Locally uniform posterior convergence and the displayed discretization constant appear independent of the moment value.
  2. Using the existing Gaussian-class formulas in Appendix L, how do the overlap and coupling routing bounds compare as stopping noise shrinks? Which term is informative in that example under the moving-grid schedule?
2 comments
AuthorsOpenReview ↗

Rely to Reviewer ocZd (Block 1)

Dear Reviewer ocZd,

Thank you for your careful reading. We have added a tighter routing comparison and clarified the moment assumptions in Appendix O, with the main conclusions summarized in the text.

W1. A maximum diameter can charge experts unrelated to a routing perturbation.

We agree that the maximum-diameter estimate can be overly conservative. We have added a sharper coupling bound that accounts for the expert pairs involved in the routing perturbation, so an unchanged distant expert need not increase the routing cost. Thank you for pointing this out.

The idea is to keep unchanged probability mass on the same expert and account for the distances over which the remaining mass must move. Appendix O.2 formalizes this through a coupling between the true weights aa and learned weights bb. With learned expert predictions hlh_l, define

Ci(x)=min⁡γ∈Π(a,b)∑l,jγlj∥hl−hj∥2.C_i(x)=\min_{\gamma\in\Pi(a,b)}\sum_{l,j}\gamma_{lj}\|h_l-h_j\|^2.

Here Π(a,b)\Pi(a,b) is the set of label couplings with marginals a,ba,b. Gaussian comparison and Minkowski's inequality then give, for finite displayed energies,

\le\frac{\Delta_i}{2}\left(E_i+\sqrt{\mathbb E_{p_i}C_i(X)}\right)^2.$$ Since $C_i(x)\le D_i(x)^2\|a-b\|_1/2$, this is at least as sharp as the original diameter-based coupling bound. In particular, a perturbation between two nearby experts can be controlled by their separation even when other experts are far away. Proposition O.2 gives the full proof. > **Q1. Does the oracle discretization estimate need a finite second moment?** The second-moment assumption supports the explicit initialization and clean-data error estimates in our overall analysis; the oracle discretization bound itself does not need it. We have now made this distinction explicit in Appendix O.1 and Table 11. Our framework combines discretization with initialization, network, and routing errors, and then relates the noisy endpoint to clean data. The common second-moment assumption supplies the quantitative initialization and clean-data smoothing estimates, as well as the moment-sensitive interval refinement. This explains its use in the overall presentation. Your observation lets us state the discretization result more generally: Proposition O.1 proves the same accumulated JGC bound, with its constant unchanged, for **every Borel data distribution** on equal log-SNR grids with positive finite SNR endpoints and the original mesh condition. The proof defines the oracle through Gaussian Bayes integrals and passes from truncated data laws to the original law using posterior convergence and lower semicontinuity of TV. Table 11 identifies precisely which conclusions use a data moment and which do not.
AuthorsOpenReview ↗

Rely to Reviewer ocZd (Block 2)

W2 / Q2. Compare overlap and coupling on the same shrinking-noise grid.

We have proved the comparison on the grid you suggested: for fixed positive-variance Gaussian classes with exact experts, both routing bounds have coefficients that remain bounded as stopping noise vanishes. The overlap bound also gives stronger small-error scaling. This illustrates the value of the time-dependent expert geometry in Theorem 3.2, even when the uniform-diameter specialization in Corollary 3.3 has a growing coefficient.

The reason is that at high SNR, the experts' posterior predictions move closer together, precisely where the absolute SNR increments grow. The routing cost depends on their product, ΔiDi2\Delta_iD_i^2. Taking separate worst-case values for the increment and diameter loses this compensation.

Appendix O.3 works out the equal log-SNR grid η0=N2\eta_0=N^2, ηN=N−2\eta_N=N^{-2} for Gaussian classes with fixed means and common variance τ2>0\tau^2>0. Although Δmax⁡∼4Nlog⁡N\Delta_{\max}\sim4N\log N, we prove that the finite sum ∑iΔiDi2\sum_i\Delta_iD_i^2 stays bounded. If W‾N\overline W_N is the largest root-mean-square routing error across the sampling nodes, the resulting TV routing budgets satisfy

R_{\rm cp}=O(\sqrt{\overline W_N}),$$ with constants depending on the fixed class geometry, rather than on $N$. Thus small routing errors give a tighter scaling through overlap, and both budgets vanish when routing accuracy improves so that $\overline W_N\to0$. Proposition O.3 contains the explicit constants and the finite-grid accumulation proof. The example shows how retaining expert separation at each noise level keeps the routing bound informative as stopping noise decreases. > **W3. Implications for partition and router design.** The analysis identifies two quantities to balance: uncertainty within each partition and the geometry-weighted cost of routing mistakes between experts. We now state this interpretation explicitly in the main text and Appendix O.4. Appendix L shows that refining a nested partition reduces within-class posterior uncertainty and its associated discretization budget. The routing analysis then quantifies the cost of confusing the resulting experts: nearby predictions tolerate more probability error than widely separated predictions. A partition should therefore be assessed through both its classwise sampling errors and the routing accuracy needed to combine its experts. The pairwise refinement in Appendix O.2 makes the second part more precise by focusing on the affected expert pairs. Thank you again for these constructive suggestions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear Reviewers, AC, SAC, and PCs.

Thank you for the careful reviews and constructive questions. We have revised the manuscript and prepared detailed responses to each reviewer.

Our two contributions are Joint-Space Gaussian Cancellation (JGC) and the convergence analysis of learned routing in decentralized diffusion. JGC gives a comparable first-order, linear-dimensional rate for standard posterior-variance DDPM sampling through exact Gaussian-channel interpolation and covariance accumulation. Its classwise extension preserves the rate, while Gaussian-mixture overlap measures routing error through expert geometry. The main text now explains this proof strategy and its consequences for partition and routing choices.

The main changes are:

  • Moment assumptions (Appendix O.1 and Table 11). The discretization bound extends to every Borel data law; the table separates it from moment-dependent initialization and clean-data estimates.
  • Pairwise routing bound (Appendix O.2). A coupling cost measures confusion between the affected experts instead of using a global maximum diameter.
  • Gaussian example on a shrinking-noise grid (Appendix O.3). For fixed positive-variance Gaussian classes with exact experts, overlap and coupling have bounded coefficients on the same shrinking-noise schedule and give O(W)O(W) and O(W)O(\sqrt{W}) contributions for routing-error scale WW. The calculation retains the change in expert separation across the grid.
  • Clearer interpretation and experimental documentation. The main text explains partition and routing trade-offs; Appendix J provides additional parameter tables, the precise MMD estimator, and evaluation details. Appendix P briefly connects the analysis to stochastic/ODE convergence and exposure-bias research.

We have also prepared an anonymous supplementary package containing the synthetic DDPM evaluation source, model files, and documentation, for submission with the revised PDF. Deployment launchers are excluded for commercial reasons. The existing experimental values are unchanged.

Thank you for the opportunity to clarify and improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.