acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Functional Diffusion Models with Global and Local Statistics

Abstract

Function generation is a promising and challenging problem in generative modeling. Many scientific and spatial data are more naturally viewed as random functions or fields, where the goal is to generate an entire function that can be evaluated at any locations and resolutions. Functional diffusion models provide a natural framework for this setting, but a central difficulty is that the Bayes optimal denoiser is conditioned on an infinite dimensional noisy function. Existing approaches typically rely either on finite dimensional latent representations, which may discard important predictive information, or on a large number of sampled function values, which can be computationally expensive. We propose a hierarchical functional diffusion model that addresses this tradeoff by separating global and local predictive structure. A moderate dimensional diffusion model first generates a global state that captures long-range dependence, after which a conditional functional diffusion model generates the residual using compressed local information around each query. We develop an information based error decomposition that separates discretization, localization, compression, and learning errors, and derive Wasserstein bounds for the resulting functional distribution. Our algorithm is particularly well suited to hierarchical Gaussian process mixture models, which allow globally non-Gaussian and multimodal distributions while requiring Gaussian structure only conditionally at the residual scale. The proposed framework provides a scalable, coarse-to-fine, and resolution free approach to generative modeling on function spaces.

Primary area:
generative models
Keywords:
Diffusion Models, Conditional Generation, Functional Data, Localized Denoising

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LJKmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a hierarchical diffusion framework for function generation. A linear encoder decomposes each function as F0=RA0+U0F_0 = RA_0 + U_0, where A0A_0 represents global structure and U0U_0 is the residual. A finite-dimensional diffusion model first generates the global state, followed by a conditional residual diffusion model that uses the noisy query value and fixed-dimensional local polynomial statistics. The theoretical analysis decomposes the residual denoising error relative to the Bayes predictor into discretization, localization, compression, and learning errors, and derives conditional and end-to-end Wasserstein bounds under stated assumptions. Evaluation includes a hierarchical Gaussian process mixture benchmark with analytically available Bayes denoisers, CIFAR-10 image generation, and signed distance fields derived from MNIST and Fashion-MNIST. These experiments examine information loss from localization and compression, the influence of global-stage errors, and the accuracy and model-size trade-offs between local and full-context residual denoisers. The image experiments also consider variants with learned neighborhood features.

Critical strengths
  1. A useful framework for understanding information loss in functional denoising. The hierarchical formulation distinguishes uncertainty about global structure from residual dependence after conditioning on the global state. Theorem 1 then gives an exact decomposition into discretization, localization, compression, and learning errors, without requiring conditional Gaussianity for this identity. This is consequential because it separates limitations of the available information from limitations of the learned predictor, providing an interpretable framework for evaluating local representations (Section 4.1).

  2. Controlled experiments that directly examine the proposed mechanism. The Gaussian process mixture benchmark provides analytically available Bayes denoisers under different information sets, allowing localization and compression losses to be measured separately from neural-network learning error. Its nonstationary and anisotropic settings make this a useful diagnostic of when local compression works (Section 5.1; Figures 1–2). The signed-distance-field experiments provide complementary practical evidence: on the 28×2828 \times 28 datasets, the local residual model uses 0.12M parameters versus 2.5M for the full-context baseline and achieves lower held-out denoising errors at the reported noise levels (Table 5). Together, these results support the usefulness of compact local conditioning.

Critical weaknesses
  1. The end-to-end bound has materially different discretization dependence in the main text and the proof. Theorem 10 gives localization and compression terms of r(d−1)/2e−r/(2lc)r^{(d-1)/2}e^{-r/(2l_c)} and rk+d/2r^{k+d/2}, whereas Equation (39) in Appendix D.10 multiplies these by m/λmin⁡(CU)\sqrt{m/\lambda_{\min}(C_U)} and m\sqrt{m}, respectively. Although the main text refers to the appendix for the precise form, these factors cannot be absorbed into constants when studying refinement in the context size mm. This discrepancy is consequential because the end-to-end guarantee is a central contribution, and the omitted dependence changes what the bound establishes as the discretization is refined. The main statement should explicitly retain this dependence or specify that the design is fixed.

  2. The image experiments do not establish the claimed reduction in global-state sensitivity. Section 5.2, finding (i), interprets the reduction of FID(gen) minus FID(oracle) from 27.4 to 18.5 as a smaller Lgen(θ)L_{\mathrm{gen}}(\theta). However, Theorem 4 is an upper bound involving function-space Wasserstein distances; the difference between two FID values does not identify this conditional Lipschitz constant. Table 2 illustrates the ambiguity: adding noise at sampling reduces the gap from 18.5 to 11.6 while worsening gen/oracle FID from 30.8/12.3 to 48.8/37.1, as the authors acknowledge. Thus, the reported diagnostic does not substantiate the claimed stability mechanism. This limits the strength of the theory-to-experiment connection, which is the stated purpose of the image experiments.

Questions for authors
  1. Is Theorem 10 intended only for a fixed context design, or does it describe how the error behaves as the context size mm increases? Please reconcile its localization and compression terms with the additional factors m/λmin⁡(CU)\sqrt{m/\lambda_{\min}(C_U)} and m\sqrt{m} in Equation (39). Which quantities are held fixed, and what conclusion about discretization refinement follows from the bound with all dependencies retained? This would clarify the scope of the main theoretical guarantee.

  2. What justifies the inference in Section 5.2(i) that a smaller FID(gen) minus FID(oracle) implies a smaller Lgen(θ)L_{\mathrm{gen}}(\theta)? In particular, how does this inference account for Table 2, where sampling-time perturbations reduce the gap while worsening both FIDs? Is there evidence directly measuring the conditional sampler's sensitivity to changes in the global state, beyond the FID gap? This would help determine whether the experiments validate the proposed stability mechanism or establish an improvement in sample quality without identifying that mechanism.

  3. In Appendix B.3, do the 28×2828 \times 28 and 64×6464 \times 64 results use the same trained residual-network weights, or separately trained models with the same parameter count? Please clarify whether the demonstrated resolution independence concerns architecture size, transfer of a trained sampler across grids, or both. If the same weights are reused, how are the global coefficients, local window scales, and residual noise defined consistently across resolutions?

4 comments

We sincerely thank Reviewer LJKm for the thoughtful review and the precise questions on the dependence of Theorem 10 on mm and on what the FID gap can identify. We are glad that the reviewer finds the information-loss framework useful for understanding functional denoising, and values the controlled experiments that examine the proposed mechanism directly.

Following your suggestions, we now measure the sensitivity Lgen(θ)L_{\mathrm{gen}(\theta)} directly and test resolution transfer with a single trained model, marked in blue in the revised PDF.

W2 / Q2 (FID gap and Lgen(θ)L_{\mathrm{gen}(\theta)}): A smaller FID gap does not identify a smaller Lgen(θ)L_{\mathrm{gen}(\theta)}, and Table 2 shows sampling noise lowering the gap while worsening both FIDs.

Response: We agree. Theorem 4 bounds the Wasserstein distance between the generated and true function distributions, and a difference of FIDs cannot identify a conditional Lipschitz constant. Following the reviewer's suggestion, we now estimate the sampler's sensitivity Lgen(θ)L_{\mathrm{gen}(\theta)} to the global state directly with a coupling probe L^\hat L (Section 5.2, Appendix B.2, Tables 2 and 4).

  • Design. The proof of Proposition 3 bounds the synchronous coupling: two residual runs with the same terminal noise and different global states. We run exactly this coupling on 1,000 held-out images, with A0A_0 and A0+δA_0+\delta, δ=ε⋅sd(A0)⊙ξ\delta=\varepsilon\cdot\mathrm{sd}(A_0)\odot\xi, ξ∼N(0,I)\xi\sim\mathcal N(0,I), and report L^=(E∥U^0(A0+δ)−U^0(A0)∥2/E∥Rδ∥2)1/2\hat L=\big(\mathbb E\Vert\hat U_0(A_0+\delta)-\hat U_0(A_0)\Vert^2/\mathbb E\Vert\mathcal R\delta\Vert^2\big)^{1/2} at ε=0.05\varepsilon=0.05, which is a local, average-case estimate. Since R\mathcal R is orthogonal, the factor of Theorem 4 is 1+Lgen(θ)1+L_{\mathrm{gen}(\theta)}.
  • Results.
    1. Conditioning Augmentation lowers Lgen(θ)L_{\mathrm{gen}(\theta)}. With the same 200 extra epochs, L^=5.9\hat L=5.9 without augmentation and 3.13.1 with it, so the estimated factor 1+L^1+\hat L falls from about 6.96.9 to 4.14.1. This now supports finding (i) directly.
    2. The probe distinguishes cases that FID gaps cannot. The local r=8r=8 trunk has a smaller FID gap than the full-context one (13.0 vs 17.3) but a slightly larger L^\hat L (0.92 vs 0.79). Trained longer, the full-context network's FID worsens while L^\hat L falls slightly (0.79 to 0.69), so the loss is in the residual error, not in Lgen(θ)L_{\mathrm{gen}(\theta)}.
    3. Adding noise to the global state during sampling narrows the FID gap without lowering Lgen(θ)L_{\mathrm{gen}(\theta)}. With σs>0\sigma_s>0 in Table 2, the network is unchanged, and the gap narrows only because the oracle FID rises faster than the generated one. We now use FID gaps only as a qualitative ranking, and quantify the sensitivity with L^\hat L.
    4. Magnitude. Networks trained from scratch with augmentation have L^\hat L between 0.7 and 1.0, so the estimated factor 1+L^1+\hat L is below 2 for a well-trained model.

Q3 (same weights for 28228^2 and 64264^2?): It is unclear whether the 28228^2 and 64264^2 SDF results use the same weights, and how the global coefficients, window and noise are defined across resolutions.

Response: In the submitted version they did not. The 64264^2 model was trained separately with the same parameter count. The revised experiment now uses one set of weights, trained on 28228^2 only and used on every grid without retraining. The updated SDF results are in Section 5.3, Appendix B.3, Table 5 and Figures 13–14.

  • Design. Each SDF is computed once on a fine 4492449^2 grid, so its values at 28228^2, 56256^2 and 1122112^2 resolution are samples of the same function. The global coefficients are the first 64 modes of an orthonormal cosine basis on [0,1]2[0,1]^2, computed as integrals of the function, so they do not depend on the resolution. The window (∥x−q∥∞≤3/28\Vert x-q\rVert_\infty\le 3/28) and bandwidth (1.5/281.5/28) are in physical units, and the residual noise is a trace-class Gaussian field with λk∝(1+∣k∣2/k02)−2\lambda_k\propto(1+|k|^2/k_0^2)^{-2}, k0=8k_0=8, so one noise draw is a function that can be evaluated at any point, and the same draw is used at every resolution. We train on 28228^2, with half of the training queries at uniform random off-grid points and half on the grid, as Theorem 2 suggests.
  • Results.
    1. The denoising error does not change with resolution. On 56256^2, 1122112^2 and 4,096 scattered points, the held-out U0U_0-MSE over the residual variance stays within 2.4% of its 28228^2 value for t≤0.5t\le0.5. On MNIST, it is also within 3% of a model trained directly on 56256^2.
    2. Sample quality is kept, and shapes are consistent across resolutions. Samples generated on 1122112^2 keep their eikonal error, and the regions inside their zero level sets match those of the same samples generated on 28228^2 (IoU 0.97).

We address W1 / Q1 (Theorem 10 and its dependence on mm) in a separate comment.

W1 / Q1 (Theorem 10 and its dependence on mm): The main text Theorem 10 omits the factors (m/λmin⁡(CU))1/2(m/\lambda_{\min}(C_U))^{1/2} and m\sqrt m of Equation (39). Is it intended only for a fixed design, which quantities are held fixed and what follows under refinement?

Response: Theorem 10 is a bound for a fixed design XmX_m. Following your suggestion, the current revision specifies this in the statement ("a fixed quasi-uniform design XmX_m"), and the paragraph after it (in blue) states the two factors of (39) and explains where they come from. The new Appendix E adds a bound that does not grow with mm. Both bounds are meaningful, since Theorem 10 does not need the kernel condition of Appendix E.

  • What is held fixed for Theorem 10. The design XmX_m, the radius r=hr=h, the degree pp and the stopping time ϵ\epsilon are fixed (Appendix D.10). The theorem describes the error as the number of training functions NN grows: the three statistical terms tend to zero, and what remains is the approximation error of this design and the early-stopping error.

  • Reconciling Theorem 10 with (39). In (39) the localization term is (m/λmin⁡(CU))1/2r(d−1)/2e−r/(2lc)(m/\lambda_{\min}(C_U))^{1/2}r^{(d-1)/2}e^{-r/(2l_c)} and the compression term is m1/2rk+d/2m^{1/2}r^{k+d/2}. In Theorem 10 the two factors are constants for a fixed design, so the main statement shows only the dependence on rr. As you note, they cannot be absorbed when mm varies, so the text after Theorem 10 now states them. Both errors are energy norms of kriging weights: w⊤S−1ww^\top S^{-1}w for localization (Theorem 5) and min⁡φ∥wN−Whφ∥ΣN∣q2\min_\varphi\Vert w_N-W_h\varphi\Vert_{\Sigma_{N|q}}^2 for compression (Proposition 8). The proofs of Theorem 6 and Proposition 9 bound them through the Euclidean norm ∥w∥22\Vert w\Vert_2^2, which brings in mm and 1/λmin⁡(CU)1/\lambda_{\min}(C_U).

  • What follows under refinement. With all dependencies retained, (39) grows with mm, so it gives no rate as the design is refined. This growth is in the bound, not in the error.

    1. The approximation error is bounded uniformly in the design. By Theorem 1, the discretization, localization and compression errors sum to E[dBayes∗−dlocal∗]2≤σU2:=sup⁡qE[Var(U0(q)∣A0)]E[d_{Bayes}^{\ast}-d_{local}^{\ast}]^2\le\sigma_U^2:=\sup_q E[Var(U_0(q)\mid A_0)] for every design. So the three approximation terms of (39) can be replaced by σU\sigma_U, and the bound stays bounded as m→∞m\to\infty (new Proposition 14, Appendix D.10).
    2. Appendix E gives a bound that does not grow with mm. Under refinement nN∥wN∥22n_N\Vert w_N\Vert_2^2 grows, while the energy norm of the kriging weights and compression error Δcomp\Delta_{\mathrm{comp}} saturate (Figure 6). The energy norm is a variance reduction, bounded uniformly in the design (Lemma 2), and Appendix E works in that norm. It assumes that the Bayes denoiser is an integral of UtU_t against a square-integrable kernel ζ\zeta, which holds for stationary Matérn fields with suitable smoothness orders (Proposition 15) and for our finite-rank benchmark (Section 5.1). The three approximation terms then become ω(hN)+τ(r)+rk+d/2\omega(h_N)+\tau(r)+r^{k+d/2} (Proposition 17): a quadrature error that vanishes under refinement, the tail of ζ\zeta outside the window and a compression error without the factor m\sqrt m. So at fixed rr and pp the bound converges as m→∞m\to\infty to a floor set by the window and the degree. This floor is the price of computational efficiency: the denoiser sees only a (Dp+1)(D_p+1)-dimensional summary of an infinite-dimensional UtU_t, so the input dimension of the network and its cost per query do not grow with mm, and a finite summary loses some information in general. In Figure 1 each approximation term is at most a few percent of the residual variance.
    3. The measured error does not grow either. Without retraining, the denoising error for t≤0.5t\le0.5 stays within 2.4% from 28228^2 to 1122112^2 (Table 5).

Together with our first comment, this responds to every weakness and question in your review, and we hope it addresses your concerns. We thank you again for your careful reading and valuable suggestions.

Thank you for the clarification. I will maintain my rating of 3 (weak acceptance).

Thank you for your acknowledgement and for the time you spent on our paper. We appreciate your support and constructive suggestions.

Reviewer_wtEzOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes a hierarchical functional diffusion models by combining a compact global representation and a conditional residual using local information at each query location. It evaluates the proposed approach analyzing errors from discretization, localization, compression and learning and evaluates the approach on synthetic Gaussian-process mixtures, CIFAR-10 and signed-distance fields.

Critical strengths
  • The paper presents a well-suited domain decomposition. Solving optimization/training problems in a coarse-to-fine is one of the oldest and most reliable tricks in the book, and this paper is another instance of successfully applying this idea.
  • The paper provides a structured error analysis, distinguishing from discretization, localization, compression and learning errors. This is facilitated by the proposed Gaussian process mixture models benchmark, which provides a closed-form reference for denoisers.
  • The paper shows informative ablations and experiments are thorough.
Critical weaknesses
  • The mathematical formulation is dense, and it obscures the central idea and its practical value. A concrete motivating application and a clearer distinction between the general framework and results requiring conditional Gaussianity would improve accessibility.
  • Perhaps, the paper could also benefit from a rewriting to help less familiar readers. The idea could be presented in a simpler setting, while the dense and strict mathematical formalism could be simplified. Proofs could be potentially moved to the Appendix.
  • There's also limited evidence for resolution independence. The fixed-grid CIFAR experiments seem to not test this property, while the higher-resolution distance-field experiment uses resampled data on fixed grids. Evaluating one trained model at unseen resolutions and irregular query locations would provide stronger evidence of the paper's claims.
Questions for authors
  • Can a single trained model generate consistent samples at unseen resolutions or irregular query locations without retraining?
1 comment

We sincerely thank Reviewer wtEz for the careful reading and constructive feedback, in particular the suggestions on improving the presentation of the paper. We also appreciate your acknowledgment of the global-local framework and its error analysis, the closed-form Gaussian process mixture benchmark, and the thoroughness of the experiments. In response, we have added a resolution-transfer experiment on signed distance fields and a Navier–Stokes experiment, marked in blue in the revised PDF.

W3 / Q1 (unseen resolutions and irregular query locations): The evidence for resolution independence is limited, since the CIFAR-10 experiments do not test it and the higher-resolution signed distance field experiment uses resampled data on fixed grids. Can a single trained model generate consistent samples at unseen resolutions and irregular query locations without retraining?

Response: We agree that the CIFAR-10 experiments are mainly diagnostic, and that the submitted SDF experiment did not test resolution independence directly, since it retrained the model at 64264^2. We have therefore revised the SDF experiment: our model is now trained on 28228^2 only and evaluated at unseen resolutions and irregular query locations without retraining (Section 5.3, Appendix B.3, Table 5 and Figures 13–14).

  • Design. Each SDF is computed once on a fine 4492449^2 grid, so its values at 28228^2, 56256^2 and 1122112^2 resolution are samples of the same function. The global state is 64 cosine coefficients computed as integrals of the function, so it does not depend on the resolution. The residual noise is a trace-class Gaussian field, so one noise draw is a function that can be evaluated anywhere. We train the local denoiser on 28228^2, with half of the training queries at uniform random off-grid points and half on the grid, as Theorem 2 suggests. We then (i) measure its denoising error on 56256^2, 1122112^2 and 4,096 random off-grid points, (ii) compare it with the same model trained on 56256^2, and (iii) generate samples from the same global state and noise on 28228^2 and 1122112^2 and compare their shapes.
  • Results.
    1. Unseen resolutions. On both datasets, the held-out U0U_0-MSE on 56256^2 and 1122112^2 stays within 2.4% of its 28228^2 value for t≤0.5t\le0.5. On MNIST, it is also within 3% of a model trained directly on 56256^2 (Table 5).
    2. Irregular query locations. At 4,096 random off-grid points, the denoising error is the same as on the 28228^2 training grid (MNIST, t=0.1t=0.1: 0.0171 vs 0.0168), and the model can generate fields directly at these irregular points (Figure 14).
    3. Consistency across resolutions. The regions inside the zero level sets on 1122112^2 match those on 28228^2 (IoU 0.97, Figure 13), and sample quality is kept. In comparison, samples from monolithic Fourier neural operator (FNO) diffusion baselines change 2–4× more between resolutions (Table 6).
    4. Off-grid training. Training on grid queries only makes the model 18–42% worse off the training grid at t=0.1t=0.1 (Table 5, bottom), as Theorem 2 predicts.

Together, these results show that a single model trained on 28228^2 generates consistent samples at unseen resolutions and irregular query locations without retraining.

We repeated the test on real functional data, Navier–Stokes vorticity (Appendix B.4, Table 8, Figure 15). Trained on 64264^2 only, the model keeps its denoising error on 1282128^2, 2562256^2 and scattered points, and its samples change 2–6× less between resolutions than those of the FNO baselines.

W1 / W2 (presentation and motivation): The mathematical formulation is dense. A concrete motivating application and a clearer separation between the general framework and the results that need conditional Gaussianity would help.

Response: The error decomposition and the Wasserstein bounds for two-stage generation need no Gaussianity and hold for any data distribution. Conditional Gaussianity of the residual is needed only for the localization and compression results, where it gives closed forms for these errors. In fact, conditionally Gaussian models are motivated by a wide range of real-world applications, including natural image modelling (Portilla et al., 2003), spatial statistics (Rue et al., 2009) and inverse problems in function space (Dunlop et al., 2018). In each case the signal is Gaussian given a latent variable. Our Gaussian process mixture class is the function-space form of this structure, and it is flexible: continuous mixtures of Gaussian processes can represent multimodal and strongly non-Gaussian distributions on function space (Proposition 7).

References

  • Portilla et al. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Trans. Image Processing, 2003.
  • Rue et al. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. JRSS B, 2009.
  • Dunlop et al. How deep are deep Gaussian processes? JMLR, 2018.
Reviewer_WwUvOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a two-stage functional diffusion model. A function F0F_0 on a Sobolev space is split by a linear encoder into a finite-dimensional global state A0=SLF0A_0 = S_L F_0 and a residual U0=(I−RSL)F0U_0 = (I - RS_L)F_0. A standard diffusion samples the global state; a conditional functional diffusion then samples the residual with a pointwise denoiser that sees A0A_0, the noisy value at the query, and a fixed-dimensional local polynomial statistic of the noisy residual around the query. Both reverse chains are probability-flow ODEs.

Theory: an orthogonal decomposition of the denoising error into discretization, localization, compression and learning terms (Thm. 1); Gronwall-type W2W_2 bounds for the residual flow and the end-to-end model (Thms. 2, 4; Prop. 3); explicit localization and compression bounds under conditional Gaussianity and a kriging screening assumption (Thms. 5, 6; Props. 8, 9); an end-to-end rate (Thm. 10). Experiments: a finite-rank nonstationary GP-mixture benchmark with closed-form Bayes denoisers, a CIFAR-10 diagnostic with a wavelet split, and signed-distance fields of MNIST/Fashion-MNIST.

Critical strengths

Clean error decomposition. Theorem 1 is elementary but gives the right vocabulary, and every term is measured in closed form on the synthetic benchmark. Theorem 5 (exact localization error via a Schur complement) is a nice statement.

Honest diagnostics on CIFAR-10. The paired gen/oracle evaluation separates global-stage from residual error, and the paper does not hide that the monolithic baseline beats every hierarchical variant.

Score-free formulation. Working with conditional expectations rather than scores avoids density-existence issues in function space, in line with Pidstrigach, Marzouk, Reich and Wang, "Infinite-dimensional diffusion models", Hagemann, Mildenberger, Ruthotto, Steidl and Yang, "Multilevel diffusion: infinite dimensional score-based diffusion models for image generation", and Franzese and Michiardi, "Generative diffusion models in infinite dimensions: a survey" [not cited by authors].

Reusable benchmark with closed-form Bayes denoisers. SDF experiment shows a case where a six-dimensional local statistic suffices with a 20×20\times smaller model.

Critical weaknesses

The end-to-end bound diverges with the discretization, Theorem 10's NN-rates are assumed rather than derived (Assumption 8)

No experiment is in the infinite-dimensional regime and resolution independence is never tested. The benchmark has rank 841, CIFAR-10 is declared diagnostic, the SDF experiment retrains at 64264^2 with white (non-trace-class) residual noise whose per-mode SNR is grid-dependent, no real functional dataset is used, and no prior function-space method appears as a baseline.

Missing related work. The residual stage is conditional functional diffusion given a noise-free linear observation A0=SLF0A_0 = S_L F_0; this is a known setting, and the paper neither cites it nor uses it to justify the validity of its residual sampler. Specifically:

  • Baldassari, Siahkoohi, Garnier, Sølna, de Hoop. Conditional score-based diffusion models for Bayesian inference in infinite dimensions. NeurIPS 2023. [not cited] Proves validity of conditional reverse dynamics in HH for linear observations; this is exactly the residual stage.

  • Baldassari, Siahkoohi, Garnier, Sølna, de Hoop. Taming score-based diffusion priors for infinite-dimensional nonlinear inverse problems. arXiv:2405.15676, 2024. [not cited] Nonlinear extension of the above.

  • Pidstrigach, Marzouk, Reich, Wang. Infinite-dimensional diffusion models. JMLR 25(414), 2024. Cited, but not for its conditional-sampling results, which apply here.

  • Park, Choi, Lim, Lee. Stochastic optimal control for diffusion bridges in function spaces. arXiv:2405.20630, 2024. [not cited] Conditional generation and Bayesian learning in function space.

  • Ho, Saharia, Chan, Fleet, Norouzi, Salimans. Cascaded diffusion models for high fidelity image generation. JMLR 23(47), 2022. Cited only as coarse-to-fine generation; it introduced "conditioning augmentation", which Sec. 5.2 presents as if new.

  • Hagemann, Mildenberger, Ruthotto, Steidl, Yang. Multilevel diffusion: infinite dimensional score-based diffusion models for image generation. SIAM J. Math. Data Sci. 7(3), 2025. Cited once in a list; it is a wavelet multilevel functional diffusion that the CIFAR-10 construction closely resembles, and it is never compared against.

  • Phillips, Seror, Hutchinson, De Bortoli, Doucet, Mathieu. Spectral diffusion processes. arXiv:2209.14125, 2022. [not cited] Diffusion on truncated basis coefficients, i.e. the global stage with the basis encoder.

  • Dupont, Kim, Eslami, Rezende, Rosenbaum. From data to functa: your data point is a function and you can treat it like one. ICML 2022. [not cited] Same remark.

  • Kerrigan, Migliorini, Smyth. Functional flow matching. AISTATS 2024. [not cited] Same interpolant and deterministic reverse dynamics in function space

  • Lim, Yoon, Byun, Kang, Kim, Lee, Choi. Score-based generative modeling through stochastic evolution equations in Hilbert spaces. NeurIPS 2023. [not cited]

  • Franzese, Michiardi. Generative diffusion models in infinite dimensions: a survey. Phil. Trans. R. Soc. A 383:20240322, 2025. [not cited]

Questions for authors

Is there any regime in which the current bound tends to zero?

Please prove, or cite the proof from other works, that the residual flow with independent global and residual noise is a valid sampler of U0∣A0U_0 \mid A_0 in HH.

Train the SDF model at one resolution and sample at different one without retraining, with a trace-class residual noise. Does the local model keep its error?

Define the simultaneous baseline precisely. Why is it 3×3\times worse than the monolithic model with the same architecture?

Report the monolithic model class-conditionally and add at least one existing function-space baseline under the same budget.

Position the contribution explicitly against uncited literature

7 comments

We would like to thank Reviewer WwUv for the valuable comments, especially the concrete requests on the experiments, related work comparison and theory development, which help us strengthen the paper considerably. Following your suggestions, we have redone and added several experiments, and the new text and results are marked in blue in the revised PDF. Please see our responses below.

W2 (resolution independence) / Q3: Resolution independence was not tested, since the SDF model was retrained at 64264^2 with white, non-trace-class residual noise instead of being trained at one resolution and sampled at another.

Response: We agree. The revised experiment now tests resolution independence directly: our model is trained on 28228^2 only and sampled on 56256^2, 1122112^2 and at scattered points without retraining, with trace-class residual noise. The updated SDF results are in Section 5.3, Appendix B.3, Table 5 and Figures 13–14.

  • Design. Each SDF is computed once on a fine 4492449^2 grid, so its values at 28228^2, 56256^2 and 1122112^2 resolution are samples of the same function. The global state is 64 cosine coefficients computed as integrals of the function, so it does not depend on the resolution. The residual noise is a trace-class Gaussian field with λk∝(1+∣k∣2/k02)−2\lambda_k\propto(1+|k|^2/k_0^2)^{-2}, k0=8k_0=8, so one noise draw is a function that can be evaluated at any point. We train the local denoiser on 28228^2, with half of the training queries at uniform random off-grid points and half on the grid, as Theorem 2 suggests.

  • Results.

    1. The denoising error does not change with resolution. On both datasets, the held-out U0U_0-MSE over the residual variance on 56256^2, 1122112^2 and scattered points stays within 2.4% of its 28228^2 value for t≤0.5t\le0.5 (e.g. 0.0168 on 28228^2 vs 0.0172 on 1122112^2 for MNIST at t=0.1t=0.1). On MNIST, it is also within 3% of a model trained directly on 56256^2 (Table 5).

    2. Sample quality is kept and shapes are consistent across resolutions. Generated on 1122112^2, MNIST fields have eikonal error 0.116 (0.120 on 28228^2) and single-component fraction 0.974 (data 0.978). The regions inside their zero level sets match those of the same samples generated on 28228^2 (IoU 0.97). The model can also generate directly at scattered points (Figure 14).

    3. Off-grid training matters. Trained on grid queries only, the model is 18–42% worse off the grid at t=0.1t=0.1 (Table 5, bottom), as the degeneracy of CλC_\lambda in Theorem 2 predicts.

Together, these results show that with trace-class noise, one trained model keeps its denoising error and sample quality at any resolution or set of query points, which is the resolution independence the paper claims.

W2 (Function-space baselines) / Q5: No existing function-space method appears as a baseline under the same budget.

Response: The revised paper now includes three function-space baselines, all trained with the same data, noise and training schedule as our model (Appendix B.3, Tables 5–7).

  • Design. The three baselines are: (i) Monolithic Fourier neural operator (FNO) diffusions without hierarchy, at two sizes: FNO-s (0.16M parameters) and FNO-l (2.39M, 5×5\times our 0.48M in total). (ii) FNOs as the residual denoiser inside our hierarchy. (iii) A DeepONet-style full-context denoiser (1.43M) with our per-query inputs and off-grid training. All are trained on 28228^2 for 500 epochs with the same data and noise. Our model has 0.48M parameters in total, with 0.12M of them in the local denoiser.

  • Results.

  1. Our samples have better global structure than both monolithic FNOs, including FNO-l with 5×5\times more parameters. We measure global structure by FDA\sqrt{\mathrm{FD}_A}, the Fréchet distance of the global coefficients in % of their RMS, whose value between two sets of real fields is the floor. Ours is 0.5–1.0 above the floor on both datasets and resolutions, against 1.2–1.6 for FNO-l, and lower in every sampling seed. Local quality (eikonal error) is similar (Table 6, Figure 13).

  2. Our samples are 2–4× more consistent across resolutions than those of the monolithic FNOs. A sample generated on 1122112^2 and pooled to 28228^2 is compared with the same sample generated on 28228^2 (Table 6, bottom).

  3. Only our local denoiser can be queried at any point. FNOs inside the hierarchy denoise better at middle and high noise, but FNO-l's error at t=0.1t=0.1 rises 40% from 28228^2 to 1122112^2, and FNOs read the whole field, so they cannot be queried at scattered points (Table 7). DeepONet's input layer fixes the context to 28228^2, and our 0.12M local denoiser has 9–24% lower error than it at every noise level (Table 5).

Together, the baselines show that the hierarchy gives better global structure and consistency across resolutions than monolithic FNOs with more parameters, and that the local denoiser is what allows queries at any point.

Our responses to the remaining points continue in the next comment.

Q4 (simultaneous baseline): The simultaneous baseline is not precisely defined, and why is it 3× worse than the monolithic model with the same architecture?

Response: The simultaneous model runs a single diffusion chain on the whole field Ft=(At,Ut)F_t=(A_t,U_t) and denoises the global and residual parts together, so it follows the probability flow of (At,Ut)(A_t,U_t). The revised paper states this definition and evaluates the model under it (Appendix B.2, Table 1). The submitted version mistakenly updated the global part from AtA_t alone, so its 3× gap came from this implementation rather than from simultaneous denoising. In fact, under the precise definition, the gap disappears with full context.

  • Design. The simultaneous and monolithic models use the same architecture as our residual denoiser and predict F0F_0 from FtF_t. The simultaneous model also receives the decoded noisy global state RSLFt\mathcal{R} S_L F_t, while the monolithic model has no global input.

  • Results.

    1. With full context, the two single-chain models are close. FID is 22.3 for the simultaneous model and 21.5 for the monolithic one.
    2. With a local trunk, the hierarchy is clearly better. With r=8r=8, the simultaneous model scores 48.8 and the monolithic one scores 74.8, against 32.6 for the hierarchy.

Given only the noisy AtA_t, distant pixels still carry information about A0A_0 that a window cannot use, and conditioning on A^0\hat A_0 removes this nonlocality (Section 3.1).

Q5 (class-conditional monolithic model): The monolithic model should be reported class-conditionally.

Response: The revised Table 1 now includes it, with FID 20.0 against 23.0 for our class-conditional hierarchy, and Section 5.2 states that with full context, single-chain models are better end to end. Given the true A0A_0, however, our residual stage reaches 12.3, so the hierarchy loses through its global stage, which Table 3 shows is limited by memorization. Regularizing the global stage lowers the end-to-end FID from 30.8 to 27.8.

W2 (infinite-dimensional regime and real functional data): No experiment is in the infinite-dimensional regime: the benchmark has rank 841, CIFAR-10 is declared diagnostic, the SDF experiment retrains at 64264^2, and no real functional dataset is used.

Response: We agree that the submitted experiments did not test this regime, and we have clarified the role of each.

  • Gaussian process mixtures. The finite rank (841) makes the Bayes denoisers available in closed form, so this benchmark validates the theory, its assumptions and its predicted error terms exactly. It is not meant to test the infinite-dimensional regime.
  • CIFAR-10. It is a diagnostic experiment on a fixed grid, as we now state at the start of Section 5.2. It measures the terms of Theorem 4 and identifies changes that reduce them, such as conditioning augmentation and regularizing the global stage (Tables 1–4).
  • SDF. The submitted experiment used white noise and retrained at 64264^2. The revised experiment in our previous comment now works in this regime: each SDF can be evaluated at any point, the noise is trace class, and one model trained on 28228^2 is used at any resolution and at scattered points.

We further apply our model to real functional data from the 2D Navier–Stokes equations (Appendix B.4, Table 8, Figure 15).

  • Design. The data are the vorticity of 2D Kolmogorov flow at Re=1000\mathrm{Re}=1000, with 11,000 training and 1,000 test fields from a pseudo-spectral solver, so each field can be evaluated exactly at any point. The global state is the 69 Fourier coefficients with ∣k∣2≤20|k|^2\le20, and the residual noise is the same trace-class field as for SDFs. We train our model on 64264^2 only and compare it with the same model trained on 1282128^2 and with the two monolithic FNOs.
  • Results.
    1. The denoising error does not change with resolution. U0U_0-MSE at t=0.1t=0.1 is 0.0256 / 0.0254 / 0.0253 / 0.0253 on 64264^2 / 1282128^2 / 2562256^2 / scattered points, the same as the model trained on 1282128^2 (0.0254, Table 8, top).
    2. Our samples have better global structure than both monolithic FNOs. At every resolution, FDA\sqrt{\mathrm{FD}_A} is higher than ours by 5.6 for FNO-s and 6.0 for FNO-l (Table 8, bottom).
    3. Our samples are 2–6× more consistent across resolutions than those of the FNOs. A sample generated on a finer grid and read at the 64264^2 nodes differs from the same sample generated on 64264^2 by 6–8% of the residual RMS for ours, against 18–39% for the FNOs (Table 8, bottom, Figure 15).

Together with our previous comment, this completes our response to the experimental points. Our responses to W1 / Q1 (Theorem 10), W1 (Assumption 8), Q2 (validity of the residual sampler) and W3 / Q6 (related work and contribution) will follow in a separate comment soon.

W3 / Q2 / Q6: (related work and validity) The paper should discuss related work on function-space diffusion, position its contribution, and show that the residual flow with independent global and residual noise is a valid sampler of U0∣A0U_0\mid A_0.

Response: We thank the reviewer for the detailed list. We now cite and discuss all listed related works in Section 2, and below we compare each with our method and show the validity of the residual sampler.

  1. Baldassari et al. (2023), Conditional score-based diffusion models for Bayesian inference in infinite dimensions.
  • Their setting. They sample the posterior of a Bayesian inverse problem, X0∣Y=yX_0\mid Y=y, where X0∈HX_0\in H is the unknown function and Y=AX0+B∈RnY=AX_0+B\in\mathbb R^n is a noisy linear measurement of X0X_0. The measurement Y=yY=y is fixed and never diffused. A single-stage diffusion runs on XtX_t, and their denoiser approximates E[X0∣Xt,Y=y]\mathbb E[X_0\mid X_t,Y=y] from whole field XtX_t and yy.

  • Differences.

    • We generate functions unconditionally in two stages, while they have only one.

    • Unlike their fixed measurement yy, our global state A^0\hat A_0 is not measured but generated by a first diffusion.

    • Their yy is a noisy copy of the measured component of X0X_0, whereas our A0A_0 and U0U_0 are complementary components of F0F_0 (SLU0=0S_LU_0=0), statistically dependent only through the data distribution.

    • Our residual diffusion runs only on ker⁡SL\ker S_L, and its denoiser reads local statistics, while their denoiser reads the whole field XtX_t.

  • Validity (Q2). Since SL(I−RSL)=0S_L(I-\mathcal RS_L)=0, both U0U_0 and noise U1=(I−RSL)F1U_1=(I-\mathcal RS_L)F_1 lie in VU=ker⁡SLV_U=\ker S_L. For each fixed aa, the residual stage is therefore an unconditional diffusion on the Hilbert space VUV_U, with target μU0∣a\mu_{U_0\mid a} and noise N(0,CU)\mathcal N(0,C_U). Validity of its reverse dynamics, and hence of the sampler for U0∣A0U_0 \mid A_0, then follows from existing unconditional function-space diffusion results (Pidstrigach et al., 2024; Kerrigan et al., 2024), under their regularity assumptions on μU0∣a\mu_{U_0\mid a} and CUC_U.

The residual noise must be independent of the generated global state A^0\hat A_0, which at sampling time is a deterministic function of the global noise A1A_1. Drawing A1A_1 and U1U_1 independently ensures this, so given A^0=a\hat A_0=a the residual chain starts from N(0,CU)\mathcal N(0,C_U) as the unconditional results require.

  1. Baldassari et al. (2024) extend posterior sampling to nonlinear inverse problems, which is distinct from our setting.

  2. Pidstrigach et al. (2024). As in their work, we write the reverse dynamics with the conditional expectation E[U0∣A0=a,Ut]\mathbb E[U_0\mid A_0=a,U_t] instead of a score, which is not defined without a Lebesgue density in infinite dimensions. Their results give the validity of our residual sampler (item 1).

  3. Park et al. (2024) also sample functions conditionally in function space, conditioning on observed values through a controlled SDE. We instead use a two-stage diffusion with a generated global state and a local residual denoiser.

  4. Ho et al. (2022) introduced conditioning augmentation. We now credit them in Section 5.2. What we add is a direct measurement of how augmentation reduces Lgen(θ)L_{\mathrm{gen}(\theta)}, the sensitivity of residual sampler to global stage error.

  5. Hagemann et al. (2025) train one FNO score across discretization levels, with trace-class noise. Their levels are spectral truncations of a single diffusion of the whole function, whereas we generate a global state first and the residual conditioned on it. Our monolithic FNO baselines (Appendices B.3, B.4) are this model class, and our model is better in global structure and cross-resolution consistency.

  6. Phillips et al. (2022) diffuse truncated basis coefficients, which corresponds to our global stage. They generate the function from these coefficients alone, whereas our residual stage generates what the truncation discards.

  7. Dupont et al. (2022) generate the latent codes of neural fields, a learned nonlinear counterpart of our global stage. Same remark as item 7.

  8. Kerrigan et al. (2024). We use their interpolant and deterministic dynamics.

  9. Lim et al. (2023) formulate function-space diffusion through stochastic evolution equations.

  10. Franzese and Michiardi (2025) survey infinite-dimensional diffusion models.

Positioning (Q6). Conditional diffusion in function space, diffusion on finite representations and coarse-to-fine generation are established, and we build on them.

Our contribution is a residual stage that denoises each query from local statistics after conditioning on a generated global state, so one small model works at any resolution and set of points with a fixed cost per query. We also give an error decomposition and bounds on how the error of the generated global state propagates, which we measure directly. None of the listed works has these.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We summarize the main revisions below. Changes are marked in blue in the revised PDF.

  1. Resolution independence. One model, trained once at a single resolution with trace-class residual noise, keeps its denoising error and generates consistent samples at unseen resolutions and scattered points (Section 5.3, Appendix B.3).
  2. Real functional data and function-space baselines. We added Navier–Stokes vorticity, a concrete scientific application (Appendix B.4), and monolithic FNO, FNO residual denoisers and DeepONet baselines with the same data, noise and schedule (Section 5.3, Appendices B.3 and B.4).
  3. Theory. Theorem 10 now states that it is a bound for a fixed design. Proposition 14 bounds the error uniformly in the design, the new Appendix E gives a bound that does not grow with resolution, and Appendix C discusses the rates of Assumption 8.
  4. Sensitivity to the global state. A direct estimate of the sampler's sensitivity complements the FID gap (Section 5.2, Appendix B.2).
  5. Related work. We discuss all suggested works and the validity of the residual sampler (Section 2).

We thank the reviewers and the Area Chair for their time and helpful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.