- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies what is referred to as "frequency budget for a diffusion residual estimate". The contribution seems to take an RKHS-based approach to evaluate the discrepancy between the learnt diffusion and the true probability paths.
Critical strengths
-
The paper provides a technically detailed theoretical analysis, including matching upper and lower bounds for the two precision criteria.
-
The distinction between the two precision requirements is potentially interesting, and it could become a contribution
-
Numerical experiments strengthen the contribution
Critical weaknesses
My main concern is the paper's clarity and positioning. The paper is very difficult to follow. It took me several passes to understand its basic elements, such as the problem being addressed, the proposed contribution, and its relevance to machine learning. I have to say I am still somewhat unsure that I fully understood these points.
Detailled items as follows:
Unclear problem formulation and terminology.
The abstract and introduction introduce terms such as "frequency budget", "weak diffusion residual", "signed Gaussian-kernel U-statistic", and "exact-risk precision" without adequately explaining their meaning or connecting them to a concrete problem. While the paper eventually formalises these concepts, it assumes too much familiarity with its particular setting. The presentation covers concepts from diffusion models, weak continuity equations, kernel discrepancies, random features, and statistical precision without establishing a clear conceptual connection between them.
Unclear contribution and positioning.
It took me considerable effort to identify the central research question, which seems to be how many random Fourier features are required to approximate a kernel statistic at a predefined accuracy. The paper would benefit from stating this question explicitly. The motivation for considering the two presented precision criteria is also not well developed.
Limited practical relevance.
The connection to modern diffusion models is not clear. As far as I understood, no experiments involved learned diffusion models or realistic data. With this, the practical experimental value of the contribution is questionable
Final remarks.
Though the technical effort is commendable, the paper needs a substantially clearer motivation, exposition, and convincing argument for its significance
Questions for authors
-
How can your results be used to design and implement improved diffusion models? how is the concept of frequency budget relevant in this setting?
-
Also, in practice, when are the two precision criteria presented in the paper particularly relevant?
-
To what extent do the theoretical results extend beyond the particular periodic construction considered in the paper?
-
Could you confirm, in a succinct and clear manner, the main research question and your contribution?
Response to Reviewer ius4: question, observable, precision and trained evidence
Thank you for identifying the need to make the question, observable and intended use clear. I have rewritten the Abstract, Introduction and Section 2 around the following question: How many Fourier frequencies are needed to approximate the same weak-residual statistic at a specified statistical precision?
1. Problem and practical use. Weak Schedule Deviation (WSD) is a weak self-consistency diagnostic for a fixed diffusion/flow velocity, comparing it with the interpolation of its own endpoint law. Generate endpoints from the model, draw independent interpolation noise, and evaluate the model velocity at the interpolated points. With the coupling in Section 2,
Y_t=\nabla_1k(X_t,\cdot)^\top\Delta_t,\qquad q=\lVert\mathbb{E}Y_t\rVert^2=\mathrm{WSD}_k^2.$$ Thus no known true probability path is required. The observable is a derivative feature weighted by velocity mismatch, rather than an ordinary kernel embedding of residual vectors. The frequency budget controls additional approximation MSE for the same signed off-diagonal exact-kernel statistic; it is not the training budget. The revised Section 2 keeps raw signed estimates without clipping or square roots. An externally fixed endpoint law instead defines a reference comparison. **2. Why two precision criteria?** Signal-relative precision controls added MSE relative to the squared diagnostic signal, $\Delta R\leq\varepsilon^2q^2$. Exact-risk precision preserves the statistical fidelity of exact-kernel evaluation, $\Delta R\leq\varepsilon^2R_{n,\infty}$. Since $q=\mathrm{WSD}_k^2$, the first denominator is $\mathrm{WSD}_k^4$. The revision explains these uses before the technical results. Neither criterion alone establishes detection power. The same trained-model cell, $n=512,m=256$, illustrates the distinction. Across three training seeds, iid added MSE divided by exact-data MSE is 0.05933–0.06119, whereas added MSE divided by $q^2$ is 0.5717–2.6729. Thus approximately 6% additional exact-data risk can coexist with substantially different signal-relative accuracy. Independent Exact-U repetitions estimate both denominators; a joint whole-group bootstrap retains their dependence with approximation losses (accompanying Supplement, Table S6). No single Exact-U observation is treated as population $q$. **3. Contribution and extension.** Section 3 states matching budgets $\Theta(\sqrt{\log n}/\varepsilon^2)$ and $\Theta(n/\varepsilon^2)$, uniformly over unknown $q\geq b/n$ in the stated scalar Gaussian periodic-loop class, for fixed class parameters and sufficiently large $n$. Optimality is over admissible common iid importance proposals for this estimator. The joint iid analysis retains the data–frequency interaction, $\Delta R=(V_\nu+J_n^\nu)/m$, and identifies the intermediate-scale width mechanism. The residual construction and Hilbert-space U-statistic algebra are more general under suitable independence and moment assumptions. Sharp rates additionally use scalar spectral packing, Gaussian Fourier decay/overlap, periodic first-harmonic noncancellation and positive interpolation noise. The proof route and these dependencies are set out in Appendix A1–A2 of the accompanying rebuttal_supplement_v4.pdf. They do not imply a high-dimensional learned-field theorem. **4. Trained-model evidence and its limits.** Section 5 adds one four-component 2D Gaussian-mixture task, three independent 20,000-step trainings, fixed final checkpoints, and own-model endpoints from 256-step RK4. The grid is $n=128,512,2048$, with seven budgets $m=16,\ldots,1024$, comparing iid Gaussian and Gaussian-marginal ORF. These are three trainings of one synthetic task, supporting conditional diagnostic calibration rather than training or generation-quality improvements. At fixed tolerance the exact-risk theorem gives $m=\Theta(n)$, hence no asymptotic scalar evaluation advantage over Exact; finite cost–error tradeoffs are reported in Section 5. The scope remains explicit: scalar experiments compare the theoretical proposals, while the trained-model cost experiment compares iid Gaussian/ORF. Matched total computation of the theorem-attaining broad/critical proposals on trained models has not been directly tested. The accompanying Supplement, Tables S1–S10, reports the numerical evidence and its statistical limitations. The revised paper also includes the new protocols and comparisons in Appendix H and a one-page proof roadmap in Appendix I.