acceptodds
Under review as a conference paper at ICLR 2027

Sharp Bounds for Conditional-Variance Estimation in Diffusion Regression

Abstract

Estimating the conditional variance from conditional diffusion samples involves model-law mismatch and Monte Carlo error. Small Kullback–Leibler divergence alone cannot control variance, even for smooth Gaussian mixtures. To limit the influence of extreme model samples, we analyze a clipped pairwise estimator with exact variance correction for early-stopped Ornstein–Uhlenbeck diffusion. Under a target pairwise -moment bound and a target-to-model conditional KL bound, we establish matching upper and lower bounds for estimators using only model samples. For scalar covariates, we also prove a new-covariate risk bound for common-radius selection using independent validation data. We evaluate the estimator through counterexample, controlled, and learned-model experiments. The experiments distinguish clipping gains from validation selection loss and show diminishing improvements as the Monte Carlo sample size increases.

Primary area:
generative models
Keywords:
conditional diffusion regression, diffusion generative model, finite-sample guarantees, conditional variance estimation, nonasymptotic analysis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ehWzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work studies a conditional-variance estimator for diffusion regression, where the goal is to estimate the variance using samples from the diffusion model. After showing that small KL error (as typically established in the literature) does not guarantee small variance estimation error, the authors investigate the theoretical guarantee of a (clipped version of) typical pairwise variance estimator, and they further show that the rate is minimax optimal among a designed distribution class. A practical way to set the clipping radius is also studied with theoretical guarantees. Numerical studies on synthetic data are provided.

Critical strengths
  1. The proof is largely sound.
  2. The paper is based upon an existing variance estimator with theoretical results that achieve the minimax rate (by proving a lower bound).
Critical weaknesses
  1. The largest weakness to me is the one-dimensionality starting from Section 4.2. While all the content before can be applied to multivariate XX, it is not the case to select the common radius, where fixed anchors on the interval are used for interpolation. For multi-dimensional features, such interpolation becomes very hard. This is also the case in the experiment, which is based upon an one-dimensional XX.
  2. Besides, even with one-dimensional XX, the current experiment does not fully suggest the benefit of the common radius. Specifically, in Table 1, the Variance RMSE with common radius might not be advantageous over the unclipped version. However, from Theorem 1, an unbounded RR would yield a worse upper bound, for sure. While the authors admit this in the experiment, more discussion needs to be given.
Questions for authors
  1. Please address my two comments above. Also, for the one-dimensionality issue, is it possible to let RR be a hyper-parameter itself and perform some hyper-parameter tuning? (Might be left for future works)
  2. While (7) and (8) are based on existing works, the later time correction in (9) is added. The logic is clear: it is needed to guarantee un-biasedness. However, just as in classical estimation theory, unbiasedness does not guarantee the minimum MSE. Thus, is it possible to get around such a time-correction and obtain a smaller MSE?
  3. Suggestions on writing: Please discuss assumptions and interpret your theoretical results. For example, as far as I see, Assumption 2 is not standard in the literature. While it might be a direct consequence of statistical learning bounds, this must be stated explicitly to relate to existing assumptions. Also, the terms in Theorem 1 needs to be discussed. For example, suppose RR becomes large and given that all higher moments are bounded, there still remains the last term to grow unbounded. Why is that term, and is it due to the analysis technique or to the problem nature?
Reviewer_bsZjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper recalls the standard conditional diffusion model with its grid-based learning and sampling formulation, and suggests using the learned conditional diffusion sampler to construct a clipped pairwise Monte Carlo estimator for estimating the conditional variance. The authors keep track of different sources of approximation errors to establish a conditional KL bound and minimax optimal variance estimation error rate given finite model sample size (MM). Then in a 1-D special case, the author proposes a seemingly new validation-based interpolated estimator constructed from the previous clipped estimator and accompanying theorem. The numerical study tries to verify the statements of previous theoretical statements using low-dimensional synthetic data.

Critical strengths

None

Critical weaknesses

The overall presentation requires substantial work to be ready for general audience.

  1. The introduction is mostly a related work section without sufficient motivation and setup of the problem. A reader with only broad familiarity rather than domain knowledge cannot extract meaningful takeaways from each paragraph and appreciate the contrasts of the work against those related works. Concepts and notations are not sufficiently defined before using, which requires readers’ additional efforts in inferring their meanings from the context. For example, in paragraph starting from Line 51, “target-to-model conditional KL divergence” appears before the authors have set up the concept of what refers to the target and the model. The following line also provides a bound and explains that “... matches this dependence on BB, κ\kappa, and MM, up to constants depending only on qq, but the readers have not seen these notations at all. There are numerous examples in later sections, including abusion of notations UU and ww.

  2. The numerical section is hard to extract meaningful takeaways and the negative results contradicting the hypothesis are not fully analyzed and/or affect the technical advance. For example, in Section 5.1, to verify Proposition 2, the hypothesis is that for certain specified laws, rare and large responses can change variance while keeping KL small, yet the section ends with demonstrating a non-divergence variance through Figure 1, with no analysis at all.

The only contribution is restricted. About half of the paper recalls known components such as the conditional diffusion regression and its grid-based discretization, and an established clipped conditional variance estimator, and develops bounds that are generally expected by synthesis of established proof tools . If any nontrivial component is required to show the results, they should be properly highlighted. While Section 4.2 seems to propose a nontrivial interpolated estimator, it implicitly requires the covariate to lie in 1-D, is not sufficiently motivated and connected to previous sections, introduced with a lot of new notations, and its accompanied theorem does not look comparable to previous propositions and has limited interpretation of any implications.

Questions for authors

For the interpolated estimator, can you motivate it better, clarify how to choose wkw_k, and why we need to further clip/project the individual estimator given that the estimator itself already involves a clipping? Why do you introduce λ=R2\lambda = R^2, and what presentation benefits it brings? Can you generalize a common-radius estimator when the covariate is in a general Borel space?

Reviewer_saWXOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies conditional-variance estimation from diffusion-model samples under distribution mismatch. It analyzes a clipped pairwise estimator with OU variance correction, matching estimation bounds, and validation-based clipping-radius selection.

Critical strengths

The paper addresses a relevant uncertainty-estimation problem and distinguishes model-law mismatch, Monte Carlo error, and validation-selection error, with theoretical results and experiments examining these sources separately.

Critical weaknesses
  1. The introduction and experimental narrative do not communicate the contribution clearly enough. The first two introductory paragraphs (lines 027–050) devote substantial space to related-work summaries before explaining, through an accessible example, why accurate conditional-law approximation may still produce poor variance estimates. The third paragraph repeats the abstract's KL–variance observation and introduces a “pairwise q-moment bound” (lines 051–063) without a plain-language explanation. In the fourth paragraph (lines 064–069), the relationship between the Wang et al. (2008) difference-based criterion and the paper's new selection guarantees is not sufficiently explicit. Although the main results are stated, their novelty and practical implications are difficult to extract from the citation-heavy exposition. Table 1 (lines 433–456) also combines several metrics and two sample budgets without making the primary variance-estimation comparison immediately clear. Please reorganize the introduction around the problem, a motivating example, the gap, and a concise account of the new results; explicitly separate inherited components from new analysis, and foreground the variance RMSE/NRMSE comparison and its interpretation in the experiments.

  2. The assumptions and theoretical results need more intuitive guidance. Section 3.1 (lines 159–177) moves quickly from moment conditions to a score-risk budget and reverse-process admissibility, with substantial probability and stochastic-process terminology. The fitting sigma-field G_n is defined in lines 079–081, but its meaning should be recalled when Assumption 3 conditions on it. There is some interpretation of the KL terms in lines 193–197, yet readers still need a clearer explanation of which assumptions concern the data, the fitted score, or the sampler, and what each excludes. Likewise, the pairwise-moment and KL budgets in Theorem 1 (lines 218–230) would benefit from an intuitive account of how they control clipping bias and model mismatch. Please introduce each assumption with its purpose and a representative example, and accompany the principal bounds with a plain-language explanation of their terms, scope, and practical consequences.

Questions for authors
  1. Introduction, lines 051–069: Could you state explicitly which estimator or validation components are inherited from Ke et al. (2019) and Wang et al. (2008), and which guarantees are new? A short contribution summary and an intuitive explanation of the pairwise q-moment condition would help clarify the advance.
  2. Section 3.1, lines 159–177: Could you explain the role of each assumption in ordinary statistical terms and give a representative diffusion-regression setting in which the conditions hold? Please also recall that conditioning on G_n means treating the fitted data and fitting randomness as fixed.
  3. Table 1, lines 433–456: Which comparison should readers regard as the main empirical takeaway for conditional variance? Please distinguish RMSE from NRMSE and explain more prominently why common-radius selection improves the latter while worsening the former relative to unclipped estimation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.