- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies errors in diffusion-based likelihood and mutual information estimates caused by fixed integration windows. The authors show that when learned representations change scale, integrand mass can fall outside the window, leading to underestimation, and examine high-SNR overestimation issues. They audit existing models and propose practical evaluation checks.
Critical strengths
- Addresses an important evaluation problem where truncation artifacts can mimic real model improvements.
- The analysis goes beyond a single example, including norm/window ablations, Gaussian tasks, and a Plaid positive control.
- The practical recommendations (such as recomputing scale-dependent bounds) are useful for the community.
Critical weaknesses
-
The paper treats its wider dense-grid integral as a reliable reference, but lacks strict tail guarantees for high-SNR regimes and learned denoisers.
-
The Plaid experiment modifies multiple components simultaneously (removing normalization, fixed windows, and unnormalized models), failing to isolate whether normalization removal alone breaks the evaluation. Numerical precision and out-of-distribution model errors are also not cleanly separated.
-
Some claims outrun the evidence. For example, Section 5.1 claims analytic predictions match MINDE within 1 nat, but Table 4 shows a discrepancy of 2.22 nats (5.80 vs 3.58).
-
The proposed input-rescaling check needs more rigorous definitions, as standard learned model likelihoods do not obey simple variable changes without consistent transformations.
Questions for authors
-
Can you provide rigorous tail bounds and grid-refinement results for the headline dense-grid evaluation?
-
In the Plaid experiment, can you separate the effects of numerical precision, evaluation outside the training window, and embedding normalization?
-
How do you account for the 2.22-nat difference in Table 4, and what exact transformations are required for the rescaling check?