- Rating
- 4(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors propose a method called JANE for blind image denoising. Specifically, under the assumption of additive white Gaussian noise, they show that the noise variance can be read directly from the eigenvalues of the analytical score Jacobian, defined as the Hessian of . Provided that the score can be learned, this yields a method for blind denoising since according to Tweedie's formula. However, the authors notice that the eigenvalues of the analytical score Jacobian do not match the ones from the learned one and propose a theory to explain this phenomenon, as well as a pratical method to estimate from the eigenvalues of the leaned score Jacobian anyway. In the end, comparisons are made with Noise2Score and UNSURE for image denoising, showing that JANE estimates more accurately and so provides better reconstructions.
Critical strengths
Noise2Score (2022) showed that the noise variance could be estimated directly from a single noisy image , provided that the score is learned. However, it was not clear why it worked, the arguments being based only on intuitions. This paper derives a theory to explain why the noise variance does hide in the score, and more precisely in the spectrum of its Jacobian. I think this is a piece of theoretical contribution, even though JANE is to be seen only as a proof of concept, as in practice, I don't think it will adopted due to its very high computational burden.
Critical weaknesses
Major:
-
In your experiments, you consider denoisers of the form , where is the step size that needs to be estimated and for which the optimal value is (assumed unknown) according to Tweedie's formula. You propose three different ways to estimate : your method JANE, Noise2Score (that requires computing and ), and the oracle parameter that you call Genie MMSE where . But you should also compare with methods that can estimate by directly estimating the noise level (see for example [a, b]).
-
For estimating , JANE needs to compute the eigenvalues of \nabla_y s_\hat{\theta}(y_i) for , where is a sufficiently large subset of the training images. I believe that the computational cost is tremendous, all that for just estimating ... Indeed, for a 512x512 image , I am not even sure that computing \nabla_y s_\hat{\theta}(y) is feasible. Therefore, JANE is only applicable to very small images. Could you please provide a time comparison between all competing methods? Also, Noise2Score (2022) estimates with only one single image at inference, while JANE requires a subset for estimation, it is an important difference that needs to be highlighted.
-
The noise assumptions are very restrictive: JANE is tested for additive Gaussian noise with constant . What happens if the noise level is non-constant and varies for each image in the training set as in [c]?
-
Section 4 is particularly hard to follow. To be honest, I did not understand it. I think a complete rewrite is necessary to make it clear. In particular, some terms are obscur to me:
- Line 268, what is the residual kernel exactly?
- Line 269, what is the cross-output derivative covariance ? What does and even represent?
- Line 270, what does stand for?
- Line 282, what is the subset ? Is it the one defined in line 377?
- Some sentences are enigmatic such as "For independent readouts the blocks are diagonal; with weight sharing across output positions, as in convolutional networks"...
[a] Stanislav Pyatykh, Jurgen Hesser, and Lei Zheng. Image noise level estimation by principal component analysis. IEEE TIP, 2013.
[b] Guangyong Chen, Fengyuan Zhu, and Pheng Ann Heng. An efficient statistical method for image noise level estimation. ICCV, 2015.
[c] Sebastien Herbreteau and Etienne Meunier. Divergence-free neural networks with application to image denoising. ICLR, 2026.
Minor:
-
You could specify that the are the eigenvalues of in equation (6). Also, for improved clarity, please explain that line 147 comes from the fact that since a covariance matrix always exhibits non-negative eigenvalues.
-
Missing reference for practical learning of (notably used in Noise2Score): Lim, J. H., Courville, A., Pal, C., & Huang, C. W. AR-DAE: Towards unbiased neural entropy gradient estimation. ICML, 2020.
-
Please add the PSNR values of the denoised images in Figure 3.
Questions for authors
-
Line 050, you say that UNSURE can estimate from a learned score network. Although I am familiar with UNSURE, I don't see what you mean. Can you explain how UNSURE estimates ? Because if it could, it would have the same performance as Noise2Score, but it does not seem to be the case (see [c]). If I am not mistaken, line 090, you mention that it is learned from the score norm but only estimates the Lagrange multiplier , not .
-
Why choosing {12, 25, 51, 77}? It seems weird. Usually, people use {15, 25, 50, 75}.
-
In order to denoise images of size higher than 64x64, I wonder if it is possible to estimate the value of on randomly chosen patches of size 32x32 of a single noisy image (to form the subset )? Once is estimated, you could simply output y + \hat{\sigma}^2 s_\hat{\theta}(y). Indeed, I think the computational burden of JANE is mainly on the estimation of . If you could provide PSNR results on standard test datasets such as the ones used in [c], it would strengthen the paper.
-
Why is UNSURE so bad on Figure 3? Do you have explanations for that? What if you use the method recommended by the authors of UNSURE, that is searching for a saddle point? (the authors of UNSURE indeed state in their paper that they obtained slightly worse reconstruction results in their experiments with the score-approach compared to the saddle-point approach).
-
Line 406, what does "the color configuration uses an ensemble of 32 networks" mean?
[c] Sebastien Herbreteau and Etienne Meunier. Divergence-free neural networks with application to image denoising. ICLR, 2026.
9 comments
Response to Reviewer 3nWu (2/7)
Major 2. For estimating , JANE needs to compute the eigenvalues of for , where is a sufficiently large subset of the training images. I believe that the computational cost is tremendous, all that for just estimating ... Indeed, for a 512x512 image , I am not even sure that computing is feasible. Therefore, JANE is only applicable to very small images. Could you please provide a time comparison between all competing methods? Also, Noise2Score (2022) estimates with only one single image at inference, while JANE requires a subset for estimation, it is an important difference that needs to be highlighted.
We thank the reviewer for the comment. Indeed, the eigendecomposition of the full Jacobian was one of the most important limitations of our original design, and it limited the previous submission to small images. During the revision we have addressed this in three ways.
- Scalable variants. We have included JANE-crops, which builds the spectrum from the Jacobian blocks of random crops of each image and follows your suggestion in Question 3, and JANE-SLQ, which approximates the spectrum by stochastic Lanczos quadrature from Jacobian–vector products. Neither of them forms the full Jacobian. With JANE-SLQ, building the spectrum costs to GPU-seconds per image. Due to the limited time and computational resources of the revision, we have not been able to finish the evaluation on images. However, the cost of JANE-SLQ per image is Jacobian–vector and vector–Jacobian products plus operations (Table 7), and for a convolutional network both terms grow only linearly with the signal dimension . A image has four times as many pixels as a one, so we expect about 8 to 9 GPU-seconds per image, that is, about 15 GPU-minutes for 100 pooled images. This is still far below the cost of training the score network, which already takes more than 12 GPU-hours at (Table 3).
- Time comparison (Table 3, and every configuration in Table 15). Training dominates the cost of JANE-SLQ and of every baseline. The score network is trained once and shared by JANE and Noise2Score, so considering the training time, the overall time cost of JANE-SLQ is within 4% of that of Noise2Score on every configuration. The per-image estimation of JANE-SLQ is still slower than that of Noise2Score, but with training included the total time of the two methods is similar. For Neighbor2Neighbor, Noise2Self and UNSURE, the cost is their own training, from a few minutes to about 18 hours depending on the method and configuration.
- A single image. JANE does not need a subset, and it works even with a single image's eigenspectrum, which we now state explicitly. Estimated from each image alone, on the six small configurations, JANE's average error in is %, lower by a factor of than that of Noise2Score, which also uses a single image at inference. Its estimate is closer to than that of Noise2Score in 23 of 24 settings, and its PSNR is on average within dB of denoising with the true (Table 14). Pooling more observations further reduces the spread of the estimate (Table 9).
Changed text: Section 5; Table 3 and the last two paragraphs of Section 6; Appendix D.4 (Tables 14 and 15); Appendix E.
Response to Reviewer 3nWu (3/7)
Major 3. The noise assumptions are very restrictive: JANE is tested for additive Gaussian noise with constant . What happens if the noise level is non-constant and varies for each image in the training set as in [c]?
We thank the reviewer for the comment. The per-image noise setup is a new scenario for us, and we have tried our best to be close to the standard workflow in [c] within the limited revision time and computational resources. In the revised manuscript, we have added experiments where the noise level of each image is drawn uniformly from (Table 2). JANE works in this setting without change, because each image is denoised with the estimate from its own spectrum, even though per-image noise violates the noise model of our analysis. JANE's PSNR is on average dB below that of the oracle that knows every . Averaged over the six configurations, it is at least dB higher than every baseline except Noise2Score (Table 2). Noise2Score is comparable in PSNR, and JANE estimates the noise more accurately, with a mean error of % against % (% for JANE-SLQ).
Our noise range starts at 5 rather than 0 for the following reason. By Equation (6), the noise eigenvalues of the score Jacobian lie at about , so they diverge as : in the directions without signal, the noisy samples concentrate at a single point. When and all images share the same noise level, pooling the spectra of many images still locates the peak regardless of the width of the spectrum. With per-image noise, however, each image has to be estimated from its own spectrum, and for a noise level close to 0 it is very hard to locate a peak from the limited number of eigenvalues of one image, spread over such a wide range. As long as the noise level is bounded away from 0, the width of the spectrum stays bounded and the peak can be located.
We also agree that CENSURE [c] should be evaluated in this scenario. Due to the limited time and computational resources of the revision, we have not been able to finish this evaluation. We have therefore added it, together with the standard training and test sets of [c], to the evaluation plan in the limitations (Appendix E), and have cited this insightful work.
Changed text: Section 6 (paragraph on single-image and per-image estimation); Table 2; Appendix E.
Response to Reviewer 3nWu (1/7)
Thank you very much for your time and review! Please also see our general response above, which summarizes all revisions and the main results. We hope that our clarifications and additional experimental results have adequately addressed your concerns. We would greatly appreciate it if you could reconsider your initial assessment and update your rating accordingly.
We thank the reviewer for the suggestion. During the revision we have extended the evaluation to broader classes of blind denoising methods, and the classical noise-level estimators of Pyatykh et al. (2013) [a] and Chen et al. (2015) [b] are now among the benchmarks. We have cited both papers in the revised manuscript: Chen et al. in the related work (Section 2.1), and both in the experimental setup of Section 6 and in Appendix D.1, where their settings are described. They are run on the same noisy images and plugged into the same Tweedie step with the same score network, so that only the noise estimate differs. Averaged over the 48 settings, JANE's error in σ2 is lower by a factor of 8.8 than that of Chen et al. and 9.3 than that of Pyatykh et al., and its PSNR is on average at least 1.1 dB higher (Tables 1, 12 and 13). JANE's PSNR is equal to or higher than both in 47 of the 48 settings. Both estimators rely on local patch statistics, which can fail badly. On the channel data, for example, Chen et al. is on average 6.9 dB below the oracle. On colour images, the estimate of Chen et al. is closer to σ2 than JANE's in 14 settings, but JANE still denoises equally well or better, with an equal or higher PSNR in 13 of them.
Changed text: Section 2.1 (citation of Chen et al.); Section 6 (setup, Table 1 and results); Appendix D.1; Appendix D.4 (Tables 12 and 13).