- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a new approach to generate noisy/noisy pairs for training denoising neural networks in a self-supervised fashion. The method is based on the assumption that an initial denoiser is available and improves it with a two-stage procedure: 1) Learn re-corruption, 2) Fine-tune denoiser. The authors show state-of-the-art results on two popular real-world datasets: SIDD and FMD.
Critical strengths
The paper is very well-written and based on a solid mathematical result (Proposition 1). The ultimate goal is highly relevant and well motivated: how to generate noisy/noisy pairs for self-supervised learning. Finally, the experimental results (Table 2 and 3) are impressive.
Critical weaknesses
My main concern is the similarity with DCD-Net [a] and the high dependence on the initial denoiser . Instead of generating noisy/noisy pairs in the spirit of Noise2Noise (N2N) as in [a], the authors propose to generate noisy/noisy pairs in the spirit of Recorrupted-to-Recorrupted (R2R). Both approaches rely on an initial denoiser . The authors should at least compare with DCD-Net [a] since N2N is theoretically superior to R2R (N2N predicts from , while R2R predicts from a noisier version of , namely , which degrades performance).
[a] Zou, Y., Yan, C., & Fu, Y. Iterative denoiser and noise estimator for self-supervised image denoising. ICCV, 2023.
My overall rating is weak rejection but I may consider improving my rating based on the responses of the authors. Here are more details about my evaluation:
Major:
-
The proposed method relies on an initial denoiser which is obtained via the R2R framework. It means that the method is not entirely self-contained. Moreover, R2R is supposed to work under the assumption of additive white Gaussian noise (AWGN). What if this assumption is not fulfilled and this initialization stage (see Fig. 1) fails? According to Table 5, it seems that the success of the method highly depends on , and that the gain of the method when is properly chosen is not so important (only +0.23 dB). Conversely, when is not properly chosen, the method fails (35.88 dB vs 38.21 dB, see Table 5). I believe it is essential to include in Tables 2 and 3 and Figure 2 to better assess the gains achieved by the proposed method.
-
I notice some similarities with the approach proposed in DCD-Net [a] (for which the reference is missing and should be added), inspired by the Noise2Noise framework. In particular, since you assume that you have access to pairs from noisy images thanks to , where and are good approximations of and , respectively, I wonder if it is not possible to simply learn to generate samples from the same distribution as with a diffusion model for example (potentially conditioned on ), or maybe using the same noise estimator as [a]; and, in a second stage, simply go back to equation (1) where is replaced by the generated samples. In this setting, Proposition 1 is not necessary. You could also imagine an iterative scheme to boost performance as in [a]. I therefore strongly recommend comparing your method with DCD-Net [a].
-
Real-world noise datasets, namely FMD and SIDD, are considered for comparisons, which is great. But I would appreciate if the authors could also validate their method on synthetic noises (for example synthetic Poisson noise and compare with GR2R as upper bound). Also supervised baselines are missing in Table 2 and 3.
-
L2R is also a work close to this one. However, as you mention, line 850, "[L2R] results should be interpreted as a transfer study rather than a reproduction of the original L2R experiments". I think that comparing with L2R trained from scratch on SIDD is necessary rather than a simple transfer study. However, when I see Table 6, I am confused. Did you train from scratch a network with the L2R loss?
Minor:
- Line 073, you state that the gap from supervised training is determined by but you forget to mention that the denoiser has input (instead of for supervised learning), which could also introduce a non-negligible gap.
- Notations: Line 065, you introduce the notations , , , , without specifying what they represent or which spaces they belong to. Even though those notations are quite common in the image denoising literature, you should define them properly when using them for the first time. By the way, for the sake of clarity, it is common to write vectors in bold (for example, , , , ), to better distinguish them from scalar quantities such as or . In Equation (3), you do not define what is . You should mention that .
- For the sake of clarity, you could mention that (1) is obtained by developing the squared norm.
- Typos: line 229 and 235, equation 11 -> Equation 11 and line 231, equation 12 -> Equation 12
Questions for authors
-
In Equation 1, you consider the symmetric re-corruption pair but in the original R2R paper, the pair is considered instead with for extra flexibility. Why don't you consider this parameter as well from the very beginning? You call this hyperparameter afterwards in equation (26).
-
I didn't get why you need a stop-gradient in your implementation? Can you clarify please.
-
Instead of considering (21) as learning objective, the generator objective is instead (25). Can you explain why simply considering (21) does not work? As you say, "Overall, the two stages are deliberately decoupled", while theory suggests that they should be coupled according to (21). Can you discuss this point?
-
What happens when you iterate the process in the spirit of DCD-Net, that is replacing by the refined denoiser you have just learned and so on?
-
Why not considering a blind-spot network for instead, as in [a], potentially more robust, the latter being based on less restrictive assumptions?
-
Is the with white Gaussian (34.96 dB) in Table 5, the same as R2R in Table 2 (35.04 dB)?
-
What do you mean by "R2R plain" in Figure 3? Does it correspond to ( / White Gaussian) in Table 5? If so, could you also provide a visual comparison with ( / NLF and ISP)?