acceptodds
Withdrawn from ICLR 2027

LoTA-N2N: Local Trace Adaptation for Zero-Shot Self-Supervised Image Denoising

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Many self-supervised image denoising methods minimize the mean squared error (MSE) between denoiser predictions and surrogate targets constructed from noisy observations. They justify this training under different assumptions, such as suitable noise independence and unbiasedness, weak corruption, clean-neighbor similarity, or a compatible corruption model. We revisit these training objectives and show that self-supervised MSE differs from supervised MSE by a parameter-independent constant and a trace term. From this perspective, existing assumptions eliminate or suppress the trace term, narrowing the gap between self-supervised and supervised objectives up to a constant. This helps explain why training with surrogate targets generated from noisy images can yield effective denoising. Motivated by this understanding, we propose to estimate and penalize its magnitude alongside reconstruction, reducing reliance on assumptions that suppress it implicitly. However, a small global trace can conceal large regional discrepancies through cancellation of positive and negative contributions. We therefore introduce LoTA-N2N, a two-stage zero-shot denoising framework with local trace correction. A denoiser first learns from complementary sub-images of a single noisy observation and is frozen to provide clean-image proxies. These proxies enable interaction estimates whose absolute magnitudes are penalized region by region during adaptation. We prove that this local penalty prevents cancellation between regions and yields a supervised-risk upper bound with slack controlled by teacher-estimation error. Experiments demonstrate improvements over iteration-matched MSE adaptation under independent, spatially varying, and mixed noise.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Self-supervised, Image Denoising

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1ZdCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a unified view of self-supervised methods for image denoising, highlighting the role played by the so-called trace term that needs to be close to zero. Based on this trace-decomposition, the authors construct a two-stage single-image denoiser named LoTA-N2N that penalizes the trace term. They show results on synthetic Gaussian, Poisson and mixed Poisson-Gaussian noise.

Critical strengths

This paper tries to give a unified framework of self-supervised methods for image denoising, advocating all methods are linked by the fact that the so-called trace term is zero (or close to zero) whatever the parameters θ\theta are (section 3).

Critical weaknesses

Major:

  1. I don't believe in the numerical values in Table 1. For example, the PSNR values of BM3D and DnCNN are under-estimated when compared to the ones given in [a]. The gap between the supervised approach (DnCNN here) and N2N should be close to zero, see for example [b]. Why does the performance of DnCNN collapse on Poisson noise when evaluating on McMaster18 instead on Kodak24? All of this is aberrant. Please clarify. Moreover, it is well-established that BM3D outperforms ZS-N2N on both Gaussian noise and Poisson noise with appropriate variance-stabilizing transformation, see for example [c]. Given the close scores between ZS-N2N and your method, I cannot believe that LoTA-N2N beats BM3D, contrary to what Table 1 suggests. Same holds for Table 8 where the performance of Self2Self is under-estimated (see [c, d]). Indeed, the performance of Self2Self should be way higher than ZSN2N on Gaussian noise, not 2dB lower... In summary, the clear aberrant PSNR values in Table 1 and 8 are very suspicious to me and the authors have lost my trust. This is a sufficient reason for me to reject this paper.

  2. The method is very close to ZSN2N both in terms of construction and experimental results (see Table 1 and Figure 3 and 10 where the same denoising artifacts are produced). The problem is that ZSN2N is already far from being state-of-the-art, see [c] for example.

  3. The paper is not really well written and hard to follow. For example, some notations are used without being defined, or sometimes are defined further, etc.

[a] Zhang, K., Zuo, W., & Zhang, L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE TIP, 2018.

[b] Wang, Z., Liu, J., Li, G., & Han, H. Blind2unblind: Self-supervised image denoising with visible blind spots. CVPR, 2022.

[c] Herbreteau, S., & Unser M. Self-Calibrated Variance-Stabilizing Transformations for Real-World Image Denoising. ICCV, 2025.

[d] Quan, Y., Chen, M., Pang, T., & Ji, H. Self2self with dropout: Learning self-supervised denoising from single image. CVPR, 2020.

Minor:

  1. Line 150, you define u\boldsymbol{u} as a self-supervised input. Do you mean the noisy input image? If so, I recommend renaming it as y\boldsymbol{y} which is the common notation in image denoising. But, from what I understand, you want to encompass recorrupted noisy images as in R2R with u=y+z\boldsymbol{u} = \boldsymbol{y} + \boldsymbol{z} for example. The problem is that, in this case, the supervised loss is not ∥eθ(u)∥22\Vert \boldsymbol{e}_{\theta}(\boldsymbol{u}) \Vert_2^2 =

∥fθ(y+z)−x∥22\Vert f_\theta(\boldsymbol{y} + \boldsymbol{z}) - \boldsymbol{x} \Vert_2^2 but rather ∥fθ(y)−x∥22\Vert f_\theta(\boldsymbol{y}) - \boldsymbol{x} \Vert_2^2, so that the self-supervised and supervised minimization do not agree whenever T(θ)=0,∀θ\mathcal{T}(\theta) =0, \forall \theta, contrary to your claim line 158. Please clarify.

  1. The trace has a precise meaning in linear algebra (sum of the diagonal elements of a square matrix). Since you use the same denomination to denote the quantity defined in (3), I recommend warning the reader about it to avoid potential confusion.

  2. Are u\boldsymbol{u}, x\boldsymbol{x}, r\boldsymbol{r} matrices? Generally, an image is interpreted as a vector. Therefore, you do not need the Frobenius norm and Frobenius inner product in equations (1)-(3).

  3. Line 834, you say that "k1\boldsymbol{k}_1 and k2\boldsymbol{k}_2 average the anti-diagonal and diagonal pixels of each non-overlapping 2×2 block". However, this does not seem to be the case in Figure 2. Can you please clarify?

  4. In equation (4), you do not define the quantities JJ, B\boldsymbol{B}, etc... As these notations are more or less introduced in the following paragraphs, I recommend to transform equation (4) into a Figure instead for improved clarity.

  5. In Figure 2, MSE vs y2y_2 is written twice in Stage 1. I think this is a typo.

  6. Appendix C should probably be moved to the main paper.

  7. I don't understand equation (30). What is LMSE(θ)L_{MSE}(\theta)? Is LMSE(θ)=E∥fθ(y)−x∥22L_{MSE}(\theta) = \mathbb{E}\Vert f_{\theta}(y) - x\Vert_2^2?

  8. Line 873, what is exactly Jθ,iJ_{\theta, i}? Why does it depend on ii? What is ∥.∥op\Vert .\Vert_{op}?

  9. Please use correcty \citet and \citep when citing in your Latex file.

Questions for authors
  1. Why do you need differentiation and expectation to be interchanged to write equation (8)? Isn't it a trivial consequence of equation (2)? If I understand correctly, when T(θ)=0\mathcal{T}(\theta) = 0 for all θ\theta, then ∇Lsup=∇Lss\nabla L_{sup} = \nabla L_{ss} and there is no gradient-gap (of course in practice, it is not true since we use instead an empirical approximation of the expected value, see [e] for example).

[e] Klug, T., Atik, D., & Heckel, R. Analyzing the sample complexity of self-supervised image reconstruction methods. NeurIPS, 2023.

  1. As mentioned line 249, "because neighboring clean pixels may differ and their noise may be correlated, the resulting pair need not satisfy N2N assumptions". What solution do you propose for correlated noise?
Reviewer_JZy4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoTA-N2N, a zero-shot self-supervised image denoising framework based on local trace correction. The paper interprets the gap between self-supervised and supervised MSE through a residual–prediction interaction term. A frozen teacher is used to estimate this interaction, and its absolute magnitude is penalized within local regions to prevent cancellation between positive and negative interactions. Experiments on synthetic Gaussian, Poisson, mixed, spatially varying, and cross-domain corruptions show improvements over iteration-matched MSE adaptation.

Critical strengths

This work targets the critical surrogate-clean target mismatch in single-image zero-shot denoising. Its trace decomposition compactly unifies N2N, blind-spot, sub-image pairing and re-corruption paradigms. Using local absolute values to prevent global cancellation is clearly motivated. The method adds no trainable parameters and generalizes across different backbones.

Critical weaknesses
  1. Theorem 1 establishes a bound on the population-level sub-image risk, but does not directly guarantee the risk of the final full-image output. Moreover, the finite-sample estimation error and the assumed teacher-proxy error are not sufficiently controlled, limiting the theoretical justification of the proposed method.
  2. The experiments rely predominantly on synthetic noise, with limited evidence under correlated noise and no sufficient validation on real-noise benchmarks.
Questions for authors

1.Theorem 1 bounds the expected supervised risk of the complementary sub-image objective. It does not establish a bound for the final full-image inference risk \thea(y). Moreover, the practical loss is computed from a single noisy image and a finite number of regions, while the theorem is written using population expectations. The finite-sample estimation error is not theoretically controlled. The abstract and introduction should clearly state that the theorem concerns population-level sub-image risk and objective mismatch, rather than claiming a general supervised denoising guarantee for the final full-image output.

2.The experiments are conducted almost entirely on synthetic additive corruptions, while real sensor noise is often signal-dependent, spatially correlated, and non-stationary. Since the proposed sub-image construction and local trace estimation may be sensitive to these correlations, the limited improvement under correlated Gaussian noise in Table 4 raises concerns about real-world transferability. Evaluation on a real-noise benchmark such as SIDD or DND, or a clearer limitation of the paper’s scope, would strengthen the claims.

3.The method relies on a frozen teacher to generate a pseudo-clean proxy, but the teacher is trained from the same noisy observation used for adaptation. Hence, the proxy may retain noise components correlated with the input; stop-gradient prevents gradient flow but does not make the proxy an independent target. The theoretical error bound \epsilon is assumed but neither estimated nor validated on real noisy data. The paper should provide multi-crop or repeated-sampling stability analysis, performance under different teacher errors, and proxy-quality evaluation on real-noise data. The current synthetic checkpoint analysis is useful but insufficient.

Reviewer_9WCJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the gap between self-supervised and supervised MSE denoising objectives through a residual–prediction interaction T(θ)T(\theta), which the paper interprets as a trace term. The formulation is presented from a relatively general perspective: under ideal assumptions, T(θ)T(\theta) can vanish for standard N2N or blind-spot supervision, while violations of these assumptions can leave a nonzero interaction. In practice, LoTA-N2N instantiates this idea in the more specific setting of ZSN2N-style sub-image pairing, where neighboring clean pixels are not identical and the corresponding noises may also remain dependent, so a nonzero T(θ)T(\theta) can arise from the sub-image construction itself. The method uses a two-stage pseudo-clean estimation scheme: the Stage-1 denoised output is used as a pseudo-clean proxy, and Stage 2 fine-tunes the model with MSE together with a local residual–prediction interaction penalty. Specifically, the global interaction is decomposed into regional terms, and the triangle inequality is used to upper-bound the absolute global interaction by the sum of their absolute values, which motivates the patch-wise absolute penalty and avoids cancellation between regions. The associated supervised-risk bound further connects this local regularization to the supervised objective while accounting for proxy-estimation error.

The method shows consistent improvements over iteration-matched ZSN2N training under several controlled noise settings. The general formulation is potentially interesting beyond this specific sub-image setting. A nonzero residual–prediction interaction can also arise when the ideal assumptions behind N2N or blind-spot supervision are violated by real-world noise, and using a pseudo-clean estimate to correct this interaction is a reasonable idea. If this can be done reliably, the framework could become promising for real-world denoising as well. However, this broader potential is not yet demonstrated: the current experiments mainly focus on sub-image mismatch, comparisons with newer zero-shot methods are limited, and no standard real-world sRGB benchmark such as SIDD is included.

Critical strengths

A main strength is that the paper turns the residual–prediction mismatch into an explicit trainable regularizer. Instead of only considering the global interaction T(θ)T(\theta), it estimates the interaction locally using pseudo-clean proxies and penalizes its absolute value over patches, which helps avoid cancellation across spatial regions. This provides a simple and practical way to explicitly reduce the mismatch in per-image self-supervised denoising.

Critical weaknesses
  1. The empirical validation is narrower than the general formulation.
    The paper presents T(θ)T(\theta) as a general mismatch term for MSE-based self-supervision, but the proposed correction is mainly demonstrated on ZSN2N-style sub-image pairing. It remains unclear whether the same idea can help when the assumptions behind N2N or blind-spot methods fail under real-world noise. No standard real-world sRGB benchmark such as SIDD is included, and comparisons with stronger recent zero-shot or pseudo-pair methods are also limited. These experiments would be important for determining whether the proposed correction is a general principle or mainly benefits the particular sub-image construction used here.

  2. The role of the "trace" formulation in the actual method is limited.
    Although the paper frames the residual–prediction interaction as a trace term, the proposed method itself operates on local Frobenius inner products and their absolute values. I do not see any trace-specific property being used beyond the equivalence between the Frobenius inner product and Tr(A⊤B)\mathrm{Tr}(A^\top B). The main methodological contribution therefore seems to come from the pseudo-clean interaction estimation and local absolute regularization, rather than from the trace formulation itself.

Questions for authors
  1. Generality beyond sub-image pairing.
    The proposed correction does not seem inherently specific to sub-image pairs: in principle, once a pseudo-clean estimate is available, the residual–prediction interaction could also be estimated for other MSE-based self-supervised constructions. This seems particularly interesting for real-world denoising, where the assumptions behind N2N, blind-spot, or re-corruption methods may be violated and T(θ)T(\theta) can remain nonzero. Could the authors clarify whether LoTA is intended to extend to these settings? Comparisons or integration with representative recent real-world methods, such as Positive2Negative [1] or BSN-NLR [2], on a benchmark such as SIDD would substantially strengthen the general claim.

  2. Comparison within pseudo-pair methods.
    Even within the current pseudo-pair setting, the empirical comparison is limited. Pixel2Pixel [3] is already discussed in the paper, and recent methods such as ZS-HPD [4] use stronger local/non-local strategies to construct pseudo samples. Does a significant T(θ)T(\theta) remain for these stronger pair constructions, and can the proposed local correction further improve them? This would help separate the benefit of the proposed correction from the particular ZSN2N-style pair construction used in the current implementation.

References

[1] (2025). Positive2Negative: Breaking the information-lossy barrier in self-supervised single image denoising. CVPR.

[2] (2025). Zero-shot blind-spot image denoising via cross-scale non-local pixel refilling. NeurIPS.

[3] (2025). Pixel2Pixel: A pixelwise approach for zero-shot single image denoising. IEEE TPAMI.

[4] (2026). Zero-shot image denoising via hybrid prior-guided pseudo sample generation. CVPR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.