- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the convergence of plug-and-play (PnP) image restoration algorithms under annealed denoising noise levels. The authors establish asymptotic stationarity guarantees for several deterministic and stochastic PnP variants, including RED-GD, PnP-PGD, SNORE, and ERED, under appropriate regularity and structural assumptions. Stronger convergence results are derived under additional conditions on the annealing schedule. The theoretical analysis is complemented by experiments on several image inverse problems, including inpainting, demosaicing, super-resolution, and tomography, demonstrating the potential benefits of noise annealing.
Critical strengths
-
Understanding the convergence behavior of PnP algorithms under varying denoising noise levels is an important question, particularly because annealing is frequently used in practice without corresponding convergence guarantees. The paper provides a relatively systematic analysis across multiple deterministic and stochastic formulations, extending the theoretical understanding of these methods.
-
The authors carefully distinguish asymptotic stationarity from stronger convergence guarantees and address nontrivial issues involving boundedness, stochastic updates, and the regularity of the underlying denoising operators. These technical considerations strengthen the theoretical contribution and go beyond a straightforward extension of fixed-noise convergence results.
Critical weaknesses
- The practical relevance of the theoretical guarantees remains insufficiently established due to restrictive and unverified assumptions.
The convergence analysis relies on strong global regularity and structural assumptions on the denoising operators and their associated objectives. However, the experiments employ learned denoisers whose compliance with these assumptions is not rigorously established. The empirical spectral regularization and finite-sample diagnostics used in the paper do not constitute guarantees that the required properties hold globally.
Although the authors discuss theoretical modifications intended to enforce certain regularity conditions, it remains unclear whether these modifications preserve all the structural properties required by the different convergence results or whether the modified operators correspond to those actually used in the experiments. This distinction is particularly important for the stochastic formulations, where the theoretical objective may depend on the denoiser's behavior beyond the region explored by finite experimental trajectories.
Consequently, the paper establishes convergence for a mathematically restricted class of operators, but does not convincingly demonstrate that its guarantees apply to the practical learned-denoiser algorithms used to motivate and validate the work. This substantially limits the practical implications of the theoretical results.
- The experimental evaluation does not directly validate the paper's central convergence claims or adequately isolate the effect of annealing.
The empirical evaluation primarily focuses on reconstruction quality, objective values, and differences between consecutive iterates. While these measurements are useful for assessing numerical behavior, they do not directly establish convergence to stationary points of the limiting objective, which is a central theoretical claim of the paper. In particular, diminishing update magnitudes can arise from decreasing step sizes without necessarily demonstrating the claimed stationarity.
Furthermore, the constant-noise and annealed configurations do not always optimize the same limiting objective, making it difficult to determine whether the observed improvements result from the annealing trajectory itself or from differences in the effective optimization objectives and hyperparameter configurations.
The experiments also predominantly employ rapidly decaying annealing schedules, leaving the broader theoretical conclusions concerning less restrictive annealing conditions largely unexplored. As a result, the empirical evidence demonstrates that annealing can improve reconstruction performance in certain settings, but provides limited direct validation of the distinctive convergence guarantees developed in the paper.
Questions for authors
-
To what extent do the theoretical convergence guarantees actually apply to the learned denoisers used in the experiments? The required global regularity and structural assumptions are not established by finite-sample diagnostics or empirical spectral regularization. Could the authors clarify which theoretical assumptions are rigorously satisfied by the implemented algorithms, and whether the proposed regularity-enforcing modifications preserve the additional structural properties required by the convergence analysis?
-
Can the authors provide direct empirical evidence supporting convergence to stationary points of the limiting objective? The current experiments mainly report reconstruction quality, objective values, and iterate differences, which do not directly verify the central stationarity guarantees. Could the authors evaluate appropriate stationarity residuals and demonstrate that the observed convergence behavior is consistent with the theoretical predictions, particularly for the stochastic variants?
-
Can the authors disentangle the benefits of annealing from changes in the limiting objective and optimization configuration? Since the constant-noise and annealed settings do not always share the same final noise level and effective objective, the reported performance improvements cannot be attributed solely to the annealing trajectory. Could the authors provide comparisons under matched limiting objectives and clarify whether the corresponding hyperparameters were independently optimized?
5 comments
W2/Q2
Can the authors provide direct empirical evidence supporting convergence to stationary points of the limiting objective?
We thank the reviewer for this suggestion. All the Figures now also report, for each algorithm, the normalized terminal stationarity residual
{\|\nabla F_\varepsilon(x_0)\|}$$ During the initial phase, when $\sigma_k\gg\varepsilon$, this gradient norm remains high, since the algorithm follows a different objective $F_{\sigma_k}$. Once $\sigma_k$ approaches $\varepsilon$, it decreases toward zero. This terminal gradient norms decrease in line with the theoretical best-iterate rates: $O(1/\sqrt{k})$ for the deterministic algorithms and $O\left(1/\sqrt{\sum_{j=0}^{k}\gamma_j}\right)$ for SNORE and ERED. This supports our stationarity results even when the stepsizes decrease: the gradient norms decrease too, so the small updates are not just a consequence of $\gamma_k\to0$. **Q3** > Can the authors disentangle the benefits of annealing from changes in the limiting objective and optimization configuration? [...]Could the authors provide comparisons under matched limiting objectives and clarify whether the corresponding hyperparameters were independently optimized? We thank the reviewer for this suggestion. In the original comparisons, the constant-noise and annealed configurations were independently tuned on the validation set to maximize PSNR. In the revised manuscript, the different Figures additionally compare annealing against constant $\sigma=\varepsilon$ (visual results and convergence curves). Both configurations share the same limiting objective but the constant-$\varepsilon$ configuration always produces worse reconstructions. For example on Figure 2, the large inpainting hole remains unfilled, and on Figure 3 substantial artifacts persist in demosaicing. These observations confirm the observation from introduction: constant small denoising level is insufficient to correct large scale degradations. Starting at a larger noise level with annealing enables to handle large structural degradations before reducing the denoising strength to recover finer details. > The experiments also predominantly employ rapidly decaying annealing schedules, leaving the broader theoretical conclusions concerning less restrictive annealing conditions largely unexplored. [...] We added in Appendix C.4 comparisons, on inpainting and demosaicing experiments, between the geometric schedule used in the paper and "polynomial schedules" $$\sigma_k=\epsilon+(\sigma_0-\epsilon)(1+k/k_0)^{-\beta}$$ These schedules were advocated for diffusion sampling by Karras et al (2022). The geometric schedule is recovered in the limit $\beta\to\infty$. For all these schedules, the terminal objective $F_\epsilon(x_k)$ levels off and $|\nabla F_\epsilon(x_k)|$ decreases, as predicted by our convergence results. Near the floor, the gradient norm decreases more slowly for smaller $\beta$. This is consistent with our convergence theory which proves $|\nabla F_{\sigma_k}(x_k)-\nabla F_\varepsilon(x_k)|=O(\sigma_k-\varepsilon)$. Concerning reconstruction quality, we observe that inverse problems with large missing regions, such as large-hole inpainting, need enough iterations at large $\sigma$: small $\beta$ fails to fill the hole, while larger $\beta$ matches the geometric schedule. For demosaicing, which does not have such a lare missing region, all schedules perform similarly. We thank the reviewer for their constructive feedback. We believe that the resulting revisions have significantly strengthened our paper and hope that our responses and additional results have addressed the reviewer's concerns. **References** Stanley H. Chan, Xiran Wang, and Omar A. Elgendy. Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging. 2017. Samuel Hurault, Arthur Leclaire, and Nicolas Papadakis. Proximal denoiser for convergent plug-and-play optimization with nonconvex regularization. ICML 2022. Karras, Tero, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. NeurIPS 2022.Thank you for the detailed responses and additional experiments. However, I believe several fundamental concerns remain unresolved.
-
The gap between theoretical guarantees and practical PnP implementations remains substantial. The authors acknowledge that proximality in the experiments is assessed only through empirical Hessian spectral norms on validation data, which cannot establish the required global condition. The theoretical existence of a sufficiently small relaxation parameter does not imply that the experimentally chosen value () satisfies this condition. Consequently, whether the PnP convergence guarantees actually apply to the implemented algorithms remains unestablished.
-
The additional stationarity experiments do not fully justify the claimed theoretical convergence rates. The theoretical best-iterate rates generally concern the moving objective , whereas the reported curves show the gradient norm of the terminal objective . These quantities are not interchangeable without additional error bounds. More importantly, for the stochastic variants, the theoretical rate bounds concern expected best-iterate gradients of the moving objective, while terminal stationarity is established asymptotically almost surely. Thus, the observed finite-iteration decay does not, by itself, establish the claimed rate behavior. The distinction is particularly important when interpreting decreasing update magnitudes in stochastic algorithms.
-
The additional matched-objective experiments do not fully disentangle the benefits of annealing. Comparing annealing with constant only demonstrates that starting directly with a small denoising level is ineffective. It does not establish that annealing outperforms the best fixed-noise configuration. Conversely, the independently tuned constant- baseline optimizes a different objective. Therefore, the experiments still do not cleanly isolate the contribution of the annealing trajectory from the effects of objective selection and hyperparameter tuning.
Overall, the revisions improve the experimental presentation, but they do not resolve the fundamental gap between the theoretical assumptions and the practical algorithms, nor do they fully establish the empirical claims made in support of the theoretical contributions. In particular, the global proximality required for the PnP convergence analysis remains unverified in the actual experiments. I therefore maintain my original rating of 2 (Weak Reject).
We thank the reviewer for recognizing the relevance of our analysis. We address each comment and question below. We have also uploaded a revised version of the manuscript, with changes in blue.
W1/Q1
First, we emphasize that our assumptions are sufficiently general to cover exact MMSE denoisers, unlike some existing convergence analyses of annealed PnP that rely on more restrictive assumptions, such as the bounded-denoiser condition of Chan et al. (2017) or the log-concavity assumption of Pesme et al. (2025). This is particularly important since trained denoisers are typically designed to approximate MMSE denoisers. We believe that establishing convergence guarantees in this setting is an important theoretical contribution. Beyond this, we explicitly design our trained denoisers to satisfy the structural assumptions of our convergence analysis, rather than relying solely on their approximation of MMSE denoisers.
More precisely:
For RED-GD and SNORE, the learned gradient-step denoiser Dσ=Id−∇gσ satisfies the assumptions required for convergence: Appendix A.1 establishes the required uniform Lipschitz continuity of ∇gσ and coercivity of gσ, after regularization outside an arbitrarily large box B.
For PnP-PGD and SNOPnP, the denoiser must be a proximal operator. Our construction builds on Hurault et al. (2022). A sufficient condition for the gradient-step denoiser to be a proximal operator is Lg<1, where Lg uniformly bounds the Lipschitz constants of ∇gσ. We use the following relaxation:
=\mathrm{Id}-\rho\nabla g_\sigma.$$ for which the sufficient condition becomes $\rho L_g<1$. Since $L_g$ is provably finite, **there always exists a sufficiently small $\rho>0$ satisfying this condition**. The soft spectral penalty complements this approach by encouraging smaller $L_g$, thereby reducing the need for strong relaxation. In practice, we estimate $L_g$ from Hessian spectral norms on validation images and select $\rho$ accordingly. These estimates do not certify a global bound on $L_g$, and hence do not rigorously guarantee proximality for the experimental choice of $\rho$. We explicitly acknowledge this limitation in conclusion and in Appendix C.1. > It remains unclear whether these modifications preserve all the structural properties required [...] or whether the modified operators correspond to those actually used in the experiments. We understand this concern as now referring to the regularization of the learned potential proposed in Appendix A.1, equation (17). This construction modifies the learned potential outside a prescribed box $\mathcal B$ to ensure global Lipschitz continuity of its gradient and coercivity. This modification preserves the gradient-step structure of the denoiser, and proximality can still be guaranteed through sufficiently strong relaxation, as explained above. Moreover, **the box $\mathcal B$ can be chosen arbitrarily large**. The regularization leaves the potential unchanged inside $\mathcal B$, and therefore does not affect denoiser evaluations occurring within this region. In practice, we observe that all deterministic and stochastic iterates of our experiments remain within $\mathcal B=[-10,10]^d$, so this choice of $\mathcal B$ do not change the iterates. In general, for Gaussian-perturbed inputs, the probability of leaving $\mathcal B$ during a finite run can also be made arbitrarily small by enlarging $\mathcal B$. > Could the authors clarify which theoretical assumptions are rigorously satisfied by the implemented algorithms, and whether the proposed regularity-enforcing modifications preserve the [...] properties required by the convergence analysis? To summarize, the Assumptions 3.3 for RED-GD and SNORE convergence is satisfied. For PnP algorithms, proximality of the denoiser can be guaranteed through a sufficiently small relaxation parameter. In practice, the relaxation parameter is selected from empirical spectral estimates.