acceptodds
Under review as a conference paper at ICLR 2027

From Gradient Magnitude to Level-Set Geometry: A 2nd-Order Aligned Regularizer for 1st-Order Image Priors

Abstract

Natural images exhibit coherent spatial organization of local variations, particularly in gradient orientations along contours and smoothly varying structures. Image corruption can disrupt this organization, causing neighboring gradient orientations to fluctuate irregularly even when individual variations remain plausible. In image denoising, 1st-order regularizers control local variation but do not explicitly constrain how these variations are organized across space. This raises a question: can spatial organization be regularized without introducing a preference that conflicts with the original 1st-order prior? In this paper, we show that it can be done using the first variation of a 1st-order prior. This defines a spatial response on the level-set geometry whose squared norm, the 2nd-order Aligned Regularizer (SOAR), penalizes how the normal profile and the level-set curvature vary across the image. In a local tangential Fourier picture, SOAR adds a 4th-order spectral response on top of the 2nd-order spectral response of the parent prior, acting as an additional low-pass filter on geometric fluctuations while preserving the parent prior's stationary points. For denoising, we combine image-adaptive weighting of the 1st-order regularizer with selective SOAR activation. The resulting single-image zero-shot denoising method consistently improves over the 1st-order baseline and achieves the highest PSNR on real-world benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image denoising, variational regularization, higher-order regularization, single-image denoising

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ibPQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SOAR, which uses the squared variational derivative of a first-order image prior JϕJ_\phi, Kϕ(u)=12∥δJϕ/δu∥L22K_\phi(u)=\frac12\|\delta J_\phi/\delta u\|_{L^2}^2, as an additional regularizer. It decomposes the variational response into the change in the magnitude of the intensity transition and the curvature of the level lines (isophotes), and shows, in a local edge model, that SOAR adds a k4k^4 response to the existing prior's k2k^2 response, thereby suppressing fast edge oscillations more strongly. Taking into account both the noise reduction from the extra smoothing and the loss of true structure, it also gives a local gain condition that holds when the difference in directional roughness is sufficiently large.

The actual algorithm first restores the image with DIP+TV whose strength is adjusted per image, and then applies the TV-derived SOAR only when the relative directional-roughness difference between the observed image and the TV result exceeds a threshold. In center-crop evaluations on PolyU, SIDD-Small, and FMD, it improves the mean PSNR over adaptive DIP+TV by about 0.043, 0.453, and 0.236 dB, respectively, and also improves SSIM. Among the reported baselines, its PSNR is the best on all three datasets, while its SSIM ranks first on PolyU and second on SIDD and FMD. Thus, the empirically demonstrated contribution is, together with the proposal of a general construction principle, the improvement of a system that combines SOAR with a selection procedure in TV-based restoration.

Critical strengths
  1. It clearly connects a concise regularizer construction with an interpretable geometric analysis. SOAR is defined from the variational response of the chosen prior, and Eq. (2) decomposes this response into the change of the intensity profile and the curvature. The logical flow from the problem setup to the variational decomposition, the local edge model, and selective application is clear. The local model explains why the additional term suppresses the high-frequency components of edge displacement more strongly, as well as the trade-off that it may also deform true structures. The appendix distinguishes the preservation of the prior's minimizers from the solution of the full restoration problem, and explicitly states the scope of the analysis: the local theory does not guarantee the success of the practical selection rule. This analysis is valuable in that it goes beyond simply adding a higher-order term and tries to interpret when it can help. (§3.1, Appendices A.1–A.3)

  2. It demonstrates both improvement over a controlled baseline and the practical utility of the selection rule. Compared with the same adaptive DIP+TV, mean PSNR and SSIM improve on all three datasets. Table 4 directly compares the gain from applying SOAR to selected versus excluded images: the mean PSNR change is +0.665 dB vs. −0.448 dB on SIDD, and +0.378 dB vs. −0.933 dB on FMD. A permutation test against random selection with the same number of activations also reports a significant separation of PSNR gains on all three datasets (10510^5 permutations, p≤10−3p\le10^{-3}). This is concrete evidence that the selection rule identifies images that benefit, on the evaluation data. It matters that the paper experimentally confirms why selection is needed, rather than only reporting an overall average improvement. (Tables 1, 3–4)

Critical weaknesses
  1. The contributions of the SOAR term itself and of the selection strategy to the final advantage are not sufficiently disentangled. Table 3 compares always-on SOAR with the Laplacian and Euler–Elastica, so an ablation of the term itself is not absent. I also acknowledge that the strengths of the existing regularizers were set via the search in Appendix B before comparison. With SOAR's default setting r=12r=12 in Table 3, always-on SOAR does increase PSNR on SIDD, but it lowers SSIM on all three datasets and lowers PSNR on FMD by about 0.113 dB. Under this setting, the selection procedure must be combined for both metrics to improve over adaptive DIP+TV on all three datasets. However, there are no results applying the same selection procedure to the existing regularizers. If the existing terms also achieve similar gains when applied only to favorable images, a substantial part of the current results could be explained by the strategy of selective higher-order regularization rather than by the way SOAR is derived. A comparison under the same DIP+TV, the same selection rule, and equivalent strength tuning is needed to evaluate the unique utility of the proposed term more clearly. This does not negate the improvement of the final system; rather, it is a lack of the controls needed to judge the size of the core contribution. (Table 3, Appendix B)

  2. The configuration process and validation scope of the key selection rule are limited. The local theory is formulated with respect to the clean structure, but the actual algorithm uses the TV result as a proxy and decides whether to apply SOAR with a threshold of 0.9 on the relative roughness difference. Because the score can also be large when TV has erased true structure, this rule is an empirical approximation. The selection rule does have motivation from the local theory and experimental support, and the results of applying the same setting to the three datasets and the sensitivity analyses in Tables 5 and 8–9 are positive. However, these analyses alone do not reveal on which data and by which criterion the final threshold, regularization strength, and weighting constants were determined, nor whether the data used for configuration were separated from the final evaluation data. Therefore, the concern is not the use of a heuristic or a lack of empirical support, but that the unclear configuration process and independent validation make it difficult to judge generalization to new conditions. The average gain of the selected group alone does not reveal how often individual images get worse or how accurate the selection is under new conditions. The three benchmarks cover different real capture environments, but there is no analysis that separates noise type, strength, and spatial correlation. Comparing noise that produces irregular directional fluctuations with directional corruption such as stripes would have given a deeper understanding of the operating conditions of the proposed geometric explanation and selection rule. This is a suggested analysis to make the operating conditions of the proposed mechanism concrete, not a request for performance on every noise type. The restoration experiments are also limited to smoothed TV and a single DIP architecture. While the theoretical construction is general, the scope supported by the current empirical evidence is selective restoration in this setting. This scope and the configuration procedure need to be clearer in order to assess the reproducibility and generality of the contribution, including the selection algorithm. (§3.1, §4.1, Tables 4–5, 8–9, Appendix B.1)

Questions for authors
  1. Comparison to isolate the unique effect of SOAR: How do the results change if the same G^\widehat G-based selection rule is applied to the Laplacian and Euler–Elastica in Table 3? I would like to see whether SOAR's advantage holds when DIP+TV and the optimization settings are fixed and each regularizer's strength is given an equivalent tuning opportunity. This is not a request to newly develop every alternative method, but a question to separate the roles of the proposed term and of the selection strategy in the current final performance.

  2. Configuration and reliability of the selection rule: I have checked the sensitivity analyses in Tables 5 and 8–9. Separately from those, could you explain on which data and by which criterion the final settings — τ^=0.9\widehat\tau=0.9, μ=λ/12\mu=\lambda/12, and the constants of the adaptive TV weight — were determined? Please also clarify whether the data used for configuration were separated from the final evaluation data. In addition to the averages in Table 4, can you report the proportion of selected images whose PSNR or SSIM gets worse, and by how much? I acknowledge that the same rule was used on all three datasets, but this information is needed to judge the reliability of the selection and the risk of losing true structure.

Side note — optional research direction: Interestingly, the objective E+μ∥∇E∥2E+\mu\|\nabla E\|^2 discussed in §4 of Barrett and Dherin's Implicit Gradient Regularization has a mathematical construction similar to SOAR. It is intriguing that this structure appears in problems as different as model training and image restoration. Exploring the connection between the two perspectives may yield additional interpretations or ideas for extensions. This is not a request for a comparison or an answer needed for my evaluation, but a comment on a possibility that may be worth exploring in the future.

1 comment

We thank the reviewer for the feedback and the suggestions.

Q1. SOAR vs. selective activation

To isolate the contributions of the regularizer and selective activation, we apply the same activation rule ((\widehat G>0.9)) to Laplacian, Euler–Elastica (EE), and SOAR, keeping the DIP+TV baseline, optimisation settings, and selected images identical.

MethodActivationPolyUSIDDFMD
LaplacianAlways-on38.58/.963336.35/.913633.00/.8568
Selective38.56/.963436.33/.913732.99/.8574
EEAlways-on38.60/.963536.48/.912533.02/.8544
Selective38.58/.963536.49/.915233.11/.8614
SOARAlways-on38.60/.962736.62/.904432.90/.8418
Selective38.60/.963636.76/.915333.25/.8613

PSNR (dB) / SSIM. We retain the regularisation ratios from the original experiments (Laplacian: (r=0.5), EE: (r=1), SOAR: (r=12)); the Laplacian and EE ratios were selected through the cross-dataset PSNR sweeps reported in Table 7.

The results show that selective activation also benefits EE, particularly on SIDD and FMD, while its effect on Laplacian is limited. Under the same selection rule, SOAR achieves the highest PSNR across all datasets, with competitive SSIM. This suggests a contribution from the SOAR regularizer beyond the selection strategy alone.

Q2. Configuration and reliability

Parameter selection. The hyperparameters were determined through a global sweep over all images from PolyU, SIDD, and FMDD, based on average PSNR and SSIM. The same images were used for the final evaluation, without a separate validation split. A single configuration was adopted across all three datasets.

Reliability of selected images. We analyse the individual PSNR/SSIM changes among the selected images in Table 4, reporting the proportions showing improvement or degradation and their corresponding mean changes.

DatasetMetricImproved (rate / mean gain)Degraded (rate / mean loss)
PolyUPSNR92.3% / +0.1877.7% / −0.076
SSIM76.9% / +0.0013623.1% / −0.00030
SIDDPSNR67.9% / +1.21732.1% / −0.500
SSIM61.5% / +0.0175038.5% / −0.02010
FMDDPSNR40.0% / +2.14760.0% / −0.801
SSIM40.0% / +0.0582960.0% / −0.02796

Selective activation improves PSNR for most selected images on PolyU (92.3%) and SIDD (67.9%). For FMDD, larger gains on 40% of selected images outweigh the more frequent degradations. While these results support the overall effectiveness of our strategy, they also suggest its limitations and the need for more image-adaptive selection. We appreciate the reviewer’s suggestions on this issue and will take them into account in future revisions.

Reviewer_3GGUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SOAR, a second-order regularization framework designed to address the limitations of conventional first-order regularization while preserving the minimizers of the original prior. By penalizing the squared norm of the variational gradient of the prior, the method aims to suppress high-frequency structural perturbations more effectively without fundamentally changing the underlying prior. The authors further develop a selective activation criterion based on orientation roughness to apply SOAR only when it is expected to be beneficial. Experiments on several real-world image denoising datasets demonstrate improvements over existing zero-shot denoising approaches.

Critical strengths
  • The local geometric analysis offers an interesting perspective on the limitations of first-order regularization. In particular, the analysis suggests that the proposed second-order term can penalize changes in transition orientation in addition to variations in transition strength, leading to a stronger suppression of high-frequency perturbations (scaling as O(ω4)O(\omega^4) rather than O(ω2)O(\omega^2)) under the local curve model. This is a promising theoretical motivation for the proposed approach, although the connection between this local analysis and the original image-domain objective requires further clarification.

  • The experiments on multiple real-world denoising datasets show promising improvements over existing zero-shot denoising methods. In particular, the selective activation strategy appears useful for avoiding quality degradation that can occur when the second-order regularizer is applied indiscriminately.

Critical weaknesses
  • The spectral analysis in Section 3.1 shows that the second-order term in the local curve model penalizes high-frequency perturbations more strongly, with a O(ω4)O(\omega^4) response compared to the O(ω2)O(\omega^2) response of the first-order term. However, the connection between this local model and the proposed image-domain regularizer Kϕ(u)K_\phi(u) in eq. (1) is not sufficiently clear. In particular, it remains unclear to what extent the spectral behavior established under the local curve approximation carries over to the original image-domain objective. A more explicit derivation connecting these two formulations would strengthen the theoretical justification for the proposed method.

  • Similar clarification should be made for Proposition 1, which establishes the preservation of minimizers of the original prior. However, preserving the minimizers of JϕJ_\phi does not necessarily imply that the minimizers of the full reconstruction objective, which also includes the data-fidelity term, remain unchanged. Moreover, the additional regularizer may alter the optimization landscape and the trade-off between data fidelity and regularization. The implications of the stated preservation property for the actual reconstruction problem therefore need further clarification.

  • The proposed activation criterion is motivated by the difference in orientation roughness between a noisy observation and its corresponding clean image. Since the clean image is unavailable at inference time, the method instead uses a TV reconstruction as a proxy, as expressed in eq. (6). However, TV regularization can suppress genuine fine-scale structures and alter edge orientations, meaning that the estimated roughness gap may reflect both noise removal and the loss of image details. Although the experimental results provide evidence that the criterion can identify images that benefit from SOAR on the evaluated datasets, its reliability under different image structures and noise distributions remains unclear.

  • The proposed approach requires an initial TV-based reconstruction followed by the selective application of SOAR. Given the per-image optimization required by DIP, the additional procedure may introduce non-negligible computational overhead. A more comprehensive evaluation of computational cost would help establish whether the reported quality improvements justify the additional cost relative to DIP+TV and other zero-shot denoising baselines.

Questions for authors
  • Connection between the local model and the image-domain regularizer: Could the authors provide a more explicit derivation connecting the local curve model Qμ\mathcal{Q}_\mu to the proposed image-domain objective in eq. (1)? Appendix A.2 seems to provide the connection between the curvature component of the variational gradient and the local model formulation, but the approximation assumes that the normal-profile and cross terms are negligible. Under what conditions is the cross term sufficiently small, and how well do these assumptions hold for practical denoising scenarios?

  • Scope of Proposition 1: Does the preservation guarantee apply only to the minimizers of the original prior JϕJ_\phi, or does it imply any preservation property for the full reconstruction objective that includes data fidelity? Could the authors clarify how adding Kϕ(u)K_\phi(u) affects the minimizers and optimization behavior of the full objective?

  • Reliability and generalization of selective activation: How robust is the activation criterion when the TV reconstruction differs substantially from the underlying clean image, particularly in regions containing fine details or complex contours?

  • Computational cost: Could the authors report the computational costs (e.g., end-to-end runtime) of the complete pipeline, including the initial TV reconstruction and the subsequent SOAR optimization, and compare these costs with DIP+TV and other zero-shot denoising baselines?

2 comments

We thank the reviewer for the feedback and the constructive questions.

Q1. Local analysis vs. actual regularizer

Theoretical connection. We acknowledge that Appendix A.2 does not fully justify the reduction from the image-domain objective in Eq. (1) to the local curve model QμQ_\mu. The decomposition in Eq. (2) is exact, but the local model retains only the curvature contribution of SOAR, neglecting the normal-profile and cross terms. Under the fixed-profile, small-slope assumptions, the curvature contribution produces the k4k^4 response, while the original first-order prior contributes the k2k^2 response. However, these assumptions alone do not establish that the omitted terms are negligible in general.

Practical validation. We analyse all images from PolyU, SIDD, and FMD. On strong edges of clean and DIP+TV images, the profile and absolute cross contributions are below 0.02% and 0.8% of the curvature contribution, respectively. These strong-edge regions contribute only 7–14% of the total SOAR energy across the full image.

The findings support our ``local curvature interpretation'' under suitable conditions, but not general enough for full SOAR. We will clarify this limitation, the scope of the local analysis, and think of a more comprehensive theory in the future revision.

Q2. Proposition 1 and the full objective

Proposition 1 establishes that SOAR preserves the minimizers of the original prior, demonstrating its compatibility with the underlying first-order regularization. It does not imply that the minimizers of the full reconstruction objective remain unchanged.

For Fμ(u)=12∥u−y∥2+λJ(u)+μK(u)F_\mu(u)=\frac12\|u-y\|^2+\lambda J(u)+\mu K(u), where K(u)=12∥∇J(u)∥2K(u)=\frac12\|\nabla J(u)\|^2, adding SOAR changes the stationarity condition to (u−y)+λg(u)+μH(u)g(u)=0(u-y)+\lambda g(u)+\mu H(u)g(u)=0, with g=∇Jg=\nabla J and H=∇2JH=\nabla^2J.

For sufficiently small μ\mu, a nearby solution satisfies uμ=u0−μλ(I+λH0)−1H0(y−u0)+O(μ2)u_\mu=u_0-\frac{\mu}{\lambda}(I+\lambda H_0)^{-1}H_0(y-u_0)+O(\mu^2), where u0u_0 minimizes F0F_0 and H0=H(u0)H_0=H(u_0), assuming sufficient smoothness and local invertibility. Thus, SOAR preserves the prior's minimizers while allowing the full reconstruction solution to change through its interaction with the data-fidelity term.

Q3. Reliability of selective activation

We compare the TV-based gate with a clean-reference gate (threshold 0.9) to assess the effect of TV reconstruction errors.

DatasetTV-based gateClean-reference gate
PolyU38.60 / .963638.60 / .9636
SIDD36.76 / .915336.79 / .9183
FMD33.25 / .861333.22 / .8596

PSNR (dB) / SSIM. Both gates select from the same precomputed reconstructions.

Activation decisions agree on 96.0%, 96.9%, and 97.9% of images, with comparable average performance, suggesting that TV-based gate act similarly to clean-reference, which support our paper's original narrative. However, we acknowledge that this result alone does not fully support our desing: on highly textured SIDD image 140, the TV-based gate activates SOAR while the clean-reference gate does not, resulting in a 1.62 dB PSNR loss. This means on overall stability, there is room for improvement on fine-scale structures.

Q4. Computational cost

We acknowledge the additional computational overhead introduced by this two-stage pipeline. Roughly speaking, our approach is ~50% more expensive. Below we report the average end-to-end runtime (s/image) for the reproduced baselines in Table 1, measured on an NVIDIA L4 GPU.

MethodPolyUSIDDFMDDOverall
MASH22.9822.2320.1322.15
INS29.7330.0830.3130.00
NLR93.0092.7892.4292.79
DIP198.12199.12197.90198.60
DIP+TV199.98198.49202.29199.56
DIP+TV+SOAR253.66337.81330.56309.36
(ZS-HPD is excluded from the runtime comparison because its results were taken from the original paper.)

Thank you for the detailed responses and additional experiments.

I appreciate the clarification that Proposition 1 concerns prior minimizers rather than the full reconstruction objective. However, the question of how this property translates to the actual reconstruction solution remains. The high agreement between TV-based and clean-reference activation provides useful evidence of the criterion's reliability.

The authors' additional analysis supports the curvature-dominated interpretation in strong-edge regions. However, these regions account for only 7-14% of the total SOAR energy, leaving it unclear whether the O(k^4) interpretation characterizes the full SOAR objective more generally. I suggest clarifying the scope of this claim.

Regarding computational cost, the practical trade-off between reconstruction quality and computational cost remains a concern, particularly compared with other zero-shot methods. Overall, I appreciate the authors' efforts, but the remaining concerns regarding the generality of the theoretical interpretation and practical efficiency lead me to retain my original score.

Reviewer_g6ZqOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work proposes to use a 2nd-order regularizer along with TV in the DIP framework for denoising. The proposed method was compared with the original DIP, TV as well as other prior works, showing comparable results to DIP-TV.

Critical strengths

I do not see any critical strengths in this work.

Critical weaknesses
  1. The proposed method added a new regularizer, dubbed SOAR, but its effect was not really significant as shown in Table 1. Note that even though this work did not cite, DIP+TV was explored in (Jiaming Liu et al.,Image Restoration using Total Variation Regularized Deep Image Prior, ICASSP 2019), which was years ago. There are some recent works, mostly in the journals such as
  • Limei Huo et al., Image restoration based on transformed total variation and deep image prior, Applied Mathematical Modelling, 2024
  • Zhi-Feng Pang et al., DIP-based anisotropic total variation regularization for image denoising, Advances in Continuous and Discrete Models, 2025 DIP for denoising was also explored in (Yeonsik Jo et al., Rethinking Deep Image Prior for Denoising, ICCV 2021).
  1. DIP is not practical for denoising due to high computation complexity per image. Thus, it will be fair to compare in computation, too.
  2. More recent benchmarks for denoising, stronger denoiser baseline comparisons will make this work stronger.
Questions for authors

The most critical issue is that DIP+TV was explored very long time ago and this work did not advance it much in terms of both theory and performance. Please see the above issues.

2 comments

We thank the reviewer for the feedback and for highlighting relevant prior work.

  • We acknowledge that earlier DIP+TV studies, particularly Liu et al. (2019), should have been cited. We apologise for the omission and will include these references in future revision. DIP+TV serves as our baseline, with an adaptive TV weight.
  • We recognise the computational cost of the DIP-based framework. We will consider alternative optimisation approaches to better evaluate the proposed regularisation independently of DIP.
  • We agree that broader comparisons would be valuable. However, we would like to clarify that our evaluation already showed superior performance over several recent denoising methods: MASH (2024), INS (2025), NLR (2025), and ZS-HPD (2026), which reported to have better performance than state-of-the-art zero-shot denoising methods.

We appreciate the reviewer’s suggestions and will take them into account in future revisions.

Thank you for the rebuttal and I appreciate that. I also recommend the following recent works on DIP - they may be helpful to further improve your work. Good luck! A Ghosh et al., Optimal Eye Surgeon: Finding image priors through sparse generators at initialization, ICML 2024 I R Alkhouri et al., Image Reconstruction Via Autoencoding Sequential Deep Image Prior, NeurIPS 2024

Comments on the paper

2 comments

We thank all reviewers for their evaluations and constructive feedback. We appreciate both their positive comments and critical suggestions, which help to improve our work.

We acknowledge

  • the missing references to earlier DIP+TV methods,
  • the computational cost of the DIP-based framework and the need for further validation of selective activation,
  • the need to better connect our theoretical analysis with the reconstruction objective.

We will take the comments into account in future revisions: strengthening the theoretical analysis, redesigning the experiments for more comprehensive evaluations.

Area_Chair_iSKaOpenReview ↗

AC Reminder: Please Follow Up on the Authors’ Responses

Dear Reviewers,

Thank you for your time and thoughtful reviews. The authors have posted responses to your questions and concerns.

If you have not yet done so, please read their responses and any relevant revisions, and leave a follow-up comment visible to the authors indicating which concerns have been addressed and which remain unresolved. Please update your review and rating if warranted. If your assessment remains unchanged, a brief explanation would still be helpful.

Thank you for your continued engagement in the discussion.

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.