acceptodds
Under review as a conference paper at ICLR 2027

Physically Interpretable Multi-Degradation Image Restoration

Abstract

Although image restoration has advanced significantly, most existing methods target only a single type of degradation. In real-world scenarios, images often contain multiple degradations simultaneously, such as rain, noise, and haze, requiring models capable of handling diverse degradation types. Moreover, methods that improve performance through module stacking often suffer from limited interpretability. In this paper, we propose a novel interpretability-driven approach for multi-degradation image restoration, built upon a deep unfolding network that maps the iterative process of a mathematical optimization algorithm into a learnable network structure. Specifically, we employ an improved second-order semi-smooth Newton algorithm to ensure that each module maintains clear physical interpretability. To further enhance interpretability and adaptability, we design an explainable convolution module inspired by the human brain’s flexible information processing and the intrinsic characteristics of images, allowing the network to flexibly leverage learned knowledge and autonomously adjust parameters for different input. The resulting tightly integrated architecture, named InterIR, demonstrates excellent performance in multi-degradation restoration while remaining highly competitive on single-degradation tasks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image restoration, Interpretable, Deep unfolding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_K2mDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes InterIR, a deep unfolding network based on an improved second-order semi-smooth Newton optimization. It reformulates multi-degradation image restoration as an interpretable iterative optimization process. It designs an explainable convolution module for input-adaptive parameter adjustment. The method addresses the poor interpretability of existing image restoration models. InterIR achieves SOTA performance on composite degradation datasets and maintains strong performance on various single-degradation tasks.

Critical strengths

InterIR reformulates the image restoration task as an improved second-order semi-smooth Newton optimization problem. It introduces an explainable convolution module design. This design makes the modules an interpretable architecture rather than a black-box network that consists of stacked convolutions.

Critical weaknesses
  1. The paper claims strong physical interpretability based on the correspondence between network steps and semi-smooth Newton iterations. This claim remains at the mathematical level and lacks empirical evidence.

    • The learned degradation matrices A and B should be analyzed to verify that they correspond to real physical degradations.
    • The binary masks from EPBlock should be visualized to confirm that they localize degradation regions, beyond the t-SNE clustering evidence.
    • The intermediate iterations should also be analyzed to show that each module follows the intended optimization behavior. This analysis would support the claimed physical interpretability.
  2. The ablation study consists of module addition and removal experiments. It does not ablate the number of IPBlock unfolding layers n=16n = 16, a key hyperparameter. Deep unfolding network performance depends on the number of unfolded iterations. It is unclear whether the gains come from module design or from more iteration blocks. The threshold τ\tau in EPBlock is a learnable parameter. The paper provides no ablation on τ\tau and does not quantify the benefit of the dynamic mask mechanism.

Questions for authors
  1. In the explainable convolution, a binary mask is generated by a learnable threshold τ\tau. However, the hard threshold is non-differentiable. How is τ\tau optimized during training ? Is τ\tau shared globally or learned per layer or per channel ? Are the results sensitive to the initialization and value of τ\tau ?

  2. The paper states that IPBlock follows the ISN algorithm to maintain physical interpretability. However, in Eq. 10, F(⋅)F(\cdot) and H(⋅)H(\cdot) are not explicitly solved as in Eq. 7. They are instead learned by additional networks. How can the authors ensure that these modules still correspond to ISN iterations rather than a generic black box ? Can the authors provide a convergence analysis or a comparison with an explicit ISN solver ?

Reviewer_FaTxOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes InterIR, a physically interpretable framework for multi-degradation image restoration. The method integrates an improved semi-smooth Newton algorithm into a deep unfolding architecture and introduces an explainable convolution module for input-adaptive restoration. Experimental results demonstrate competitive performance across multiple degradation combinations and single-degradation tasks. The proposed method also shows promising generalization ability and computational efficiency.

Critical strengths
  1. The paper presents a meaningful integration of optimization-based deep unfolding and adaptive convolution, providing a well-motivated approach to multi-degradation image restoration.
  2. The experimental evaluation is comprehensive, covering multi-degradation restoration, single-degradation tasks, real-world scenarios, and unseen degradation types.
  3. The method achieves a favorable balance between restoration performance and computational complexity, supporting its practical applicability.
Critical weaknesses
  1. The influence of degradation composition order remains unexplored. Since different application orders may produce distinct degradation characteristics, it would be useful to examine whether InterIR maintains stable restoration performance under different degradation sequences.

  2. Although Table 1 demonstrates strong performance across different degradation combinations, the robustness of InterIR to varying degradation intensities has not been explicitly analyzed.

  3. The mathematical motivation for applying pooling to the binary attention mask rather than directly pooling the original features could be further clarified.

  4. The caption of Figure 1 is relatively brief and could better elaborate the core idea illustrated in the diagram, and the definition of the learnable threshold in Eq. (14) should also be clarified to facilitate understanding of the explainable convolution.

Questions for authors
  1. In Eqs. (14)–(15), the binary attention mask is spatially pooled before softmax normalization. Could the authors clarify whether this operation is simply standard pooling applied to a binary mask or involves a specialized masked-pooling mechanism? More importantly, what is the mathematical difference between pooling the binary mask and directly pooling the original features, and why is the former more suitable for adaptive kernel modulation?

  2. Regarding Table 12, I suggest that the authors clarify the input image resolution and hardware configuration used to measure inference time and FLOPs.

  3. Whether the improvements in model performance remain consistent under mild, moderate, and severe degradation levels, and whether InterIR maintains consistent restoration performance when the application order of haze, rain, and noise is changed.

Reviewer_k63rOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes InterIR, an interpretability-driven network for multi-degradation image restoration (MDIR). The framework maps an improved second-order semi-smooth Newton optimization algorithm into a deep unfolding architecture, explicitly defining the physical meaning of the network's update steps. To enhance adaptability, the authors introduce an "Explainable Convolution" (EPConv) module that dynamically modulates kernel weights using an input-specific attention mask. Extensive experiments demonstrate that InterIR achieves state-of-the-art results on synthetic multi-degradation datasets while maintaining high competitiveness and striking computational efficiency across various single-degradation tasks.

Critical strengths
  1. The integration of the semi-smooth Newton algorithm into a deep unfolding framework provides a mathematically grounded approach to the restoration process. By mapping variables like the degradation matrix and multiplier to specific optimization steps, the architecture achieves a level of transparency that is often missing in purely data-driven black-box models.

  2. The proposed Explainable Convolution (EPConv) offers a computationally lightweight mechanism for dynamic, image-specific adaptation. By modulating the convolution kernel via a threshold-based learnable mask rather than generating completely new weights, the model strikes an excellent balance between parameter efficiency (only 22G FLOPs) and robust restoration capabilities.

Critical weaknesses
  1. The core mathematical framework relies heavily on the ISN algorithm recently introduced in DeepSN-Net. Although the authors replace the explicit solver with learned network modules (Eq. 10) and introduce EPConv, the fundamental optimization derivation is almost entirely borrowed from this prior work. Consequently, the theoretical contribution of the unfolding process feels highly incremental rather than a novel paradigm shift.

  2. The terminology surrounding the "Explainable Convolution" (EPConv) is notably overstated. Mathematically, generating a binary mask via a threshold and applying softmax to modulate kernel weights is essentially a standard dynamic spatial-channel attention mechanism. Framing this adaptive operation as being inspired by the "human brain's flexible information processing" dilutes the scientific rigor of the paper. It is an adaptive feature modulation, not a fundamentally new "interpretable" mechanism.

  3. The multi-degradation (MDIR) evaluation is primarily conducted on a synthetic dataset where haze, rain, and noise are artificially overlaid. While Table 9 presents results on real-world datasets (RealRain, RTTS, SIDD), these sets inherently represent real-world single degradations rather than complex concurrent multi-degradations. The paper fails to demonstrate whether the model can effectively decouple highly intertwined, simultaneous degradations in the wild.

  4. The comparison methods are a bit outdated; for example, StarIR is a two-year-old work (based on its github repo) which was accepted in 2026. Meanwhile, according to my random search of ICLR 2026 accepted papers, there are several Multi-Degradation / All in One Image Restoration methods published in ICLR2026, while all these papers are not listed for comparison or even discussion (for example MIRAGE), which makes the proposed paper less convincing regarding performance.

  5. Finally, the evaluation settings lack the diversity needed to fully validate the effectiveness of the proposed method. Testing the model on more comprehensive multi-degradation benchmarks, such as CDD-11 or 4-Task Adverse Weather Removal, is necessary to provide a rigorous assessment of its capabilities under diverse and challenging scenarios.

Questions for authors
  1. In Eq. 10–12, F(·) and H(·) are replaced by learned modules (EPBlocks, Convs, W_κ), and A and B are updated by networks rather than by D_φ/D_ϕ. Please state precisely which terms of Eq. 7–8 are kept and which are replaced by free-form networks. Do the 16 unfolded stages still behave like an iterative solver? For example, does ‖Î_n − Î_{n−1}‖ or the data residual ‖A_n Î_n B_n − D‖ decrease across stages? If not, the unfolded structure is mainly an architectural template, and the novelty over DeepSN-Net reduces to EPConv.

  2. The title and contributions emphasize interpretability, but the paper gives no direct evidence for it. Do the learned A and B behave like degradation operators, e.g., does A Î B approximate D? Can you show the masks M from Eq. 14 for different degradation types, and do they localize the degradations in any meaningful way? Without such analysis, the interpretability claim seems to rest mainly on module naming.

  3. The mask in Eq. 14 is a hard threshold, which is not differentiable w.r.t. τ. How is τ learned (e.g., a straight-through estimator)? Also, M is H×W and is pooled down to K_h×K_w. What spatial information survives this pooling, and how does it “highlight which regions activate the kernel”?

  4. The appendix says baselines were “fine-tuned or retrained following the parameter configurations specified in their original papers.” For Table 1, which methods were retrained from scratch on the same 13,000-image MDIR training set with a comparable schedule, and which were only fine-tuned? Some gaps are very large and need explanation, e.g., on noise-only, DeepSN-Net 28.01 dB vs. Defusion 32.72 dB; on rain-only, FDTANet 29.13 dB vs. 38–40 dB for others. These gaps suggest the baselines may not have been trained under consistent conditions.

  5. The numbers in Table 3 (29.11 → 30.22) and Table 4 match the H+R+N column of Table 1 rather than the average. Is the ablation done only on H+R+N? Also, please clarify what “replace with RESM” and “replace with DMUM and MUM” mean relative to DeepSN-Net.

  6. Table 12 reports 22 GFLOPs for InterIR vs. 86 GFLOPs for DeepSN-Net, yet InterIR has more parameters and almost identical runtime. At what input resolution were FLOPs measured? How is the per-sample convolution (y_b = W_b ∗ x_b) implemented, e.g., grouped conv or a loop? With 16 IPBlocks, each containing several EPBlocks, 22 GFLOPs seems low.

  7. Please discuss and, where possible, compare with recent all-in-one / multi-degradation restoration methods published in 2025–2026 (e.g., at ICLR 2026). Currently, the MDIR comparison follows only the FDTANet protocol.

  8. Results on established composite-degradation benchmarks such as CDD-11, and on the adverse-weather setting (e.g., the 4-task / All-Weather benchmark), would make the MDIR claims much more convincing. If these can be provided during the rebuttal, I will take them into account.

1 comment

Thank you for adding the CDD-11 results and clarifying how the EPConv threshold is trained. The new comparisons are helpful, though some recent related work is still missing from the discussion. My main concern remains the physical interpretation of the unfolded stages. I understand that the reconstruction residual need not decrease at every step, but seeing how it behaves across the 16 stages would help assess whether the learned updates improve consistency with the observed image. Also, as Eq. (15) is written, the spatial softmax for a 1x1 kernel always returns one, so the 1x1 EPConv layers in Fig. 2 appear to use the same kernel for every input. I am keeping my rating at 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.