acceptodds
Withdrawn from ICLR 2027

Noise-Controlled Image Restoration via Plug-and-Play Channel-Wise Modulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Real-world weather conditions often involve combinations of rain, snow, haze, and poor illumination, yet image restoration networks are typically trained separately for individual degradations. Although these degradations may share common representation at certain feature levels, incorporating degradation-specific processing without redesigning the restoration backbone remains challenging. We propose \method, a plug-and-play conditioning framework for all-in-one image restoration that applies channel-wise affine modulation after complete backbone blocks. Fixed random Fourier features encode degradation categories, and independent lightweight controllers adapt features at different depths while preserving each block's internal computation. Multi-hot conditions represent mixed degradations; an optional jointly trained image-based predictor supplies soft conditions for label-free inference. The same interface supports convolutional and Transformer backbones with sequential or U-shaped structures. With known degradation labels, NC improves average single-weather PSNR over matched plain backbones by 0.47 dB for Restormer and 1.03 dB for PromptIR, and improves all seven CDD-11 weather mixtures for both backbones (+0.84 and +0.83 dB). The same modules accept soft conditions from a jointly trained CLIP predictor without modification: this blind variant improves the Transformer backbones by 0.82–1.40 dB on single weather and 0.17–0.30 dB on CDD-11 mixtures, and the predictor can be replaced without changing the restorer. Encoding ablations show a 0.76 dB gain from degradation-specific conditioning over constant control. These results support outside-block modulation as a plug-and-play conditioning interface for all-in-one restoration.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
all-in-one image restoration, adverse-weather restoration, mixed degradations, conditional feature modulation, plug-and-play adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xeUFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes NCRestoration, a block-external conditioning interface for all-in-one image restoration. A degradation condition is encoded as a one-hot vector for a single degradation or a multi-hot vector for a mixture, mapped through a fixed random Fourier feature, and summed without normalization. Independent block-specific MLPs then predict channel-wise scale and offset, which are applied after each complete NAFNet, Restormer, or PromptIR block, so the internal residual and attention computation of the backbone is untouched. An optional jointly trained CLIP ViT-B/32 predictor with a linear head supplies soft conditions for blind inference. The method is evaluated with WeatherBench, LOL-v2, and CDD-11, across three backbones in plain, oracle, and blind variants, and is compared against matched plain backbones.

Critical strengths
  1. This work is easy to read.
  2. The block-external modulation interface genuinely transfers across architectures. This is the paper's most concrete and reusable result, which shows that a conditioning interface can be retrofit to convolutional and Transformer backbones that were designed independently.
Critical weaknesses
  1. The novelty is limited. The conditioning operator is adopted from prior work, and the residual contribution is a migration rather than a new problem formulation.
  2. The main results select checkpoints on the evaluation images, and no recent all-in-one baseline is compared. Tables 1–2 contain only AirNet and TransWeather, while AdaIR, DFPIR, MPerceiver, DA-CLIP, IDR, OneRestore, DCPT, and MWFormer are cited but never compared.
Questions for authors
  1. Blind conditioning outperforms oracle conditioning on rain, haze, and snow for all three backbones. Oracle receives strictly more information, so if both variants share initialization and optimization, as Section 4.1 states, what mechanism explains the inversion, and does it survive when both are retrained with the same budget, early-stopping rule, and selection protocol?
  2. Table 7 shows that supplying the ground-truth one-hot condition to the same weights drops low-light PSNR from 23.97 to 8.67 dB. Under what precise condition can the predictor be replaced without retraining the restorer, as the abstract claims? If the restorer fits the statistics of the CLIP soft distribution rather than degradation identity, would condition dropout during training restore source-independence?
  3. Can you report the identities of all numbers in Table 1 and the NAFNet CDD-11 results after re-selecting checkpoints on a source-disjoint split, and add CDD-11 comparisons against OneRestore and CART under the same protocol? As written, the reported gains of 0.17 to 1.03 dB over matched backbones cannot be separated from selection bias and from run-to-run variation, since every run uses a single seed of 2026.
Reviewer_cLMoOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes NCRestoration, a conditional modulation framework for unified image restoration. The method represents degradation categories as condition vectors using fixed random Fourier encodings. A separate MLP for each network block generates channel-wise scaling and bias parameters, which apply affine modulation to the block’s output. For mixed degradations, the method combines the encodings of the corresponding categories. Under unknown degradation conditions, a jointly trained CLIP image encoder and classification head predict soft conditioning vectors. Its main contribution is a way to introduce conditioning into different restoration backbones without modifying computations within their blocks, although the backbone and added modules still require joint training. The authors evaluate single and mixed degradation restoration using NAFNet, Restormer, and PromptIR on datasets including WeatherBench, LOL-v2, and CDD-11, reporting average performance improvements over the corresponding backbones. Comparisons controlling for parameter capacity, condition encoding ablations, and condition substitution experiments with fixed model weights further assess the role of conditional modulation.

Critical strengths

The conditioning interface is simple and applicable across backbones. By adding channel-wise affine modulation after each complete network block, the method applies the same conditioning mechanism to NAFNet, Restormer, and PromptIR without redesigning their internal convolution, attention, or residual computations. Although joint training is still required, this unified integration rule reduces the implementation effort needed to extend existing restoration networks to multiple degradation tasks, offering clear practical value.

Critical weaknesses
  1. The substantive advance over existing conditional modulation methods has not been sufficiently validated. The method uses Fourier condition encoding and channel-wise affine modulation, mainly emphasizing modulation after complete network blocks and a separate controller for each block. However, the authors explicitly acknowledge in the limitations section that they do not compare modulation within versus after blocks, or separate versus shared controllers, under matched training conditions. Comparisons with simple encodings, such as directly using class vectors, are also missing. The existing ablations mainly demonstrate that conditioning information is useful and that the network relies on the conditioning branch; they do not sufficiently establish why the specific designs are necessary or superior. Given that the core operators have precedents, this evidence gap makes it difficult to substantiate the paper’s methodological contribution and degree of novelty.

  2. Some main experiments use the evaluation set for model selection, weakening the independence of the results. In Table 1 and the CDD-11 experiments with NAFNet, checkpoints are selected through periodic validation on benchmark evaluation images. This involves the evaluation set in model selection and may introduce optimistic bias; applying the same procedure to all methods does not restore the independence of the test results. The Restormer and PromptIR experiments in Table 2 use a separate held-out split from the training data, so this concern does not apply to all results. Nevertheless, it affects the credibility of important performance comparisons and requires confirmation through reevaluation of models selected using an independent validation set.

Questions for authors
  1. The Fourier condition encoding and channel-wise affine modulation used in this paper build on existing methods. Please clarify the main methodological advance over works such as FiLM and TCN. The paper explains that “plug-and-play” refers to architectural integration and that joint training is still required. Beyond this implementation convenience, what considerations specific to image restoration motivate the chosen modulation scheme?

  2. Each backbone uses separate MLPs configured according to its block count and channel dimensions. Why is modulation needed after each complete block, and why does each block require a separate controller?

  3. Degradation category indices have no inherent continuous ordering, yet the current Fourier encoding depends on their numerical assignment. Why use this encoding rather than directly feeding class scores or one-hot/multi-hot vectors into an MLP? Are the results sensitive to permutations of category indices, and can comparisons with simple encodings clarify the actual contribution of Fourier encoding?

  4. The condition substitution experiments show that the model relies on the conditioning branch but do not explain how it improves restoration. Can the authors provide scaling and bias statistics across different degradations and network depths, or targeted layer-wise interventions, to identify which layers’ modulation matters most and whether different degradations produce distinct adjustments? In particular, after joint training, do the CLIP outputs primarily serve as category indicators, or do they also carry other image information that influences restoration?

Reviewer_9AacOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NCRestoration, a conditioning framework for all-in-one image restoration. This paper aims to resolve both single and mixed weather degradations within a unified model. Specifically, the authors first utilize Random Fourier Features to encode different degradation categories and use multi-hot representations to describe mixed degradations. Lightweight independent controllers are then introduced after each backbone block to generate scale and shift, which are used to modulate features through affine transformations. Additionally, the authors introduce a CLIP-based degradation predictor that estimates soft conditions to enable blind restoration without requiring ground-truth degradation labels during inference. The authors evaluate the proposed method on different backbones, including CNNs and Transformers, under both single and mixed weather degradation settings. The experimental results demonstrate performance improvements.

Critical strengths
  1. The paper focuses on all-in-one image restoration, aiming to handle mixed weather degradations within a unified model. By considering multiple degradations in real-world scenarios, the research addresses a practically meaningful problem.
  2. The proposed lightweight framework enables all-in-one image restoration without modifying the internal structure of the backbone blocks, offering flexibility and applicability across different backbone architectures.
Critical weaknesses
  1. Unclear novelty and insufficient method motivation. The proposed method builds upon existing techniques. Meanwhile, the paper does not clearly interpret the key contributions of NCRestoration. The motivation behind the proposed design choices is also insufficiently justified.
  2. This paper is not well written. In the Introduction, there are obvious logical gaps between different parts, and the motivation behind the proposed method is not clearly explained. The motivation and research gap are not clearly established.
  3. The plug-and-play claim appears somewhat limited. Plug-and-play generally means that a module can be directly integrated into an existing system with minimal modifications. However, the authors state that "Here, plug-and-play refers to architectural integration; the backbone and controllers are trained jointly." Since the method still requires joint training, the paper's plug-and-play claim is less convincing.
  4. Outside-block Modulation is a relatively simple and common engineering design, which should not be considered a core methodological innovation. Furthermore, the paper lacks ablation studies to demonstrate the effectiveness of the specific design choices in Outside-block Modulation.
Questions for authors
  1. In the second paragraph of the Introduction, the authors mention "Existing methods..." and then immediately transition to "We study a specific design...". There is an obvious logical gap between these statements. What are the specific limitations of existing methods that motivate the proposed approach? The authors should clarify why the existing methods are insufficient and how NCRestoration addresses these limitations.
  2. Random Fourier Features and Outside-block Modulation are two key components of the proposed method. However, the motivations behind these design choices remain unclear. Why are these particular mechanisms introduced, and what advantages do they provide over alternative conditioning approaches?
  3. The authors use Random Fourier Features to represent discrete weather degradation categories, but the explanation of Random Fourier Features is not clear. Compared with more straightforward category encoding methods, such as one-hot encoding or learnable embeddings, what are the specific advantages of Random Fourier Features? Although the experimental section provides an ablation study on Learnable Embedding, a theoretical explanation is still lacking.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.