acceptodds
Withdrawn from ICLR 2027

Distilling Both Clarity and Rain: Equivariant Self-Collaborative Distillation for Unpaired Image Deraining

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Unpaired image deraining methods usually jointly learn image restoration and realistic rain synthesis from independently collected rainy and clean images, leaving degradation modeling weakly constrained. We propose Equivariant Self-Collaborative Distillation (ESCD), which converts the restoration ability of a frozen derainer into complementary clarity, rain, and objective knowledge. Given a rainy image and an equivariant view, the teacher produces aligned clean predictions and dual rain residuals. The clean predictions distill clarity into the student, while the residuals provide transferable rain structure. A Structure and Appearance Factorized Rain Generator separates clean content, rain appearance, and rain structure, then uses the fused residual representation to gate affine modulation. Rain-Responsive Collaborative Regularization further converts the residual pair into bounded spatial weights for synthesis fidelity and channel consistency, emphasizing rainy regions without discarding background supervision. Iterative Self-Collaborative Distillation promotes the refined student to the next frozen teacher only at cycle boundaries and carries forward the complete collaborative system, which progressively evolves knowledge under stable supervision within each cycle. Extensive experiments on paired synthetic, paired real, and unpaired real datasets demonstrate state of the art performance among existing unpaired deraining approaches, with consistent improvements in full reference fidelity and perceptual image quality as well as strong generalization to real rain.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Unpaired Image Deraining, Self-Collaborative Distillation, Teacher-Guided Rain Synthesis, Rain-Responsive Regularization, Stagewise Teacher Refinement

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6EhkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ESCD queries a frozen derainer under two aligned views and reuses its clean predictions and rain residuals for student distillation, conditioned rain synthesis, and spatial loss weighting. Subsequent cycles promote the student to the next frozen teacher while retaining the collaborative system. Only the student is required at inference.

Critical strengths
  1. The framework connects restoration targets, synthesis conditioning, and spatial regularization through a shared teacher signal, without requiring an external semantic or quality-assessment model.

  2. The method performs strongly across synthetic, real daytime, and nighttime benchmarks. Importantly, the complete Stage-II graph already improves Rain100L from 33.28 to 34.84 dB before iterative refinement.

  3. Single-view, fusion, generator, weighting, and cycle-count comparisons support several design choices. The teacher, generator, and discriminator introduce no additional inference modules.

Critical weaknesses
  1. The extra 0.68 dB on Rain100L follows further training cycles, without equal-budget continued-training or fixed-teacher controls. The acknowledged training-cost limitation therefore remains experimentally unresolved.

  2. Teacher residuals can contain background textures and restoration errors, which subsequently influence all three paths. Quantitative analysis against available synthetic rain layers and tests with weaker teachers are needed to characterize error propagation.

  3. The concatenation comparison supports SAFRG overall, but does not separate the effects of factorization, affine modulation, gating, and capacity. Separate encoders and identity cases alone do not demonstrate clean-content and rain disentanglement.

  4. RGSUD is absent from the main unpaired-real perceptual comparisons, and the checkpoint-selection criterion and validation split are unspecified. Small perceptual-score differences also require uncertainty estimates.

Questions for authors
  1. Do ISCD gains persist against continued training with a fixed teacher and other self-distillation controls under matched total computation?

  2. How accurately do the residual priors identify rain, and how sensitive is the system to teacher errors or background-texture contamination?

  3. Can targeted SAFRG ablations isolate factorization from modulation, gating, and additional parameters?

  4. Please clarify checkpoint selection, any use of paired ground truth during selection, and comparisons with RGSUD under a common real-rain transfer protocol.

Reviewer_PkNMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies unpaired single-image deraining, where restoration and rain synthesis must be learned from independent rainy and clean image sets. It proposes ESCD, which extracts aligned clean predictions and residual rain priors from two equivariant teacher views, then uses them for student distillation, factorized rain generation, and residual-weighted regularization. Experiments compare ESCD with prior unpaired deraining methods on paired and unpaired benchmarks.

Critical strengths

1: The paper develops a coherent use of a frozen derainer beyond ordinary pseudo-labeling. Equations (1)-(3) explicitly construct two aligned teacher predictions and residuals under grid-preserving transformations, while Equations (4)-(9) reuse these signals for rain synthesis and spatial loss weighting. This is a meaningful attempt to connect restoration supervision, degradation modeling, and training-objective allocation within one unpaired graph.

2: The empirical study is reasonably broad within the selected comparison set. Table 1 evaluates six paired synthetic and real benchmarks, Table 2 adds two real unpaired datasets, and Tables 3-4 isolate the contributions of SAFRG, RCR, dual-view fusion, and iterative refinement. In particular, the staged ablation in Table 3 supports that gains are not attributed solely to ISCD, since the complete Stage II graph already improves on the CSUD-based starting point

Critical weaknesses

The state-of-the-art positioning is not adequately supported against the strongest recent unpaired deraining literature. The abstract and conclusion claim state-of-the-art performance among existing unpaired approaches, but Table 1 compares only CycleGAN, DerainCycleGAN, NLCL, DCD-GAN, CSUD, and partially RGSUD. It omits recent directly relevant unpaired methods that jointly model rainy-to-clean translation and rain generation, including Mask-DerainGAN, as well as more recent real-world unpaired approaches based on Schrödinger bridges and iterative prompt learning. This is consequential because ESCD’s claimed contribution is precisely a new supervision mechanism for unpaired real-world deraining, and these omitted methods could plausibly alter the claimed empirical ranking. At minimum, the paper should add direct quantitative comparisons where protocols are compatible, or clearly narrow its claims to the compared baselines and explain any reproducibility or protocol barriers.

The checkpoint-selection and model-selection protocol is insufficiently specified to rule out test-set-informed selection. Section 3.5 and Appendix A/Table 5 repeatedly refer to a “selected” student or collaborative tuple that initializes the next ISCD cycle, but never state the selection criterion, whether a validation set exists, or whether paired test PSNR/SSIM was used in checkpoint selection. This matters because the final claims rely on relatively small differences on some datasets, for example the second ISCD improvement in Table 3 and the mixed result against RGSUD in Table 1. The authors should specify the checkpoint rule for every stage, identify the validation data and metric if any, and confirm that paired test references were never used for tuning, early stopping, cycle selection, or reporting selection.

Potentially Missing Related Work

Wen et al., Neural Schrödinger bridge for unpaired real-world image deraining, 2024. This is directly relevant because it formulates unpaired real-world deraining as distribution transport between rainy and clean domains, making it an important contemporary methodological and empirical comparator. It should be discussed in Section 2.2 and compared with ESCD in the main unpaired benchmark tables where training protocols permit.

Wen et al., Unpaired iterative prompt learning for real-world image deraining, 2026. This work is particularly relevant to the paper’s iterative supervision-refresh narrative, but uses CLIP-based prompt learning rather than self-distillation. It should be discussed alongside RGSUD and ISCD in Sections 2.2 and 3.5, with empirical comparison if compatible.

Wang et al., Mask-DerainGAN: Learning to remove rain streaks by learning to generate rainy images, 2024. This is a directly relevant unpaired GAN framework that jointly learns deraining and rain generation with mask-guided modeling. It should be added to Section 2.2 and included as a baseline in Table 1 or explicitly discussed as unavailable under a matched protocol.

Luo et al., Local and global knowledge distillation with direction-enhanced contrastive learning for single-image deraining, 2023. Although supervised, it is directly relevant to the paper’s positioning of knowledge distillation for deraining. It should be discussed in Section 2.3 to distinguish ESCD’s dual-view output/residual distillation from local and global feature distillation.

Hu et al., Rain-Prior Injected Knowledge Distillation for Single Image Deraining, 2022. This paper directly relates to the claim that ESCD distills explicit rain structure in addition to clean restoration knowledge. It should be cited in Section 2.3, with a concise comparison between injected rain priors and ESCD’s teacher-derived residual priors.

Su et al., Recurrent Network Knowledge Distillation for Image Rain Removal, 2022. This is relevant prior work on teacher-student knowledge distillation for deraining architectures. It should be added to Section 2.3 to better delimit the contribution of iterative self-collaborative distillation.

Questions for authors

In Section 3.5 and Appendix A, what precisely determines the “selected” checkpoint tuple passed from Stage II to Stage III and between ISCD cycles?

Reviewer_yoynOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ESCD, an unpaired image deraining framework built on CSUD. A frozen derainer processes an image and an invertibly transformed view; aligned clean predictions supervise the student, while absolute residuals guide rain synthesis and spatial loss weighting. SAFRG encodes clean content, rainy appearance, and fused residual structure, using a structure gate for affine feature modulation. RCR emphasizes high-residual locations in synthesis fidelity and channel consistency losses. After training the complete system in Stage II, two ISCD cycles promote the student to a frozen teacher and continue training the student, generator, and discriminator. Only the NAFNet student is retained at inference. Evaluation covers six paired benchmarks and transfer from Rain100L to SIRR and Real3000. The results are promising, but several controls needed to substantiate the proposed mechanisms and iterative gains are missing.

Critical strengths

The paper presents a well-motivated framework that jointly guides rain removal and synthesis without an external IQA assessor during training. ESCD reports substantial improvements over CSUD, achieving the best PSNR on five of six paired benchmarks and the best SSIM on all six among the compared unpaired methods. The ablations examine the main components and refinement strategy across synthetic, daytime real, and nighttime data. The appendices provide useful implementation details, and removing all auxiliary networks at test time preserves the student’s inference cost.

Critical weaknesses

W1. The effectiveness of Stage III is not sufficiently established. The current results do not isolate the benefit of updating the teacher from that of additional training. Starting from the same Stage II checkpoint, the authors should compare their teacher-update strategy against retaining the original Stage II teacher and continuing training for the same 150k steps under an otherwise identical training schedule.

W2. The reliability of dual-view residual fusion is insufficiently validated. Although residuals from geometrically transformed views are intended to complement each other, their fusion may also reinforce erroneous responses caused by mistakenly removed object or background textures. Direct analysis and qualitative comparisons of the residual priors are needed to demonstrate that dual-view fusion improves rain localization or reduces background contamination.

W3. Real-rain evaluation needs more convincing comparisons. The perceptual evaluation omits RGSUD, and the qualitative examples are not clearly linked to SIRR and Real3000, leaving it unclear whether the small gains in no-reference metrics translate into visible improvements. Dataset-specific visual comparisons would strengthen the evidence, while downstream detection or segmentation evaluations would further clarify the practical benefits.

Questions for authors

Q1. Effectiveness of teacher updates in Stage III. Starting from the same Stage II checkpoint, can the authors compare the proposed teacher-update strategy against retaining the original Stage II teacher and continuing training for the same 150k steps under an otherwise identical schedule? This would isolate the benefit of teacher updates from additional training.

Q2. Reliability of dual-view residual fusion. Can the authors provide direct analysis and side-by-side visualizations of single-view and fused residuals, using paired examples to assess their correspondence with actual rain degradation? Please include cases involving fine textures to examine whether fusion improves rain localization or reinforces shared teacher errors.

Q3. Visual quality and practical benefits on real rain. Can the authors include RGSUD in the perceptual evaluation and provide clearly identified qualitative comparisons on SIRR and Real3000, with local crops showing rain removal and detail preservation? Do the small gains in no-reference metrics correspond to visible improvements? As an additional evaluation of practical value, would these improvements benefit downstream object detection or segmentation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.