acceptodds
Under review as a conference paper at ICLR 2027

Learning from Failure: Leveraging Unreliable Predictions in Semi-Supervised Real-World Adverse Weather Removal

Abstract

Adverse weather image restoration aims to recover images degraded by rain, haze, snow, and other weather-induced artifacts, thereby improving the robustness of outdoor vision systems. Existing unified restoration models exhibit limited generalization to real-world scenes due to their reliance on synthetic supervision and insufficient semantic constraints. In this paper, we propose a novel student–teacher semi-supervised framework that addresses both challenges. Specifically, we introduce an unreliable database that preserves failed teacher predictions as informative negative samples for contrastive learning, while a reliable database stores high-quality teacher predictions as positive samples. By jointly exploiting reliable pseudo-ground truths and unreliable teacher outputs, the proposed framework learns to enhance desirable restoration characteristics while avoiding common failures. We further propose a phase spectrum-based semantic constraint that replaces computationally expensive text-based supervision with an efficient and naturally aligned semantic prior. An adaptive phase consistency loss is also designed to dynamically balance supervision between the degraded input and teacher pseudo-ground truths according to degradation severity. Extensive experiments on real-world benchmarks demonstrate that the proposed method consistently outperforms existing state-of-the-art approaches in restoration quality and perceptual fidelity while exhibiting stronger generalization to real-world adverse weather conditions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
adverse weather restoration, semi-supervised learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WsTcOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a student–teacher framework for semi-supervised adverse weather removal. It uses reliable and unreliable teacher predictions for contrastive learning and introduces an adaptive phase consistency loss to preserve image structure.

Critical strengths
  1. Retaining failed teacher predictions as negative samples is intuitive and produces clear gains in the component ablation.
  2. The experiments cover real-world weather removal, composite degradations, different backbones, and downstream object detection.
Critical weaknesses
  1. The selection of unreliable predictions is based on relative image quality scores, which may classify valid but lower-ranked restorations as failures.
  2. The analysis of the phase constraint mainly relies on dark-channel statistics, which do not directly demonstrate semantic preservation. The downstream detection results are reported for the complete framework, leaving the contribution of phase supervision unclear.
  3. The adaptive phase loss lacks comparisons with input-only, pseudo-GT-only, and fixed-weight phase constraints, leaving its advantage over simpler alternatives unclear.
Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer WsTc

We thank the reviewer for noting that Retaining failed teacher predictions as negative samples is intuitive and for acknowledging the empirical gain of our method.

We appreciate the reviewer’s engagement with our work and offer our responses below.

Q1. The selection of unreliable predictions

The selection of unreliable predictions is based on relative image quality scores, which may classify valid but lower-ranked restorations as failures.

We have addressed this question in the General Rebuttal. We kindly refer the reviewer to the corresponding response for further details

Q2.1. Semantic information with Fourier phase

The analysis of the phase constraint mainly relies on dark-channel statistics, which do not directly demonstrate semantic preservation.

We have addressed this question in the General Rebuttal. We kindly refer the reviewer to the corresponding response for further details.

Q2.2. Ablation study with downstream detection task

The downstream detection results are reported for the complete framework, leaving the contribution of phase supervision unclear.

We have conducted the experiment to isolate the contribution of the proposed phase consistency loss in the downstream task and report it in the table below.

These results demonstrate that the phase consistency loss independently contributes to preserving discriminative visual information relevant to downstream object detection, beyond the improvements introduced by the unreliable database. We have revised the manuscript to explicitly highlight these controlled comparisons and clarify the individual contribution of phase-based supervision (Appendix D.1.1).

BaselineLconL_{con} w/o unreliable databaseLconL_{con} w/ unreliable databaseLPIPSTextPhaseAP (0.5:0.95)AP (0.5)AP (0.75)SmallMediumLarge
✓26.127.427.228.125.526.3
✓✓31.332.932.230.326.229.9
✓✓32.133.932.738.626.933.3
✓✓✓34.938.034.643.627.635.4
✓✓✓35.938.635.143.528.535.0
✓✓✓35.338.736.644.429.935.3

Q4. Ablation study on adaptive phase loss

The adaptive phase loss lacks comparisons with input-only, pseudo-GT-only, and fixed-weight phase constraints, leaving its advantage over simpler alternatives unclear.

To address this concern, we have conducted additional ablation experiments comparing the proposed adaptive phase consistency loss with several alternative settings, including adaptive-weight phase with input-only and pseudo-GT-only, and fixed-weight phase constraints (w=0.3w=0.3 and w=0.7w=0.7).

WeatherMetricGT OnlyInput OnlyFixed w = 0.7Fixed w = 0.3Ours
RainLIQE ↑3.2993.2613.4003.3813.466
CLIP-IQA ↑0.5890.5710.5920.5900.597
SnowLIQE ↑3.8413.6993.8443.7643.852
CLIP-IQA ↑0.5840.5650.6100.5850.611
HazeLIQE ↑3.1773.1313.2213.1873.263
CLIP-IQA ↑0.5000.4870.5070.5010.507
AvgLIQE ↑3.4393.3643.4883.4443.527
CLIP-IQA ↑0.5580.5410.5700.5590.572

As shown in the table, our adaptive weighting strategy consistently achieves the best performance across all three weather conditions. These additional experiments validate the effectiveness of the proposed adaptive weighting mechanism over single-source and fixed-weight alternatives. We have included these results and the corresponding discussion in the revised manuscript.

Dear Reviewer WsTC,

We sincerely thank you for the time and effort in reviewing our manuscript and providing valuable feedback. We greatly appreciate your thoughtful comments and suggestions. Should you have any further questions or concerns, we would be more than happy to provide additional clarification and engage in further discussion.

Best regards,

The authors.

Reviewer_oREXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a semi-supervised framework for real-world adverse weather removal. It uses reliable teacher predictions as positive samples and stores failed predictions in an unreliable database as negative samples. The paper also introduces adaptive Fourier phase consistency as an alternative to vision-language-based semantic supervision.

Critical strengths
  1. Using failed teacher predictions as hard negative samples is an interesting idea.
  2. The evaluation includes component ablations, different backbones, efficiency comparisons, and a downstream detection experiment.
Critical weaknesses
  1. The definition of an unreliable prediction depends entirely on an NR-IQA score, which may not reliably identify residual weather, color distortion, or structural errors. The unreliable database stores the highest-quality failed prediction. Such samples may be very similar to reliable pseudo-labels, making the positive-negative distinction unclear.
  2. Fourier phase mainly describes spatial structure rather than high-level semantics. The claim that it replaces semantic VLM supervision is overstated. The adaptive weight uses a ratio of two uncalibrated NR-IQA scores. The paper does not show that this ratio reflects degradation severity.
Questions for authors
  1. What happens when NR-IQA incorrectly assigns a high score to a prediction containing artifacts?
  2. Why is the highest-quality failed prediction a better negative than the worst failure or multiple diverse failures?
  3. Is the NR-IQA score ratio quantitatively correlated with actual degradation severity?
  4. How does phase consistency preserve semantic content rather than only spatial structure?
5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer oREX (1/2)

We sincerely appreciate your positive remarks: "Using failed teacher predictions as hard negative samples is an interesting idea.”

We are grateful for the reviewer’s constructive comments and respond to them below.

Effectiveness of NR-IQA method (Q1 and Q3)

What if NR-IQA assigns a high score to a prediction containing artifacts? Is the NR-IQA score ratio quantitatively correlated with actual degradation severity?

In the table, we report the performance of different NR-IQA methods under varying rain densities. For each method, as the value ranges of scores are different, we normalize the scores using min-max scaling based on the predefined minimum and maximum values of the corresponding IQA metric. Higher scores indicate better perceptual image quality.

Rain DensityLIQEMUSIQNIMACLIP-IQARALI
0.00.9870.8080.7140.8780.986
0.10.9710.7880.7070.7700.813
0.20.9380.7700.5730.7690.756
0.30.8850.7700.5600.7270.672
0.40.8370.7690.5370.6710.566
0.50.7930.7670.4940.6270.499
0.60.7650.7150.4900.5400.331
0.70.7360.6710.2740.4780.231
0.80.7090.6290.2700.4660.201
0.90.6860.5780.1590.3650.103
1.00.6570.5560.1600.2280.025

We acknowledge that RALI, like other NR-IQA methods, is not perfect and may occasionally assign high scores to images containing artifacts. Nevertheless, our experiments demonstrate that RALI scores are quantitatively correlated with degradation severity. As shown in the table, RALI exhibits a consistent and substantial decrease in image quality scores as rain density increases. Compared with other NR-IQA methods, RALI demonstrates the strongest sensitivity to increasing rain degradation, making it the most suitable metric among those evaluated for identifying unreliable predictions. Furthermore, the experimental results in Table 5 (main paper) support the effectiveness of using RALI to construct the unreliable bank, demonstrating its practical utility despite its limitations.

For better visualization, we depict the table as a line diagram and report it in Figure 10 (Appendix C.8).

Q2. Best failure over worse failure

Why is the highest-quality failed prediction a better negative than the worst failure or multiple diverse failures?

Our motivation for selecting the highest-quality failed prediction is inspired by the established principle of hard-negative mining in contrastive learning. Previous studies, including [3] and [4], have demonstrated that challenging negative samples can provide more informative learning signals than easily distinguishable negatives.

In our framework, we first identify unreliable predictions by requiring their estimated quality to be worse than that of the degraded input. Among these candidates, we retain the highest-quality failed prediction, as it represents a relatively plausible restoration failure rather than an obviously poor output. We hypothesize that learning from such subtle failures provides more useful discriminative supervision than relying on severely degraded predictions.

Furthermore, retaining only one representative failure per sample reduces memory requirements compared with storing multiple diverse negative predictions. We acknowledge that exploiting multiple complementary failures could potentially provide additional supervision, and we consider investigating this direction in future work. Nonetheless, we have reported the restoration results of each strategy of selection in the table below (Table 12, Appendix C.7). For the diverse failure setting, we choose the top 3 best failures to be negative samples.

ModelLIQE ↑CLIP-IQA ↑Training Time (s/epoch)Disk Memory (MB)
w/ Worst failure3.4570.5503749342
w/ Diverse failures (top-3)3.5340.58641351026
w/ Best failure (ours)3.5270.5723757342

References

[3] Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2021. Contrastive Learning with Hard Negative Samples. ICLR 2021.

[4] Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. 2020. Hard negative mixing for contrastive learning. NeurIPS 2020.

AuthorsOpenReview ↗

Rebuttal to Reviewer oREX (2/2)

Q4. Semantic information with Fourier phase

How does phase consistency preserve semantic content rather than only spatial structure?

We have addressed this question in the General Rebuttal. We kindly refer the reviewer to the corresponding response for further details.

Dear Reviewer oREX,

We sincerely thank you for the time and effort in reviewing our manuscript and providing valuable feedback. We greatly appreciate your thoughtful comments and suggestions. Should you have any further questions or concerns, we would be more than happy to provide additional clarification and engage in further discussion.

Best regards,

The authors.

Reviewer_SNkNOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the adverse weather image restoration by introducing the reliable and unreliable databases into semi-supervised learning framework, which provide positive and negative samples for contrastive learning. A Fourier phase loss is proposed to preserve image structures. Experiments show that the proposed method achieves superior performance on several real-world weather restoration benchmarks.

Critical strengths
  1. The idea of introducing negative samples into adverse weather image restoration is interesting and the ablation study verifies its effectiveness.
  2. This paper is well-organized and easy to understand.
Critical weaknesses
  1. Algorithm 1 treats any teacher prediction worse than the historical best as an unreliable sample. However, many of these predictions are still acceptable in quality, so treating them as negative samples may be unreasonable. The authors should perform a deep analysis and explain why current design can achieve good performance.
  2. The computational cost of the proposed method should be reported and compared with existing methods. The two databases store two extra predictions for every unlabeled image, which linearly increases storage and I/O costs on large datasets. The authors should report the actual storage usage, total training time, and scalability to larger datasets.
  3. The claim of replacing VLM supervision with Fourier loss is not accurate. Fourier phase mainly captures spatial structures and edge details rather than high-level semantic information. The authors should evaluate this claim using semantic metrics or downstream semantic tasks or do not introduce Fourier loss from the VLM supervision perspective.
Questions for authors

In Weakness.

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer SNkN

We thank the reviewer for noting that “The idea of introducing negative samples into adverse weather image restoration is interesting” and for appreciating the effectiveness of our evaluations.

We appreciate the reviewer's careful evaluation of our work. Our responses follow.

Q1. The selection of unreliable predictions

Algorithm 1 treats any teacher prediction worse than the historical best as an unreliable sample. However, many of these predictions are still acceptable in quality, so treating them as negative samples may be unreasonable. The authors should perform a deep analysis and explain why current design can achieve good performance.

We have addressed this question in the General Rebuttal. We kindly refer the reviewer to the corresponding response for further details.

Q2. Computational cost

The computational cost of the proposed method should be reported and compared with existing methods. The two databases store two extra predictions for every unlabeled image, which linearly increases storage and I/O costs on large datasets. The authors should report the actual storage usage, total training time, and scalability to larger datasets..

We have extended the computational overhead comparison in Table 9 (Appendix C.2) to include disk storage requirements and training time with Input size: 256 × 256, evaluated on a single NVIDIA H100 GPU. The updated results are presented below. Additionally, we provide a comparison of the computational overhead associated with different unsupervised constraint methods in Appendix C.3.

MethodTrainable Param (M)Inference Time (ms)GPU Mem (MB)Model Size (MB)Training Time (s/epoch)Disk Space (MB)
MODEM19.96100.83551.217740910
HOGformer16.65218.82760.436533350
Ours19.6721.20497.082253757342

Q3. Semantic information with Fourier phase

The claim of replacing VLM supervision with Fourier loss is not accurate. Fourier phase mainly captures spatial structures and edge details rather than high-level semantic information. The authors should evaluate this claim using semantic metrics or downstream semantic tasks or do not introduce Fourier loss from the VLM supervision perspective.

We have addressed this question in the General Rebuttal. We kindly refer the reviewer to the corresponding response for further details.

Dear Reviewer SNkN,

We sincerely thank you for the time and effort in reviewing our manuscript and providing valuable feedback. We greatly appreciate your thoughtful comments and suggestions. Should you have any further questions or concerns, we would be more than happy to provide additional clarification and engage in further discussion.

Best regards,

The authors.

The authors have addressed all my concerns, so I will maintain my original score.

We sincerely thank the reviewer for their response and for taking the time to engage in the discussion. We greatly appreciate your thoughtful feedback and constructive suggestions.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Rebuttal

We thank the reviewers for their constructive feedback and are excited that the reviewers found our paper "The idea of introducing negative samples into adverse weather image restoration is interesting..." (SNkN), "Using failed teacher predictions as hard negative samples is an interesting idea" (oREX), and "The experiments cover real-world weather removal, composite degradations..." (WsTc).

We have uploaded a revised PDF, with major updates highlighted in red. Here, we provide general responses to common questions raised by the reviewers. In addition, we respond to each individual reviewer for their unique questions.

Semantic information with Fourier phase (SNkN, oREX, WsTC)

We agree that Fourier phase primarily encodes spatial structural information, including object contours, boundaries, and spatial relationships, rather than explicitly representing high-level semantic concepts in the same manner as VLMs. However, spatial structure and semantic understanding are not independent. Previous studies have demonstrated that the Fourier phase spectrum is closely associated with semantically relevant visual information, as it preserves spatial structures, object contours, and geometric relationships that are essential for visual recognition. In particular, [1] demonstrated that phase information can support domain-invariant semantic representations, while [2] further exploited phase-preserving Fourier transformations to maintain semantic consistency across different visual domains. These findings suggest that, although Fourier phase primarily characterizes spatial structure rather than explicitly encoding high-level semantic concepts, it retains discriminative visual cues that contribute to semantic understanding. Motivated by this observation, we exploit phase-spectrum consistency as an efficient structural prior to encourage the preservation of semantically relevant information during image restoration, providing an alternative to computationally expensive vision-language model-based supervision.

To further substantiate this motivation, as suggested by reviewer SNkN, we evaluate the models on a downstream task that requires semantic understanding: image classification. We use pretrained VGG16 as the classifier model. In this experiment, we synthesize weather conditions with ground-truth images from ImageNet100. For the Real World dataset, we download 1000 degraded images from the internet (10 for each class) with the same class as ImageNet100. As shown in the Table, our method also achieves the highest Top-1 classification accuracy among the evaluated approaches on both datasets.

MethodImageNet100 Top-1 Acc.ImageNet100 Top-5 Acc.Real Top-1 Acc.Real Top-5 Acc.
AirNet61.179.255.170.3
PromptIR62.379.954.472.1
DPIR69.588.255.480.0
Ours (using VLM)69.384.358.678.8
Ours69.985.560.780.3

We updated the manuscript to further explain our motivation (Line 72-83)

The selection of unreliable predictions (SNkN, WsTc)

We clarify that unreliable predictions are not selected solely based on their quality relative to the best-quality restoration output. Instead, a prediction is classified as unreliable only when its image quality is also lower than that of the original degraded input (Line 230). This additional criterion prevents valid restorations that perform worse than the best-quality output, but still improve upon the degraded input, from being incorrectly classified as failures. Consequently, the unreliable bank contains only predictions that genuinely degrade image quality rather than improve it. We upgraded the main paper to clarify this (Line 202-213).

References

[1] Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. 2021. A Fourier-Based Framework for Domain Generalization. CVPR 2021, 14383–14392.

[2] Minyoung Kim, Da Li, and Timothy M. Hospedales. 2023. Domain generalisation via domain adaptation: An adversarial Fourier amplitude approach. ICLR 2023.

AuthorsOpenReview ↗

Summary of Manuscript Revision

We introduced several updates to improve the manuscript, specifically:

  • Discussion of our motivation for using Fourier Phase as an alternative prior (Sec 1).
  • Verify the selection process of negative samples (Sec 3.2).
  • Ablation study on LscL_{\text{sc}} with different settings (Table 7 - Sec 4.5) (suggested by Reviewer WsTc).
  • Additional overhead comparison (Table 9 - Appendix C.2) (Suggested by Reviewer SNkN).
  • Ablation study on different unreliable prediction selection strategies (Table 12 - Appendix C.7) (suggested by Reviewer oREX).
  • Further ablation study on NR-IQA methods (Appendix C.8) (suggested by Reviewer oREX).
  • Ablation study on the effectiveness of the proposed components for downstream object detection (Table 15 - Appendix D.1.1) (suggested by Reviewer WsTc).
  • Result with image classification task (Appendix D.1.2) (suggested by Reviewer SNkN)
AuthorsOpenReview ↗

Rebuttal Summary for AC

Dear Area Chair,

Following our rebuttal and subsequent discussions, all three reviewers now rate our paper at 3 (Weak Acceptance). Reviewers SNkN and oREX explicitly acknowledged that their concerns had been resolved. Reviewer WsTC, who initially rated the paper at 3, has not provided a follow-up assessment.

To facilitate your assessment, we summarize below how each concern was addressed and whether its resolution was acknowledged. All content in the table is directly grounded in the original comments.

IssueReviewersResponseFinal Status
1. Semantic information with Fourier PhaseSNkN (W3), oREX (W2, Q4), WsTC (W2)Added claim on how Fourier phase correlates with semantic information, substantiating it with semantic downstream task results.Addressed. Explicitly confirmed by two reviewers SNkN and oREX. Reviewer WsTC did not reply yet.
2. Clarification on the selection of unreliable predictionsSNkN (W1), WsTC (W1)Added clarification stating that the selection is based on their quality relative to both best-quality restoration output and original degraded input.Addressed. Explicitly confirmed by reviewer SNkN. Reviewer WsTC did not reply yet.
3. Computational CostSNkN (W2)Added extra overhead comparison on training time and disk space.Resolved. Explicitly confirmed by reviewer SNkN.
4. Effectiveness of NR-IQA methodsoREX (W1, Q1, Q3)Added extra results of NR-IQA methods, proving NR-IQA is quantitatively correlated with actual degradation severity and that RALI is the best option.Resolved. Explicitly confirmed by reviewer oREX.
5. Clarification on selection of best failure as negative sampleoREX (Q2)Added claim and ablation study with worst failure and diverse failures strategy.Resolved. Explicitly confirmed by reviewer oREX.
6. Ablation study with downstream detection taskWsTC (W2)Added ablation study with different components on downstream task.No response from reviewer WsTC. We believe our new results directly answer WsTC's question and address the concern, despite WsTC not replying yet.
7. Further ablation study with adaptive phase lossWsTC (W3)Added adaptive phase loss comparisons with input-only, pseudo-GT-only, and fixed-weight phase constraints.No response from reviewer WsTC. We believe our new results directly answer WsTC's question and address the concern, despite WsTC not replying yet.

In summary, both reviewers who actively engaged in the discussion, reviewers SNkN and oREX, explicitly acknowledged that their main concerns had been resolved and rated our paper at 3. For reviewer WsTC, we provided targeted clarifications and supplementary experiments addressing all three concerns. Although no follow-up response was provided, we believe these additions adequately address the concerns raised in their review, particularly as several overlapping concerns have already been acknowledged as resolved by the other reviewers.

We greatly appreciate your time and consideration in reviewing our work!

Best regards,

The authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.