acceptodds
Under review as a conference paper at ICLR 2027

Trash or Treasure? Rethinking Polarization for Reflection Removal

Abstract

Polarization provides physical cues for reflection removal beyond RGB appearance, but how much modern polarization-based methods actually exploit them remains unclear. We revisit this question through a controlled counterfactual analysis that disentangles genuine polarization information from the learned restoration prior. Our analysis reveals that strong polarization-based performance does not necessarily imply strong polarization utilization, and that the benefit of polarization is highly conditional: it is most pronounced when informative physical cues coincide with ambiguity in RGB restoration, but can be redundant or detrimental otherwise. Based on this insight, we derive physically distinct observations from the same polarized capture to expose complementary restoration hypotheses using a shared frozen RGB restorer, and integrate them with polarization evidence through a lightweight fusion module rather than training another dedicated polarization restorer. Across three frozen generative restorers, this consistently improves strong RGB baselines without modifying the underlying restorer. These results recast polarization as conditional physical evidence rather than an always-on modality for reflection removal.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Polarization Vision, Reflection Removal

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_9uYhOpenReview ↗
Rating
4(+1)
Confidence
3(−1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the conditional benefits of polarization information in reflection removal and proposes an evidence-aware selective fusion framework using frozen RGB restorers. Through counterfactual analysis, the authors demonstrate that strong restoration performance does not necessarily imply effective utilization of polarization measurements. The proposed method selectively combines three restoration hypotheses and achieves consistent improvements across different pretrained RGB restoration models.

Critical strengths
  1. The paper provides an interesting perspective on the actual utilization of polarization information through well-designed counterfactual experiments. The distinction between polarization utilization and utility offers useful insights into the role of physical measurements in modern learning-based restoration.

  2. The proposed selective fusion framework is well motivated and consistently improves three pretrained RGB restoration models without modifying their parameters.

  3. The experiments are relatively comprehensive, including ablation studies, cross-backbone transfer, and perceptual evaluations, which provide convincing support for the effectiveness of the proposed approach.

Critical weaknesses
  1. The discussion in the Related Work section seems somewhat incomplete, as several recent and highly relevant studies are not discussed, including RGB-to-Polarization Estimation: A New Task and Benchmark Study (NeurIPS 2025), PolarAnything: Diffusion-based Polarimetric Image Synthesis (ICCV 2025), PolarScale: A Physics-Grounded Benchmark for Radiometrically Consistent RGB-to-Stokes Estimation (NeurIPS 2026), and Polarization State Tracing for Reflection Removal and Color-Consistent Reconstruction (CVPR 2026). These works are closely related to polarization estimation, physical consistency, and polarization-based reflection removal. Including a discussion of these recent studies would make the Related Work section more comprehensive and help better position the contributions of this paper.

  2. The physical interpretation of the smoothed polarization observation in Eq. (9) could be further clarified. Although the proposed transformation is empirically effective, Gaussian smoothing, polarization-magnitude subtraction, and luminance normalization may affect its relationship to the original Stokes measurements. It would be helpful if the authors could clarify whether this observation is intended to represent physically meaningful polarization information or primarily serves as an empirical image transformation.

  3. The proposed framework requires three forward passes through the frozen RGB restorer, introducing additional computational overhead despite the lightweight fusion module. While the consistent performance improvements are encouraging, a more detailed discussion of the trade-off between restoration quality and computational efficiency would further strengthen the practical significance of the proposed method.

Questions for authors
  1. Could the authors briefly discuss how the proposed work relates to recent advances in polarization estimation and reconstruction, and further clarify its contributions in this context?

  2. Could the authors clarify the physical interpretation of the smoothed polarization observation in Eq. (9), particularly the effects of Gaussian smoothing and luminance normalization?

  3. Given the additional computational overhead, could the authors further discuss whether the performance improvements justify the three forward passes through the frozen RGB restorer?

Overall, I find this paper interesting and well motivated, with encouraging experimental results. My concerns are relatively minor and appear addressable. I lean toward acceptance and would be happy to increase my rating if the authors satisfactorily address these points during the rebuttal.

Reviewer_DiYzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper revisits how polarization is actually used and when it is useful for reflection removal under modern strong RGB priors. The authors (i) propose a counterfactual diagnosis that perturbs only the angle-dependent polarized component to quantify polarization utilization in existing polarization-based methods, showing surprisingly weak average dependence despite good performance; and (ii) introduce an evidence-aware selective fusion that treats polarization as conditional evidence. From a single polarized capture, they derive three physically distinct observations (mixed I, intensity-compensated minimum-polarizer observation and a smoothed-polarization subtraction), pass each through the same frozen RGB restorer to obtain complementary hypotheses, and learn a lightweight per-pixel convex fusion guided by observable cues (including DoP and hypothesis disagreements). Across three strong frozen restorers, this consistently yields 0.54–0.66 dB PSNR gains while clarifying when and where polarization helps.

Critical strengths
  1. The proposed framework isolates and manipulates the angle-dependent polarization component (erase/roll/shuffle dθ), providing a clean way to assess how existing models utilize polarization information at inference without retraining.

  2. The paper presents a well-integrated framework that selectively combines polarization cues with strong frozen RGB restorers, improving reconstruction performance without requiring backbone fine-tuning.

  3. The paper provides useful ablations on hypothesis combinations and selection strategies, along with parameter sensitivity analyses for polarization smoothing strength and scale.

Critical weaknesses
  1. Insufficient isolation of the polarization contribution. The proposed framework relies on multiple input transformations and supervised fusion. A compute- and capacity-matched RGB-only multi-hypothesis fusion baseline is needed to better establish how much of the improvement comes specifically from genuine polarization information.

  2. Incomplete fusion baselines. Although the supplementary material evaluates several routing strategies, it lacks direct comparisons with uniform averaging and globally learned constant-weight fusion over the same three hypotheses. These baselines would help establish the necessity of spatially adaptive fusion.

  3. Evaluation protocol concerns. Appendix H reports results on the official test subset using out-of-fold predictions from cross-validation over the full dataset, rather than a single fusion model trained exclusively on the official training split. This limits direct comparability with methods evaluated under the standard held-out protocol.

  4. Limited generalization. All experiments are conducted on PolaRGB. Cross-backbone transfer is encouraging, but robustness to different polarization sensors, acquisition conditions, and datasets remains unclear.

  5. Computational efficiency. The method requires three backbone inference passes, resulting in approximately threefold inference cost. The additional gain from the third hypothesis is relatively modest and comes with slight degradation in some perceptual metrics, making a clearer accuracy–efficiency analysis desirable.

  6. No user study is conducted to assess perceptual reflection removal quality, transmission preservation, or potential generative artifacts. Such an evaluation would complement the reported quantitative metrics.

  7. The paper lacks a dedicated discussion of limitations and failure cases, particularly regarding inaccurate polarization measurements, challenging reflection conditions, and potential artifacts introduced by the fusion process.

Questions for authors

The paper provides an insightful counterfactual analysis of polarization utilization and demonstrates promising improvements through selective fusion. However, the attribution of these gains to genuine polarization information, the necessity of learned spatial fusion, and the comparability of the evaluation protocol require stronger validation. I am open to increasing my rating if the authors can convincingly address these concerns during rebuttal.

  1. Under the same three-call budget and fusion capacity, how does RGB-only multi-hypothesis fusion compare with the proposed polarization-derived fusion?
  2. Can official-train-only results and side-by-side comparisons with PolarFree be provided using independently selected hyperparameters?
  3. Could the authors compare the learned spatial fusion with uniform averaging and globally learned constant weights over the same three hypotheses, using the same target-disjoint folds?
2 comments

Dear Reviewer DiYz,

As the discussion period is coming to an end, we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

We sincerely appreciate your time and valuable feedback. Thank you very much for your consideration.

Best regards, The Authors

Dear Reviewer DiYz,

As the discussion period is coming to an end, we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

We sincerely appreciate your time and valuable feedback. Thank you very much for your consideration.

Best regards, The Authors

Reviewer_NtomOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how much learned reflection-removal systems use polarization information and when that information helps a strong RGB restorer. It first holds released polarization-based models fixed and removes or disrupts the angle-dependent component of their inputs. It then applies three frozen RGB restorers to the ordinary mixed observation, a polarization-suppressed observation, and a smoothed suppression variant. A small trained U-Net predicts per-pixel convex weights over the resulting three restoration hypotheses. Experiments on PolaRGB show that unconditional polarization suppression can hurt average performance, while selective fusion improves each evaluated RGB backbone. The study includes target-grouped out-of-fold evaluation, model-transfer experiments, hypothesis ablations, and supplementary perceptual metrics.

Critical strengths
  1. The counterfactual analysis is the strongest contribution for me. Erasing the angle-dependent component while preserving the mixed intensity offers a concrete test of a fixed model's dependence on polarization, without changing its learned prior through retraining. PolarFree loses only 0.16 dB on average on its held-out PolaRGB images, yet the loss is larger for strongly polarized captures and can reverse sign on individual images. The paper also reports that Lyu et al.'s model loses 4.95 dB on its own test set despite its weak or adverse average response on PolaRGB. This is an important countercheck: the results support conditional dependence rather than a sweeping claim that polarization-based models do not use their inputs. Distinguishing these cases makes the analysis a useful scientific contribution.

  2. The fusion experiments show a practical response to that conditional behavior. Applying the same frozen restorer to different observations avoids confounding the hypothesis comparison with different backbone weights. Three-hypothesis fusion improves mean PSNR by 0.543, 0.663, and 0.537 dB for DAI, GenSIRR, and WindowSeat, respectively, and improves a majority of images on every backbone (Table 4). All six cross-backbone transfer directions retain a positive mean gain, although the magnitude varies. Convex fusion is also a meaningful design choice: the output must use colors already supplied by the hypotheses, rather than hiding a new unrestricted restoration model in the fusion head. The target-grouped folds, final-epoch checkpoints, and perceptual metrics strengthen this evidence.

Critical weaknesses
  1. The study establishes that polarization-derived hypotheses are useful, but does not yet isolate their value from a comparable RGB-only hypothesis set. The full method makes three backbone calls, and the minimum and smoothed observations also change exposure, contrast, and normalization. Table 3 examines several polarization-conditioned transformations, while Table 5 includes a color-matched minimum observation and weaker selection rules. These are useful controls. However, they do not answer whether three equally costly RGB-only photometric variants, passed through the same frozen restorer and fused with the same U-Net, would recover a comparable fraction of the gain. That comparison would directly test whether the physical measurement adds information beyond generic input perturbation and selection.

  2. A related attribution question is whether the fusion head needs explicit polarization evidence once it already sees the three restored hypotheses. The disagreement maps are derived from those hypotheses, and the paper does not report a comparison that holds them fixed while removing or disrupting the additional measurement channels. I do not regard a simple fusion module as a weakness, and the existing results do demonstrate complementarity. The concern is narrower: convexity prevents free-form output correction, but does not by itself distinguish physically informed selection from ordinary fusion of differently processed inputs.

  3. The evaluation protocol supports target-grouped generalization, but its interpretation should be more precise. Appendix C states that the 640 images used to select the smoothed-observation parameters are included in the 6,500 evaluated images. The exclusion analysis is reassuring: the mean gains remain approximately 0.52–0.64 dB on the other 5,860 images. Nevertheless, the split is defined at the ground-truth-target level, so a strictly separate check should exclude complete target groups used for tuning or tune within each training fold. Removing only the selected images does not establish that their related captures were excluded from evaluation. This matters because PolaRGB contains many captures of a limited number of fixed scenes.

  4. Appendix H also explains that the “official test split” result is a restriction of out-of-fold predictions, rather than the evaluation of one fusion model trained only on the official training split. This is transparent and is not evidence of target leakage. It does mean that Table 7 is not directly comparable as a standard fixed-split training experiment. I would like that distinction stated in the main comparison, together with uncertainty clustered by target for the principal three-hypothesis gains. The protocol concerns qualify the reported effect sizes, but the exclusion results and consistency across backbones make me reluctant to treat them as grounds for dismissing the main finding.

Questions for authors
  1. With the number of backbone calls, fusion architecture, training folds, and preprocessing budget fixed, how does the method compare with a three-hypothesis RGB-only photometric ensemble? A small control using transformations of the mixed observation would help identify the incremental value of the polarization measurement.

  2. Holding the three restoration hypotheses fixed, what changes when the fusion module receives only those hypotheses, or when the additional polarization-measurement channels are removed or disrupted? This would clarify whether the claimed evidence-aware selection requires those cues beyond the information already present in the candidate outputs.

  3. Were the targets represented in the 640-image parameter-selection subset excluded as complete groups from the corresponding evaluations, or was parameter selection nested within each training fold? Please give target-grouped confidence intervals for the main gains. If the official-split superiority is retained as a headline comparison, could it also be reported for a single fusion model trained on the official training split?

2 comments

Dear Reviewer Ntom,

As the discussion period is coming to an end, we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

We sincerely appreciate your time and valuable feedback. Thank you very much for your consideration.

Best regards, The Authors

Dear Reviewer Ntom,

As the discussion period is coming to an end, we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

We sincerely appreciate your time and valuable feedback. Thank you very much for your consideration.

Best regards, The Authors

Reviewer_X8qGOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author's argument is: Does using a polarization model to reflect light effectively indicate that it effectively utilizes polarization information? In this regard, the author has demonstrated through extensive experimental analysis that the value of polarization depends on the ability of the scene and the RGB model itself, and therefore polarization should be selectively utilized. Overall, the writing style of the paper is relatively innovative, with rhetorical devices such as rhetorical questions used in multiple positions such as the title and introduction, which effectively attracts readers' interest.

Critical strengths
  1. Proposed valuable research questions and conducted targeted analysis. The paper clearly distinguishes between the "recovery performance of polarization models" and the "contribution of real polarization measurements". By fixing model parameters and erasing or perturbing angle variation components, the author examined the dependence of three existing methods on polarization information and further analyzed how polarization intensity and RGB restoration quality jointly affect the benefits of polarization. These analyses provide useful evidence for understanding the relationship between physical measurements and learning priors.

  2. The main benefits are consistent across multiple recovery backbones, with sufficient auxiliary experiments. The method achieved an average improvement of approximately 0.54-0.66dB on three different frozen backbones (Table 4). The author also provides candidate combination ablation, simple baseline selection, cross backbone migration, as well as some perceptual and structural indicators.

Critical weaknesses
  1. The novelty at the methodological level is limited. The core process of the proposed method consists of polarization input variation, pre trained recovery model, and conventional U-Net fusion network. The workload of the paper mainly lies in the combination and application of these components, and there is a lack of decoupling and targeted thinking on research problems in network design. Regarding the selective utilization of polarization information, it is recommended to refer to the confidence plot calculation method for polarization in the following two papers for design, which will be more convincing than relying solely on stacked experimental data in this paper.

[1] Shao M, Xia C, Yang Z, et al. Transparent shape from a single view polarization image[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 9243-9252.

[2] Yu K, Wang P, He H, Wu X. Structure-Aware Consistency Priors for Shape from Polarization in Complex Media[C]//International Conference on Machine Learning (ICML). PMLR, 2026.

  1. The generalization of experimental conclusions is still limited. The main analysis and fusion experiments are based on PolaRGB; Cross backbone migration still uses the same dataset, and it cannot be proven that the method can be extended to different collection devices or scene distributions.
Questions for authors
  1. Regarding generalization: Appendix G shows the transfer of backbone networks on the same dataset. Can supplementary validation be provided for different datasets? In order to evaluate the adaptability of the method to changes in the scene or equipment. If there is a temporary lack of applicable data, please further explain the expected scope of application and the main failure conditions.

  2. Regarding network design: The third contribution mentioned by the author in the introduction of the paper is the design of a lightweight fusion module for selectively fusing feature maps. But it seems that only Figure 4 in the main text and appendix is related to network design, and only Fusion U-Net is a learnable parameter for the entire network. According to the author's introduction in the Implementation details at the end of Section 4, the U-Net used by the author is the most primitive U-Net network from 2015. After reading it as a whole, I feel that there are insufficient innovative points. Can the author provide more explanation on the innovative points in network design?

  3. Regarding the table in the main text: The presentation of experimental results in the main text needs improvement. The only quantitative table in the main text of the paper mainly displays the ablation experiments on the DAI backbone, while a large number of tables such as the complete results of the three backbones and comparisons with the baseline are mainly located in the appendix. Although the main text has outlined some of the results, readers still need to frequently refer to the appendix. Can the author adjust the text to make the presentation more focused and improve the readability of the paper?

1 comment

Dear Reviewer X8qG,

As the discussion period is coming to an end, we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

We sincerely appreciate your time and valuable feedback. Thank you very much for your consideration.

Best regards, The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Follow-up on Reviewers Feedback

Dear Reviewers,

We would like to kindly follow up on our previous response to your comments.

We would greatly appreciate it if you could take a moment to review our responses and the updated manuscript, and share any further feedback you may have.

Thank you for your time.

Dear Reviewers X8qG, DiYz and Ntom,

As the discussion phase is coming to a close, I encourage you to continue engaging with the authors until the discussion reaches a clear outcome. If any of your questions or concerns remain unaddressed, now is the time to raise them.

If there are no further questions, please respond to confirm your score after rebuttal.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.