Trash or Treasure? Rethinking Polarization for Reflection Removal

Polarization provides physical cues for reflection removal beyond RGB appearance, but how much modern polarization-based methods actually exploit them remains unclear. We revisit this question through a controlled counterfactual analysis that disentangles genuine polarization information from the learned restoration prior. Our analysis reveals that strong polarization-based performance does not necessarily imply strong polarization utilization, and that the benefit of polarization is highly conditional: it is most pronounced when informative physical cues coincide with ambiguity in RGB restoration, but can be redundant or detrimental otherwise. Based on this insight, we derive physically distinct observations from the same polarized capture to expose complementary restoration hypotheses using a shared frozen RGB restorer, and integrate them with polarization evidence through a lightweight fusion module rather than training another dedicated polarization restorer. Across three frozen generative restorers, this consistently improves strong RGB baselines without modifying the underlying restorer. These results recast polarization as conditional physical evidence rather than an always-on modality for reflection removal.