acceptodds
Under review as a conference paper at ICLR 2027

Learning Polarization Image Restoration with General Restoration Priors

Abstract

Polarization imaging captures distinctive surface and geometric cues that benefit a wide range of vision tasks. However, real-world polarization acquisition is often affected by multiple coupled degradations, making image restoration essential for practical polarization vision. Existing methods are largely tailored to specific degradations and remain constrained by the limited scale and quality of polarization data. To address these limitations, we develop an all-in-one polarization restoration framework for diverse and composite degradations. We first study the impact of different polarization representations on restoration performance and identify the normalized Stokes representation as an effective choice for separating intensity and polarization information. Accordingly, we devise a dual-branch architecture that separates intensity and polarization modeling. To overcome the limitations of polarization-specific training, the intensity branch leverages pretrained general restoration priors and a mixture-of-experts extension for composite degradations, while its restoration knowledge is adaptively distilled into the symmetric polarization branch via a cross-domain feature transform. In addition, we establish a composite-degradation polarization benchmark to support all-in-one restoration research. Extensive experiments on public datasets and our proposed benchmark demonstrate the effectiveness of the proposed method.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Polarization Imaging, Polarization image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_BiFxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an all-in-one framework for restoring polarization images under diverse and composite degradations. Based on a representation study, the authors adopt the normalized Stokes representation and design a dual-branch network. In this network design, the intensity branch exploits pretrained InstructIR priors with an MoE extension, and an adaptive cross-domain module transfers these priors to a lightweight polarization branch. The authors also build PolarCDD, a composite-degradation benchmark with 11 degradation settings and report consistent gains over general AIO and polarization-specific methods.

Critical strengths
  • A valuable composite-degradation benchmark. PolarCDD covers 11 single and composite degradation settings with 50K training images per setting. It also incorporates real captured data: low-light images acquired with a DoT system and a real-noise subset. This fills a clear gap, since existing polarization datasets mostly target a single degradation, and it should facilitate future research on AIO polarization restoration.
  • The proposed method yields consistent improvements when built on NAFNet, Restormer, MoCE-IR, and InstructIR. It also maintains competitive memory usage and latency (Tab. 6).
Critical weaknesses
  • The investigation of polarization representations is presented as a core contribution (Sec. 3.2, contribution (1)). However, the comparison in Tab. 1 is conducted only with a single NAFNet backbone, so it ignores how network architecture and capacity may affect the conclusion. The representation study is insufficient to support one of the paper’s key contributions. The authors should repeat this comparison within their own proposed architecture, and ideally with at least one more backbone. This would more convincingly establish that the normalized Stokes representation is the right choice.
  • The insight and motivation behind the network design are unclear. Why MoE for the intensity branch? The authors extend the intensity branch into an MoE architecture to improve its handling of composite degradations. It is unclear why this is preferable to simply increasing network depth or the parameter count of the branch. The MoE ablation (Tabs. 10–11) compares only with and without MoE, not against a capacity-matched alternative, so the gain cannot be clearly attributed to the MoE design rather than to added capacity. Why no MoE for the polarization branch? The polarization branch does not use MoE, and the paper does not explain why. This is especially puzzling because polarization parameters are claimed to be more vulnerable to degradation than intensity (Sec. 3.1). In that case, degradation-specific routing would seem at least as relevant to the polarization branch.
Questions for authors
  • Tab. 1 compares representations on NAFNet only, and the margins are small. Please repeat the comparison within your proposed architecture, ideally on one more backbone, to show the choice of normalized Stokes holds beyond a single network.
  • Design motivation. Why extend the intensity branch with MoE instead of simply increasing its depth or width? A capacity-matched baseline is needed, given that your model is already the largest (43.4M). Why is MoE not used in the polarization branch?
7 comments
AuthorsOpenReview ↗

Response to Reviewer BiFx (Part 1/2): Choice of polarization representation (W1 and Q1)

Thank you for recognizing the motivation for our work and the value of the benchmark. Our responses are provided below:

We address why we use normalized Stokes, whether its benefits extend from NAFNet to SwinIR, and how it performs across intensity and DoLP levels.

Why we use normalized Stokes. In normalized Stokes, intensity and polarization are theoretically fully decoupled. The network directly predicts the independent polarization components qq and uu, from which we compute DoLP=q2+u2\mathrm{DoLP}=\sqrt{q^2+u^2} and AoP=12atan2⁡(u,q)\mathrm{AoP}=\frac{1}{2}\operatorname{atan2}(u,q) without using S0S_0. By comparison, S1S_1 and S2S_2 retain intensity information, so predicting [S0,S1,S2][S_0,S_1,S_2] introduces redundancy. Computing DoLP from these predictions requires division by S0S_0, which can amplify noise and degrade performance.

In our dual-branch network, this separation allows the polarization branch to focus on polarization-specific restoration patterns, while the intensity branch models intensity using general restoration priors.

Evaluation across backbones. Our model in the main paper is based on InstructIR, which uses a NAFNet backbone. We also compare the representations on SwinIR (Table R1). The results on both the CNN-based NAFNet and the Transformer-based SwinIR show that [S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] is a more effective and stable representation across these backbones.

Performance across intensity and DoLP levels. Tables R2–R4 report polarization metrics stratified by GT intensity, GT DoLP, and input intensity. The normalized Stokes representation [S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] yields more stable performance across these levels.

Table R1. SwinIR representations.

TypePSNRIPSNR_I↑PSNRpPSNR_p↑SSIMISSIM_I↑SSIMpSSIM_p↑MAEMAE↓MAErMAE_r↓
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]26.7329.010.800.7217.9214.64
[S0,S1,S2][S_0,S_1,S_2]28.2229.220.830.7217.0814.30
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]28.4529.650.840.7415.8613.07
[S0,p,ϕ][S_0,p,\phi]28.1329.010.830.7021.5216.59
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]27.0928.760.820.7018.1914.97

Table R2. PSNRpPSNR_p/MAEMAE stratified by GT intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]23.27/17.4029.57/15.8732.17/13.0033.83/14.7432.72/20.70
[S0,S1,S2][S_0,S_1,S_2]23.42/17.0029.87/15.2032.41/12.3233.94/13.8033.36/19.53
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]23.44/16.3929.92/14.9532.56/12.2234.03/13.7633.72/19.40
[S0,p,ϕ][S_0,p,\phi]23.18/18.2629.91/16.9032.34/13.8833.64/16.3033.12/23.93
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]23.27/16.6230.07/15.0032.49/11.8333.83/13.3733.26/19.96

Table R3. MAEMAE stratified by GT DoLP, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]16.1610.248.983.878.62
[S0,S1,S2][S_0,S_1,S_2]15.5010.088.853.848.64
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]15.199.768.293.638.03
[S0,p,ϕ][S_0,p,\phi]17.2311.2310.334.8310.09
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]15.1310.259.334.379.68

Table R4. PSNRpPSNR_p/MAEMAE stratified by LQ intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]24.28/16.5528.37/16.3030.30/13.4032.11/14.8931.19/20.61
[S0,S1,S2][S_0,S_1,S_2]24.43/16.2128.62/15.6030.47/12.6632.28/13.8331.26/19.07
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]24.37/15.6628.93/15.3330.60/12.5532.35/13.8231.27/19.14
[S0,p,ϕ][S_0,p,\phi]24.13/18.1328.81/17.0830.43/13.9432.02/15.6930.68/23.53
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]24.22/15.8628.95/15.3830.55/12.1432.18/13.4030.78/19.95
AuthorsOpenReview ↗

Response to Reviewer BiFx (Part 2/2): MoE in the intensity and polarization branches (W2 and Q2)

We explain why the intensity branch uses MoE and why the polarization branch remains lightweight, with comparisons to dense refinement and MoE in both branches.

MoE in the intensity branch. We use a mixture of experts (MoE) to increase the capacity of the intensity branch and improve its ability to handle composite degradations and degradations outside the pretraining distribution, such as mosaic artifacts. The parallel design lets us initialize the experts by copying pretrained weights without changing the original branch. At inference, activating only the top-ranked expert limits the added computation.

We compare MoE with a dense refinement module that stacks NAFNet blocks at the end of the network and has a comparable computational cost. The two variants achieve comparable restoration performance, while MoE has lower inference latency (Table R5).

Table R5. Comparison with Ours-Refine and Ours-MoE on PolarCDD for 100k iterations.

MethodPSNRIPSNR_I↑PSNRpPSNR_p↑SSIMISSIM_I↑SSIMpSSIM_p↑MAEMAE ↓Params(M)↓FLOPs(T)↓Latency(ms)↓
Ours-Refine32.7331.070.920.7915.8937.470.1999.72
Ours-MoE32.7031.100.920.7915.8343.430.1986.05

Keeping the polarization branch lightweight. We also investigated incorporating a MoE design into the polarization branch. As shown in Table R6, adding MoE to the polarization branch yields no improvement in polarimetric metrics. We attribute this to architectural redundancy: the intensity branch already identifies and models the degradation patterns, while the polarization branch acquires this knowledge via cross-domain transfer for polarization restoration. In this sense, the polarization branch already acts as an extra expert.

Therefore, we keep the polarization branch lightweight as possible, without introducing additional MoE or text-based modulation, and halve the number of its blocks.

Table R6. Comparison with Ours-DualMoE and Ours on the AIO3 setting for 100k iterations.

MethodPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAE ↓
Ours-DualMoE36.0532.190.940.7813.44
Ours35.8332.210.930.7813.40

Thank you again for your valuable comments. We have conducted additional experiments and updated the manuscript accordingly. We look forward to your feedback and would be happy to address any further concerns during the discussion period.

Reviewer_XkwcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an all-in-one polarization image restoration framework using normalized Stokes inputs, a pretrained InstructIR-derived intensity branch, a polarization branch, mixture-of-experts routing, and bidirectional feature modulation. It also introduces PolarCDD, a benchmark combining low-light, blur, noise, and mosaic degradations. Although the reported reconstruction scores are competitive, the methodological novelty, physical validity, evaluation protocol, and causal attribution of improvements are insufficiently established.

Critical strengths
  1. The paper evaluates multiple polarization representations and reports intensity, DoLP, and AoP metrics rather than relying on intensity PSNR alone.
  2. The experiments cover several public datasets and composite-degradation settings, and the appendix provides architecture and complexity details.
Critical weaknesses
  1. The methodological novelty is weak and insufficiently differentiated from prior polarization restoration work. The proposed framework mainly combines an InstructIR/NAFNet backbone, pretrained RGB weights, MoE routing, and conventional feature modulation. More importantly, prior work already explores joint recovery of polarization information and physically related representations, e.g., CVPR 2025, “PIDSR: Complementary Polarized Image Demosaicing and Super-Resolution.” A still closer multi-degradation architectural precedent is “Architectural Unification for Polarimetric Imaging Across Multiple Degradations” (arXiv 2026), which the submission itself cites. The paper does not isolate a sufficiently distinct conceptual or technical advance beyond these directions.
  2. The physical justification for normalized Stokes restoration is inadequate. Dividing S1 and S2 by S0 can amplify noise when S0 is small, precisely where the paper claims robustness. The study offers empirical averages but no analysis of conditioning, sensor noise propagation, numerical stabilization, or enforcement of physically admissible Stokes estimates (Sec. 3.2, Table 1). The claim of intrinsically superior robustness is therefore not established.
  3. PolarCDD does not convincingly validate real-world composite degradation. Its construction predominantly crops existing DoT data and applies synthetic blur and mosaic operators, with prescribed degradation ordering and combinations. Scene-disjoint cropping is useful but does not substitute for independent cameras, acquisition conditions, or paired real composite-degradation ground truth.
  4. The comparisons do not adequately control for pretraining, model capacity, or adaptation strategy. The proposed system has 43.427M parameters, the largest count in Table 6, and explicitly exploits pretrained RGB restoration weights; baseline methods use heterogeneous channel adaptations and architectures. Equal training iterations or approximate runtime budgets do not resolve these confounders. The reported advantage cannot be cleanly attributed to the proposed cross-domain transfer (Tables 2, 3, 6, and 7).
  5. The evidence does not support the strong mechanistic claims about selective prior transfer and MoE benefits. The gate ablation measures output changes, not whether useful RGB priors are identified rather than generic additional feature mixing. Moreover, Table 11 shows that adding MoE reduces single-degradation DoLP SSIM from 0.839 to 0.826 and dual-degradation intensity PSNR from 31.795 to 31.759, contradicting an unqualified improvement narrative. No downstream polarization task or calibrated physical-consistency evaluation establishes that visually cleaner outputs preserve the relevant polarimetric information.
Questions for authors
  1. What is the precise methodological advance over existing joint polarization reconstruction and multi-degradation architectures? Can the authors establish this with a parameter- and pretraining-matched simple dual-branch baseline?
  2. How are near-zero S0 values stabilized, and what guarantees that recovered normalized Stokes values satisfy physical constraints? Can the authors report noise-propagation analysis and performance stratified by intensity and DoLP?
  3. Can the authors demonstrate generalization on independently acquired, genuinely coupled real-world degradations with reliable ground truth, rather than primarily synthesized combinations of existing data?
  4. To my knowledge, generalization to real-world or cross-domain settings is a major challenge for restoration-based tasks. Have the authors considered cross-domain evaluation, e.g., training on one degraded polarization dataset and testing on another? How does the proposed method perform in this setting? If there is a severe domain gap, how do the authors address it? It would be even better if the authors could also provide visual results of the cross-domain tests.
7 comments
AuthorsOpenReview ↗

Response to Reviewer Xkwc (Part 1/4): Methodological contribution and model capacity (W1, Q1, and W4)

Thank you for recognizing the breadth of our experiments and for providing constructive feedback. Our responses are provided below:

We explain how our use of pretrained RGB restoration priors differs from prior polarization restoration work and present the ablations and simple dual-branch comparison.

W1 and Q1:

Core contribution. As the title suggests, our central contribution is to explore how general restoration priors can be leveraged for polarization image restoration, a direction that has remained largely unexplored. We believe that the scarcity of training data is a key bottleneck. To address it, our intensity branch retains pretrained restoration knowledge, while a symmetric polarization branch learns polarization-specific restoration and benefits from the transferred priors.

Evidence from ablations. Our symmetric dual-branch design transfers restoration priors through cross-domain interactions. Table 4 shows how individual design choices affect restoration performance, including the substantial drop when we remove the pretrained priors. Table 8 includes a simple dual-branch baseline, “Full Branch w/o IP-ATM,” with matched parameters and pretraining. Our model performs better even with a polarization branch of half the depth, showing the importance of cross-domain transfer.

The design is also architecture-agnostic. Across the backbones tested in Appendix A.5 and Table 7, it consistently outperforms the two alternative adaptation strategies.

Relation to prior work. PIDSR, SSMD, and other methods have explored joint restoration across multiple domains, but their performance remains limited by the amount of training data. PIDSR and SSMD jointly process four-angle intensity and Stokes representations. These representations are redundant because they can be converted into one another. Their designs do not readily accommodate the transfer of general restoration priors to the four-angle intensity domain without modifying the pretrained model.

W4:

Model capacity and the simple dual-branch baseline. Separating intensity and polarization restoration into two branches increases model capacity and improves restoration. To evaluate a simpler design, we remove MoE and replace advanced feature modulation with feature addition for cross-domain transfer (Table R5). With 24.82M parameters and 0.18 TFLOPs, this baseline still outperforms larger models such as MoCE-IR, supporting the effectiveness of the dual-branch design and cross-domain transfer. It also has fewer parameters than InstructIR because it uses fixed text embeddings in place of the text encoder.

Table R5. Restoration quality and complexity on AIO3 settings.

MethodParams(M) ↓FLOPs(T) ↓PSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
MoCE-IR25.350.3634.7931.490.930.7714.92
SSMD4.790.2432.6931.010.880.7614.70
simple dual-branch24.820.1837.1632.200.950.7813.38
AuthorsOpenReview ↗

Response to Reviewer Xkwc (Part 2/4): Normalized Stokes stability, physical validity, and robustness (W2 and Q2)

We explain how we handle near-zero intensity and evaluate physical validity, noise robustness, and performance across intensity and DoLP levels.

Input normalization and output decoding. Dividing S1S_1 and S2S_2 by S0S_0 amplifies input noise, but does not reduce the total information provided to the network. In practice, we compute S~i=Clip⁡(Si/(S0+ε))\tilde S_i=\operatorname{Clip}\left(S_i/(S_0+\varepsilon)\right), with ε=10−6\varepsilon=10^{-6} for numerical stability. The Clip operation limits each normalized component to [−1,1][-1,1], allowing stable processing even where S0S_0 is close to zero.

With normalized Stokes, the network directly predicts the independent polarization components qq and uu. We compute DoLP=q2+u2\mathrm{DoLP}=\sqrt{q^2+u^2} and AoP=12atan2⁡(u,q)\mathrm{AoP}=\frac{1}{2}\operatorname{atan2}(u,q) without using S0S_0. By comparison, S1S_1 and S2S_2 retain intensity information, so predicting raw Stokes introduces redundancy. Computing DoLP from those predictions requires division by S0S_0, which can amplify noise you mentioned and degrade performance.

Physical validity. Without explicit output constraints, 99.9811% of predicted pixels satisfy S02≥S12+S22S_0^2\geq S_1^2+S_2^2. We hypothesize that the polarization losses encourage this behavior.

Noise and stratified evaluation. As input noise increases, raw Stokes performance degrades more than normalized Stokes performance (Fig. 12 and Table R1). Fig. 2 and Tables R2 and R3 group the results by GT intensity and DoLP. Table R4 groups by input intensity and shows that normalized Stokes generally improves on raw Stokes, including lower AoP MAE in the darkest input bin.

Table R1. PSNRpPSNR_p/MAEMAE under different noise levels, with each column corresponding to a different noise level (σ\sigma).

Type05102550
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}] (+noise)26.83/15.7421.56/25.5715.89/31.3310.84/37.768.16/40.89
[S0,S1,S2][S_0,S_1,S_2] (+noise)27.00/15.1123.20/21.7118.60/27.0813.17/34.4711.72/38.53
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] (+noise)27.04/14.8024.54/19.9223.62/24.1722.12/30.3420.65/34.85
[S0,p,ϕ][S_0,p,\phi] (+noise)26.82/16.8016.22/27.2112.24/31.988.34/37.686.01/40.72
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)] (+noise)26.93/14.7819.58/19.6614.73/24.089.52/31.296.51/37.02
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]27.04/14.8027.04/14.8027.04/14.8027.04/14.8027.04/14.80
[S0,S1,S2][S_0,S_1,S_2]27.00/15.1127.00/15.1127.00/15.1127.00/15.1127.00/15.11

Table R2. PSNRpPSNR_p/MAEMAE stratified by GT intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]23.27/17.4029.57/15.8732.17/13.0033.83/14.7432.72/20.70
[S0,S1,S2][S_0,S_1,S_2]23.42/17.0029.87/15.2032.41/12.3233.94/13.8033.36/19.53
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]23.44/16.3929.92/14.9532.56/12.2234.03/13.7633.72/19.40
[S0,p,ϕ][S_0,p,\phi]23.18/18.2629.91/16.9032.34/13.8833.64/16.3033.12/23.93
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]23.27/16.6230.07/15.0032.49/11.8333.83/13.3733.26/19.96

Table R3. MAEMAE stratified by GT DoLP, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]16.1610.248.983.878.62
[S0,S1,S2][S_0,S_1,S_2]15.5010.088.853.848.64
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]15.199.768.293.638.03
[S0,p,ϕ][S_0,p,\phi]17.2311.2310.334.8310.09
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]15.1310.259.334.379.68

Table R4. PSNRpPSNR_p/MAEMAE stratified by LQ intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]24.28/16.5528.37/16.3030.30/13.4032.11/14.8931.19/20.61
[S0,S1,S2][S_0,S_1,S_2]24.43/16.2128.62/15.6030.47/12.6632.28/13.8331.26/19.07
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]24.37/15.6628.93/15.3330.60/12.5532.35/13.8231.27/19.14
[S0,p,ϕ][S_0,p,\phi]24.13/18.1328.81/17.0830.43/13.9432.02/15.6930.68/23.53
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]24.22/15.8628.95/15.3830.55/12.1432.18/13.4030.78/19.95
AuthorsOpenReview ↗

Response to Reviewer Xkwc (Part 3/4): PolarCDD, real captures, and downstream evaluation (W3, Q3, and W5)

We describe PolarCDD and the real-capture results, then address the effects of MoE and IP-ATM and report downstream performance.

W3 and Q3:

Paired data in PolarCDD. Capturing images with real composite degradations is straightforward, but obtaining corresponding high-quality references is particularly difficult for polarization cameras. We therefore synthesize degradations to construct paired training data. PolarCDD follows a pipeline widely used in image restoration that approximates the imaging process: we reduce exposure to simulate low light, apply spatially varying blur kernels to simulate motion during exposure, simulate sensor noise, and apply the imaging operator to generate mosaic observations.

We acknowledge the gap between synthetic and real composite degradations, but this approach remains a practical necessity for training AIO polarization restoration models. PolarCDD combines our own captures with public data collected by different institutions, covering multiple cameras, acquisition conditions, and indoor and outdoor scenes.

Results on real captures. Although we cannot evaluate restoration performance quantitatively on real degraded data, we provide qualitative results. Figure 18 in the appendix presents updated results on a vehicle-mounted video dataset captured with a DoFP color polarization camera. Vehicle motion causes blur, and single-frame acquisition introduces noise, so the mosaic measurements contain blur, noise, and mosaic artifacts. Figure 13 also uses real mosaic measurements to illustrate text-controlled restoration. Together, the examples show that our AIO model generalizes to real composite degradations, suppressing noise while recovering polarization textures.

W5:

MoE performance. We have revised Appendix A.8 to clarify that MoE improves performance overall.

Selective prior transfer. As shown in the enlarged region of the first row in Figure 6, intensity features exhibit strong activations, whereas the GT DoLP is concentrated along character boundaries. The affine parameters α\alpha and β\beta selectively enhance these boundaries rather than directly transferring intensity activations, as also reflected in the modulated polarization features. This demonstrates that IP-ATM selectively leverages intensity features while suppressing inappropriate cross-domain transfer, consistent with the improved polarization metrics achieved by gating in Table 4.

Downstream evaluation. We add two tasks to assess restoration quality: shape from polarization (SfP) and de-reflection from polarization (DfP). For SfP, we use the PolarAPP validation set and its surface normal estimation network. For DfP, we use the PolarFree validation set and its corresponding network.

We apply blur and noise to the polarization images. Table R6 and Figure 11 show that these degradations substantially impair downstream performance. Restoring the degraded images with our method yields results closest to the clean-input reference. Most competing methods perform worse than using the degraded inputs directly, which may reflect limited generalization to these data distributions. Our method uses general restoration priors and shows greater robustness to these distribution shifts.

Table R6. Downstream tasks (Sfp and Dfp). We apply blur and noise to the input.

MethodSfP Mean ↓SfP Med. ↓SfP RMSE ↓SfP 11.25° ↑SfP 22.5° ↑SfP 30° ↑DfP PSNR ↑DfP SSIM ↑
LQ15.7110.2422.3353.1477.7685.0315.980.65
PromptIR17.1511.0124.0650.3174.5482.1615.740.65
InstructIR17.0910.8424.1950.2074.8981.9515.380.64
AdaIR17.8311.9624.7649.8673.1280.3415.660.65
MoCE-IR15.9510.2023.1154.1276.8183.6817.130.68
CDIR16.7810.8323.5050.9475.2982.8815.780.65
SSMD16.719.6523.9252.6976.0382.8916.080.67
Ours12.817.7418.9362.3382.7388.6919.120.76
Clean (reference)10.585.0116.8469.3587.0992.0722.440.85
Reviewer_MKhuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an all-in-one polarization image restoration framework that leverages pretrained general image restoration priors to address diverse and composite degradations. It adopts a normalized Stokes representation and a dual-branch architecture with adaptive cross-domain feature transfer to jointly restore intensity and polarization information.

Critical strengths
  1. The paper provides a systematic investigation of polarization representations and introduces a well-motivated dual-branch framework that effectively transfers general restoration priors to polarization-specific restoration.
  2. The proposed PolarCDD benchmark covers diverse composite degradations, and extensive experiments demonstrate consistent improvements over existing methods across multiple restoration tasks and evaluation metrics.
Critical weaknesses
  1. The proposed method uses known degradation prompts at test time, while it is unclear whether competing methods receive equivalent information, raising concerns about comparison fairness.
  2. The normalized Stokes representation does not consistently outperform alternatives across all metrics, and its advantages are insufficiently validated across different architectures.
  3. The proposed prior-transfer mechanism largely relies on existing feature modulation techniques, and its advantages over simpler feature fusion strategies remain unclear.
Questions for authors

Could the authors clarify whether competing methods were provided with the same degradation information at test time to ensure a fair comparison?

Could the authors further explain the choice of normalized Stokes representation, given its inconsistent advantages across metrics, and whether its benefits generalize beyond the evaluated backbone?

Could the authors clarify how the proposed prior-transfer mechanism differs from existing feature modulation techniques, and what evidence supports its advantages over simpler feature fusion strategies?

10 comments
AuthorsOpenReview ↗

Response to Reviewer MKhu (Part 1/3): Degradation information at test time (W1 and Q1)

Thank you for recognizing the motivation for our work and the value of the benchmark. Our responses are provided below:

We explain how the compared methods use degradation information and report the performance of MoCE-IR-Ours without known degradation labels.

Task-specific models such as PIDSR are trained for individual restoration tasks and therefore do not require degradation prompts. General restoration models such as NAFNet, Restormer, and SSMD were not designed specifically for all-in-one (AIO) restoration and do not receive explicit degradation-task information. For AIO models, we follow their original designs: PromptIR uses learnable prompts, MoCE-IR handles different degradations through a mixture of experts (MoE), and InstructIR uses text instructions.

Our design is architecture-agnostic: its core idea is to transfer general restoration priors to the polarization domain. Appendix A.5 and Table 7 evaluate this design with MoCE-IR. The resulting MoCE-IR-Ours variant outperforms both baseline adaptations, MoCE-IR and MoCE-IR-9ch, on every reported metric.

We further train MoCE-IR-Ours for 200K iterations and compare it with other methods on PolarCDD (Table R1). MoCE-IR-Ours outperforms the baselines even without degradation-task information. The comparison with InstructIR-Ours suggests that providing degradation-specific prompts may be more effective than simply increasing model capacity. We use InstructIR as the backbone in the main paper because it achieves better performance, and we acknowledge that it relies on known degradation information.

Table R1. The performance on the PolarCDD.

MethodPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
PromptIR29.5228.730.870.7019.41
InstructIR29.1728.930.870.7219.84
AdaIR29.8129.230.870.7218.00
MoCE-IR30.3729.760.890.7419.24
CDIR29.4429.070.870.7218.60
SSMD30.2429.770.890.7517.83
MoCE-IR-Ours32.8930.590.920.7816.73
InstructIR-Ours33.4631.470.930.7915.14
AuthorsOpenReview ↗

Response to Reviewer MKhu (Part 2/3): Choice of normalized Stokes representation (W2 and Q2)

We explain why we prioritize normalized Stokes for polarization restoration and report results across intensity and DoLP levels and on SwinIR.

Why we use normalized Stokes. In normalized Stokes, intensity and polarization are theoretically fully decoupled. The network directly predicts the independent polarization components qq and uu, from which we compute DoLP=q2+u2\mathrm{DoLP}=\sqrt{q^2+u^2} and AoP=12atan2⁡(u,q)\mathrm{AoP}=\frac{1}{2}\operatorname{atan2}(u,q) without using S0S_0. By comparison, S1S_1 and S2S_2 retain intensity information, so predicting [S0,S1,S2][S_0,S_1,S_2] introduces redundancy. Computing DoLP from these predictions requires division by S0S_0, which can amplify noise and degrade performance.

In our dual-branch network, this separation allows the polarization branch to focus on polarization-specific restoration patterns, while the intensity branch models intensity using general restoration priors.

Performance across intensity and DoLP levels. We prioritize polarization metrics when choosing a representation. The normalized Stokes representation [S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] improves DoLP metrics and reduces AoP MAE in reliable regions. Tables R2–R4 show that its performance is more stable across intensity and DoLP levels.

Evaluation across backbones. Table R5 extends the comparison to SwinIR. The results on both the CNN-based NAFNet and the Transformer-based SwinIR show that [S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] is a more effective and stable representation across these backbones.

Table R2. PSNRpPSNR_p/MAEMAE stratified by GT intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]23.27/17.4029.57/15.8732.17/13.0033.83/14.7432.72/20.70
[S0,S1,S2][S_0,S_1,S_2]23.42/17.0029.87/15.2032.41/12.3233.94/13.8033.36/19.53
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]23.44/16.3929.92/14.9532.56/12.2234.03/13.7633.72/19.40
[S0,p,ϕ][S_0,p,\phi]23.18/18.2629.91/16.9032.34/13.8833.64/16.3033.12/23.93
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]23.27/16.6230.07/15.0032.49/11.8333.83/13.3733.26/19.96

Table R3. MAEMAE stratified by GT DoLP, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]16.1610.248.983.878.62
[S0,S1,S2][S_0,S_1,S_2]15.5010.088.853.848.64
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]15.199.768.293.638.03
[S0,p,ϕ][S_0,p,\phi]17.2311.2310.334.8310.09
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]15.1310.259.334.379.68

Table R4. PSNRpPSNR_p/MAEMAE stratified by LQ intensity, with each column corresponding to a different bin center.

Type0.10.30.50.70.9
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]24.28/16.5528.37/16.3030.30/13.4032.11/14.8931.19/20.61
[S0,S1,S2][S_0,S_1,S_2]24.43/16.2128.62/15.6030.47/12.6632.28/13.8331.26/19.07
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]24.37/15.6628.93/15.3330.60/12.5532.35/13.8231.27/19.14
[S0,p,ϕ][S_0,p,\phi]24.13/18.1328.81/17.0830.43/13.9432.02/15.6930.68/23.53
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]24.22/15.8628.95/15.3830.55/12.1432.18/13.4030.78/19.95

Table R5. SwinIR representations.

TypePSNRIPSNR_I↑PSNRpPSNR_p↑SSIMISSIM_I↑SSIMpSSIM_p↑MAEMAE↓MAErMAE_r↓
[x0∘,x45∘,x90∘,x135∘][x_{0^\circ},x_{45^\circ},x_{90^\circ},x_{135^\circ}]26.7329.010.800.7217.9214.64
[S0,S1,S2][S_0,S_1,S_2]28.2229.220.830.7217.0814.30
[S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2]28.4529.650.840.7415.8613.07
[S0,p,ϕ][S_0,p,\phi]28.1329.010.830.7021.5216.59
[S0,p,cos⁡(2ϕ),sin⁡(2ϕ)][S_0,p,\cos(2\phi),\sin(2\phi)]27.0928.760.820.7018.1914.97
AuthorsOpenReview ↗

Response to Reviewer MKhu (Part 3/3): IP-ATM and simpler fusion alternatives (W3 and Q3)

We explain the two directions of feature modulation in IP-ATM and compare affine modulation with simpler fusion methods.

We study which existing feature modulation mechanisms are best suited to polarization restoration. IP-ATM uses a different interaction mechanism in each direction. In the intensity branch, zero-initialized convolutions preserve the pretrained restoration capability. In the polarization branch, learned affine coefficients modulate polarization features, following a strategy widely used in multimodal architectures.

We compare affine modulation with feature concatenation, feature addition (Zero-Conv), squeeze-and-excitation (SE), and the convolutional block attention module (CBAM), keeping feature widths and module depths consistent (Table R6). In the SE and CBAM variants, intensity features generate channel and spatial attention maps for the polarization features. Affine modulation achieves the best results in this comparison. It modulates features more explicitly than concatenation and directly scales and shifts them, offering more flexible and stronger modulation than addition or attention-based reweighting. We therefore retain the affine design.

Table R6. Ablation study of polarization branch modulation on AIO3 settings.

ConfigurationPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
Zero-Conv36.3731.850.940.7713.62
SE36.9931.930.950.7813.69
CBAM37.1632.200.950.7813.58
Concat37.4732.540.950.7913.16
Affine Transformation (IP-ATM)37.5232.820.950.7913.01
Reviewer_PzAXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies restoration of already-acquired polarization images under individual and composite degradations. Both network input and output use a nine-channel representation consisting of RGB-valued $(S_0,S_1/S_0,S_2/S_0)$; the restored representation is analytically converted into four analyzer-angle color images and polarization parameters.

Critical strengths
  1. The quantivatie comparision show substantial improvements across metrics on a practical restoration problem. On PolarCDD dataset, intensity image PSNR, DoLP image PSNR are improved, while AoP MAE decreases (Table 3). Thus, the system's reported value extends beyond visually improving the intensity image. Coverage of individual and composite degradations is relevant to practical polarization imaging.

  2. Transferring general restoration priors into the polarization domain is considered to be a convincing research direction, and the ablation identifies where the gain actually comes from. Removing the pretrained initialization costs 3.72 dB, which is by far the largest single effect in Table 4 and matches the paper's premise about data scarcity.

Critical weaknesses

The input and reference preprocessing are insufficiently specified. Table 5 requires nine-channel normalized Stokes inputs, but Appendix A.2 does not explain how these are constructed from raw mosaic measurements or how inputs and references are radiometrically calibrated. Any interpolation or preliminary demosaicking could affect restoration errors (e.g. IFOV error) and baseline fairness, making these details essential for reproducibility.

Questions for authors
  1. Please specify the complete conversion from raw/synthetic mosaic observations to �0��0�, including any interpolation, spatial rearrangement or preliminary demosaicking, and identify which steps are shared by all comparison methods. Please also specify the linear/nonlinear image domain, exposure and response calibration of references and degraded observations in this pipeline, and the treatment of near-zero �1��1�.

  2. Do the baselines receive the same degradation-task information, pretrained weights and polarization-domain losses? Under these matched conditions and a comparable active computation budget, what is the gain of IP-ATM over a simple cross-domain interaction?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PzAX (Part 1/2): Input preprocessing and reference acquisition (Weakness and Q1)

Thank you for recognizing the motivation for our work. Our responses are provided below:

We describe how DoFP mosaics are converted to normalized Stokes, how the reference and degraded images are acquired, and how we handle exposure, calibration, and near-zero intensity.

From mosaic measurements to normalized Stokes. We first use the open-source library polanalyser to demosaic the division-of-focal-plane (DoFP) measurements into four angle images, Y=[y0∘,y45∘,y90∘,y135∘]Y=[y_{0^\circ},y_{45^\circ},y_{90^\circ},y_{135^\circ}]. These images serve as input to all methods. We compute the Stokes vector from YY using Eq. (2), then form [S0,S~1,S~2][S_0,\tilde S_1,\tilde S_2] with S~i=Si/S0\tilde S_i=S_i/S_0.

In practice, we compute S~i=Clip⁡(Si/(S0+ε))\tilde S_i=\operatorname{Clip}\left(S_i/(S_0+\varepsilon)\right), where ε=10−6\varepsilon=10^{-6} stabilizes the division and the clip operation limits each normalized component to [−1,1][-1,1]. This allows stable processing even where S0S_0 is close to zero.

Acquisition and exposure. PolarCDD combines our own captures with public datasets, all acquired with division-of-time (DoT) systems. Four angle observations are obtained by rotating a polarizer mounted in front of a color camera. The camera's image signal processor (ISP) processes the observations with disabled automatic white balance, tone mapping, and gamma correction. But the observations may still be nonlinear because of color demosaicking and denoising.

For normal exposures, we fix the exposure time, set the gain to zero, and capture 1,000 frames per scene to reduce noise. For low-light observations, we shorten the exposure and capture one frame. We form training pairs by varying the exposure time, without further exposure correction.

Response calibration. We did not perform additional response calibration because we lacked access to the necessary facilities. We measure restoration performance against the corresponding dataset references and will acknowledge this limitation in Appendix A.12.

AuthorsOpenReview ↗

Response to Reviewer PzAX (Part 2/2): Baseline settings and comparison with simpler interactions (Q2)

We clarify the degradation information, pretrained weights, and losses used by the baselines and compare IP-ATM with simpler cross-domain interactions.

Baseline settings. We fine-tune each model on our benchmarks using its own pretrained weights and the same polarization-domain losses.

Task-specific models such as PIDSR are trained for individual restoration tasks and therefore do not require degradation prompts. General restoration models such as NAFNet, Restormer, and SSMD were not designed specifically for all-in-one (AIO) restoration and do not receive explicit degradation-task information. For AIO models, we follow their original designs: PromptIR uses learnable prompts, MoCE-IR handles different degradations through a mixture of experts (MoE), and InstructIR uses text instructions.

Our design is architecture-agnostic: its core idea is to transfer general restoration priors to the polarization domain. Appendix A.5 and Table 7 evaluate this design with MoCE-IR. The resulting MoCE-IR-Ours variant outperforms both baseline adaptations, MoCE-IR and MoCE-IR-9ch, on every reported metric.

We further train MoCE-IR-Ours for 200K iterations and compare it with other methods on PolarCDD (Table R1). MoCE-IR-Ours outperforms the baselines even without degradation-task information. The comparison with InstructIR-Ours suggests that providing degradation-specific prompts may be more effective than simply increasing model capacity. We use InstructIR as the backbone in the main paper because it achieves better performance, and we acknowledge that it relies on known degradation information.

Table R1. The performance on the PolarCDD.

MethodPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
PromptIR29.5228.730.870.7019.41
InstructIR29.1728.930.870.7219.84
AdaIR29.8129.230.870.7218.00
MoCE-IR30.3729.760.890.7419.24
CDIR29.4429.070.870.7218.60
SSMD30.2429.770.890.7517.83
MoCE-IR-Ours32.8930.590.920.7816.73
InstructIR-Ours33.4631.470.930.7915.14

Comparison with simpler interactions. IP-ATM uses a different interaction mechanism in each direction. In the intensity branch, zero-initialized convolutions preserve the pretrained restoration capability. In the polarization branch, learned affine coefficients modulate polarization features, following a strategy widely used in multimodal architectures.

We compare affine modulation with feature concatenation, feature addition (Zero-Conv), squeeze-and-excitation (SE), and the convolutional block attention module (CBAM), keeping feature widths and module depths consistent (Table R2). In the SE and CBAM variants, intensity features generate channel and spatial attention maps for the polarization features. Affine modulation achieves the best results in this comparison. It modulates features more explicitly than concatenation and directly scales and shifts them, offering more flexible and stronger modulation than addition or attention-based reweighting. We therefore retain the affine design.

Table R2. Ablation study of polarization branch modulation on AIO3 settings.

ConfigurationPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
Zero-Conv36.3731.850.940.7713.62
SE36.9931.930.950.7813.69
CBAM37.1632.200.950.7813.58
Concat37.4732.540.950.7913.16
Affine Transformation (IP-ATM)37.5232.820.950.7913.01

Thank you again for your valuable comments. We have conducted additional experiments and updated the manuscript accordingly. We look forward to your feedback and would be happy to address any further concerns during the discussion period.

AuthorsOpenReview ↗

Follow-up before the discussion period closes

We have updated the results using a fixed text prompt (Ours-fix). Specifically, we use the same prompt, "make the image look better", for all images across all tasks. This allows us to retain text-driven restoration capabilities without relying on known degradation information.

Table R3 reports the comparison after 30K iterations (approximately 6 epochs). Ours-fix still shows a favorable performance trend over other baselines, suggesting that our main advantage comes from effectively leveraging general restoration priors. Meanwhile, Ours-fix performs worse than Ours, indicating that providing known degradation prompts yields better performance within the InstructIR framework.

However, this does not imply that text-driven restoration is the optimal approach. Existing restoration methods have achieved strong performance using learnable prompts, visual prompts, mixture-of-experts, and agent-based frameworks. Investigating which conditioning strategy is optimal is beyond the scope of this work. Our focus remains on leveraging general restoration priors and transferring them across domains.

Table R3. Average performance on AIO3 settings at 30k iterations.

MethodPSNRIPSNR_I ↑PSNRpPSNR_p ↑SSIMISSIM_I ↑SSIMpSSIM_p ↑MAEMAE ↓
PromptIR29.32129.7830.8390.73216.227
InstructIR30.02330.1370.8540.73815.887
AdaIR29.44629.7820.8420.73316.120
MoCE-IR29.64330.3320.8530.74015.982
CDIR29.77729.9290.8470.73516.168
SSMD30.01230.4350.8480.74215.723
Ours-fix31.60430.7500.8820.75114.622
Ours32.63131.1370.8980.75714.224

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.