acceptodds
Under review as a conference paper at ICLR 2027

POLARIS: Polarimetric Spin-2 Equivariant Adapters for Monocular Metric Depth

Abstract

Monocular depth estimation from a single RGB image is scale-ambiguous: metric scale can be recovered only through learned priors. Existing methods therefore rely on pre-trained backbones mitigating scale ambiguity partially. Polarization provides such measurements, since the angle and degree of linear polarization depend on surface orientation and thus supply geometric cues absent from intensity alone. However, the angle of polarization is measured relative to the sensor's reference axis: a camera roll shifts it in exactly the same way as a change of that axis. An encoder that ignores this transformation therefore becomes dependent on the manufacturer's arbitrary choice of reference axis leading to spin-2 angle ambiguity. To resolve the angle ambiguity, we present POLARIS, which builds a rotation-equivariant polarimetric encoder and injects its orientation-independent features into a frozen geometry backbone through a prior-preserving interface. POLARIS lowers metric AbsRel by 16% over the strongest baseline on HAMMER and ranks first or second on all metrics on GM and RPS benchmarks. Our prediction result is rotation-invariant under an arbitrary analyser-axis rotation that changes the output of a conventional encoder. As a side benefit, polarization enables POLARIS to perceive a screen showing a scene as a flat plane, while RGB-only models, including POLARIS with polarization disabled, hallucinate the depth of the displayed content. On our own real-world captures, it also recovers fine structures missed by RGB-only baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
polarization, monocular metric depth estimation, rotation equivariance, frozen foundation model, shape from polarization, surface normal estimation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_AZTCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper injects four-angle polarization into a frozen MoGe-2 backbone for monocular metric depth. AoLP is treated as a spin-2 field and encoded by a circular-harmonic encoder whose rotation-invariant outputs are added to the ViT patch tokens. With polarization zeroed, the model reduces exactly to an RGB-only network. On HAMMER, metric AbsRel drops from 0.0471 to 0.0396 compared with MoGe-2 fine-tuned on the same data.

Critical strengths
  1. Principled encoding with an exact fallback. Treating AoLP as a spin-2 field is well motivated, and the encoder follows the right transformation law. Since E(0) = 0, the same checkpoint also works on RGB-only input, and polarization can be ablated without retraining.

  2. Low overhead. The branch is cheap: 1.07M extra parameters, about 1 ms latency and 0.04 GB memory (Table 9).

Critical weaknesses

Major weaknesses

  1. Scale prediction is not analysed. The gain appears to come largely from correcting the global scale: in Fig. 2 the baseline point cloud sits behind the ground truth, and in Table 7 "table and background" (88% of pixels) improves from 0.0436 to 0.0387. Yet the scale prediction, inherited from MoGe-2, is never described, and Eq. (3) leaves the class token untouched. It is unclear how polarization reaches the scale estimate.

  2. Limited gains outside HAMMER. On HAMMER, the polarization-off checkpoint already reaches 0.0445, so polarization accounts for about 11%, not 16%. Elsewhere the gains are unclear. On GM, POLARIS is worse than MoGe-2 in AbsRel (0.0690 vs. 0.0617). On the authors' captures it is well behind Metric3D v2 (0.143 vs. 0.090). RPS is too small to be conclusive (50 frames, with a scale fitted to MoGe-2).

  3. Missing controls for the equivariance claim. Two controls are missing for the main claim that equivariance matters. (a) A conventional CNN trained with rotation augmentation: augmentation is not mentioned anywhere, so the gap between 0.0442 and 0.0396 cannot be attributed to equivariance. (b) An S0/DoLP-only branch: in Table 4, replacing AoLP with noise does not change the conventional encoder's error (0.04267 vs. 0.04268), which suggests its gain comes from S0/DoLP rather than AoLP.

Minor weaknesses

  1. Fine-structure claim not supported. The abstract claims that POLARIS recovers fine structures on the authors' captures. In Table 8, however, its B-F1 (0.036) is below that of MoGe-2 (0.048), Metric3D v2 (0.056) and Depth Pro (0.168). B-F1 also drops on HAMMER. The claim rests on Fig. 10 alone.

  2. Insufficient discussion of illumination. The Fresnel link between polarization and surface orientation assumes unpolarized incident light, but the paper never states or examines this assumption. Skylight, light reflected off glass or roads, and displays can all be polarized before they reach the surface, which may explain the weaker outdoor results. The screen experiment (Fig. 9) illustrates the issue. In a dark room, the strong and uniform polarization most likely comes from the display's own emission rather than from reflection. Because the screen happens to be planar, the experiment cannot distinguish geometric reasoning from a simpler rule such as "a uniform, highly polarized region is flat".

Questions for authors
  1. Scale and intrinsics. How is metric scale predicted, and how does polarization reach it if the class token is not injected? Did all baselines receive ground-truth intrinsics at inference?

  2. Hyperparameter sensitivity. How sensitive are the results to the zeroing probability (0.3) and to the number and placement of injection blocks?

  3. Through the window. When depth estimation is used for 3D reconstruction, windows are a common annoyance: the scene outside shows up as protrusions extending out from the window, breaking an otherwise clean indoor structure. How does your work handle scenes seen through glass? Does it still predict depth beyond the window, or does it truncate the depth at the window plane?

5 comments

Overall Response: We sincerely thank the reviewer for the careful and in-depth review, and for recognizing our principled encoding, its exact RGB-only fallback and its low cost. We show how polarization reaches the metric scale and evaluate the models on HAMMER both with and without intrinsics (W1/Q1), correct how we attributed the HAMMER gain, explain the results outside HAMMER and evaluate all models on HouseCat6D, which is not part of our training data (W2), and add a rotation-augmented conventional CNN and a conventional CNN fed only S0 and DoLP (W3). We also narrow the fine-structure and screen claims, state the illumination assumption, examine the sensitivity to the injection blocks and the zeroing probability, and discuss scenes seen through glass (m1, m2, Q2 and Q3). All changes will be highlighted in blue in the revision, and the table, figure and appendix numbers below refer to the revised paper.


W1/Q1. Scale prediction and intrinsics

We are grateful to the reviewer for this thorough analysis. As the reviewer observes, a large part of the gain is in the metric scale, the core ambiguity of monocular depth estimation. On HAMMER, 72.6% of the polarization gain of POLARIS, measured against the same weights with the polarimetric input set to zero, lies in the global metric scale. Both the gain and its scale share are larger on strongly polarized surfaces, where an intensity-only model errs most (Figure 4(b)).

How polarization reaches the scale. We apologize that the sentence after Eq. (3), "The class token is left unchanged", caused confusion. As in MoGe-2, the scale head, a fine-tuned MLP, reads the class token of the last block and predicts the scale of the point map. The polarimetric tokens are added to the patch tokens after blocks 6, 12, 18 and 24 of the 24 blocks, and self-attention in the following blocks passes this information to the class token.

Intrinsics. All models that accept intrinsics (MoGe-2, POLARIS, Metric3D v2 and Depth Pro†) received the calibrated focal length, whereas ZoeDepth, VGGT and Marigold† take none. In POLARIS, the focal length is used only after the network to recover the shift of the point map, so neither the scale head nor the polarimetric branch sees it. We also evaluated the models without intrinsics. In this setting, MoGe-2 and POLARIS recover the focal length from the predicted point map, Depth Pro† uses its own estimate, and Metric3D v2 uses the focal length of its canonical camera.

HAMMER, no intrinsicsAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑e-AbsRel ↓
Depth Pro†0.3739255.70.25330.05120.03850.09870.3689
Metric3D v20.8819613.20.00040.04070.02550.03470.8531
MoGe-2 (RGB-only fine-tune)0.114071.30.96730.04230.01580.09900.0929
POLARIS, polarimetric input set to zero0.099664.10.98650.03630.01510.10320.0799
POLARIS (ours)0.082155.30.99690.03380.01480.09050.0652

The advantage of POLARIS and its gain from polarization therefore do not depend on the given focal length. We will make the path to the scale head explicit in Eq. (3), Section 3.3 and Figure 3, state in Section 4.1 which models receive the focal length, and report the evaluation without intrinsics in Appendix M (Table 16).

W2. Polarization share and results beyond HAMMER

We greatly appreciate the reviewer's close examination of our numbers, and we sincerely apologize for our error. The reviewer is correct. Section 4.1 credited polarization with the whole 15.9% gain over MoGe-2 fine-tuned on the same data (RGB only), whereas the polarization gain within the same weights is 11.0%.

HAMMERAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑e-AbsRel ↓AbsRel change of POLARIS vs. this row
MoGe-2 (RGB-only fine-tune)0.047138.10.99810.02600.01580.10650.0427−15.9%
POLARIS, same weights, polarimetric input set to zero0.044537.60.99800.02230.01510.11010.0415−11.0%
POLARIS (ours)0.039636.20.99800.02090.01480.09660.0417–

Our captures and HouseCat6D. On our captures (Table 11), the AbsRel gap to Metric3D v2 (0.0903 against 0.1431 for POLARIS) lies in the per-frame metric scale, and the two models perform on par once each frame is aligned by its median depth ratio (0.0740 and 0.0734). Since MoGe-2 fine-tuned on the same data (RGB only) trails Metric3D v2 there by an even larger margin (AbsRel 0.1495), we attribute this lead most likely to the data on which each backbone was pre-trained rather than to polarization. Attached to Metric3D v2, our module also draws on polarization.

HAMMERAbsRel ↓RMSE (mm) ↓δ1 ↑
Metric3D v2 (RGB-only fine-tune)0.074967.60.9806
Metric3D v2 + POLARIS, polarimetric input set to zero0.077257.50.9740
Metric3D v2 + POLARIS0.071253.50.9877

On HouseCat6D, which is not part of our training data, Metric3D v2 trails POLARIS, as it does on HAMMER and GM.

HouseCat6DAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑
Marigold†0.7234378.40.00200.17500.04800.0144
ZoeDepth0.6387402.00.35910.15930.07060.0014
VGGT0.3950255.40.39610.10980.07080.0034
Depth Pro†0.2188125.90.75000.03530.04050.0090
Metric3D v20.1937159.80.73310.10930.06420.0019
MoGe-2 (RGB-only fine-tune)0.100660.50.93290.02710.02790.0118
POLARIS (ours)0.088353.60.94710.02700.02720.0074

GM and RPS. The GM test split is a single outdoor drive, so a small offset in the global scale shifts all unaligned metrics. The depth scale of RPS was fitted to the released MoGe-2, which favours MoGe-2-based models, as the caption of Table 2 notes. We will define the 15.9% and 11.0% gains in Section 4.1 and the abstract, describe these characteristics with Table 2, and add the HouseCat6D evaluation and Metric3D v2 + POLARIS to the main text (Tables 3 and 4).


W3(a). Conventional CNN with rotation augmentation

We thank the reviewer for this important suggestion, which Reviewer UGpD also raised. The AoLP is not a plain normal map. It shifts with the analyser axis, differs between diffuse and specular reflection, and depends on the lighting. Augmentation only makes the absolute angle uninformative, so a CNN must still learn the relative-angle cues from data, whereas POLARIS discards the absolute angle by construction. We have now retrained the conventional CNN of Tables 6 and 9, which matches our encoder in size, with random rotations of the analyser axis.

EncoderRotation augmentationAbsRel ↓RMSE (mm) ↓δ1 ↑AbsRel, polarimetric input set to zeroPolarization gainGain kept under AoLP noise ↓
Conventional CNNno0.044237.70.99810.051414.0%99%
Conventional CNNyes0.046137.60.99810.04895.6%108%
POLARIS (ours)not needed0.039636.20.99800.044511.0%2%

Augmentation thus does not account for the gap between the conventional CNN and POLARIS. The augmented CNN gains less from polarization and is unaffected by AoLP noise, so augmentation makes it rely even less on the measured AoLP. POLARIS, in contrast, is exactly invariant without augmentation (Table 6) and is the only encoder here whose gain depends on the AoLP content. We will add this CNN to Tables 6 and 9 and to a new Appendix K, and we will state clearly which augmentation each encoder was trained with.

W3(b). Conventional CNN fed only S0 and DoLP

We sincerely thank the reviewer for this perceptive observation. As the reviewer notes, the conventional CNN barely reacts to AoLP noise. To examine whether its gain comes from S0 and DoLP, we trained a variant of this CNN that receives only S0 and DoLP, with the same training recipe as in Table 9.

HAMMERAbsRel ↓RMSE (mm) ↓δ1 ↑AbsRel, polarimetric input set to zeroPolarization gainGain kept under AoLP noise ↓
MoGe-2 (RGB-only fine-tune)0.047138.10.9981–––
Conventional CNN, S0 and DoLP only (new)0.047738.20.99820.0475−0.3%–
Conventional CNN, S0, DoLP and AoLP0.044237.70.99810.051414.0%99%
POLARIS (ours)0.039636.20.99800.044511.0%2%

The variant gains nothing from polarization, as its branch changes the predictions but these changes cancel out across scenes. The conventional CNN therefore gains from polarization only when the AoLP channels are part of training, although it does not draw on their content at inference. Since the CNN of Table 9 that receives only the AoLP axis field also improves on the RGB-only MoGe-2, S0 and DoLP alone do not fully account for the gain of the conventional CNN. POLARIS, in contrast, cannot draw on S0 and DoLP alone, because they only scale the magnitude of its orientation features (Section 3.4). We will add this variant to Table 9 and revise Section 5.1 accordingly.


m1. Fine-structure claim

We are thankful to the reviewer for pointing this out, and we agree that B-F1 does not support the claim and that Figure 11 shows a single frame. On our captures, the strength of POLARIS lies instead in RMSE, s-AbsRel and st-AbsRel (Table 11). We will remove the sentence from the abstract and present Figure 11 only as a qualitative example.


m2. Illumination and the screen experiment

We appreciate these two points and agree with both. The Fresnel relation assumes unpolarized incident light, whereas skylight, light reflected by glass or roads, and the light of displays can already be polarized when they reach a surface. Such light can weaken or bias the cue and may contribute to the weaker results outdoors. In the screen experiment, the display itself polarizes the light (median DoLP 0.91 with a single AoLP direction) and the screen is planar, so the experiment cannot separate geometric reasoning from a simpler rule. It does show, however, that the polarimetric input separates the screen from the scene it displays, which MoGe-2 cannot do once the margin around the screen is small. We will state the illumination assumption in Section 3.1 and in the limitations, and limit Section 4.3, Figure 10 and the abstract to this claim.


Q2. Injection blocks and zeroing probability

We thank the reviewer for these questions about our design choices. We first removed single injections of POLARIS, or kept only one, at inference on HAMMER, and measured the share of the polarization gain that remains.

Injection after block6121824
Gain kept with this injection removed65%39%55%100%
Gain kept with only this injection20%25%18%−6%

The gain comes mostly from the injections after blocks 6, 12 and 18 acting together, whereas the injection after block 24, which cannot reach the scale head (our answer to W1/Q1), does not contribute. Retraining POLARIS with injections after blocks 12 and 24 only leaves a much smaller gain, in line with this ablation.

HAMMERAbsRel ↓AbsRel, polarimetric input set to zeroPolarization gain
Four injection blocks (paper)0.03960.044511.0%
Two injection blocks (after blocks 12 and 24)0.04240.04373.0%
Four injection blocks, zeroing probability 00.04660.04751.9%

During training, all runs in the paper set the polarimetric input to zero with a probability of 0.3, so that the same weights also learn the RGB-only pass. Without zeroing, the model is less accurate and its polarization gain nearly vanishes, so zeroing does more than provide the RGB-only pass and also helps the branch draw on polarization. We will report these results in a new Appendix N (Table 17).


Q3. Scenes seen through glass

We are grateful for this interesting question. Our captures include an indoor scene that faces a large window, and this scene is part of Table 11. Its stereo reference depth (Appendix H) measures the scene behind the glass, and the evaluation uses no window mask, so our setting rewards depth beyond the window rather than depth truncated at the pane. As Figure 11 shows, POLARIS places the street beyond the pane and farther away than MoGe-2 fine-tuned on the same data (RGB only). We will add this discussion to Appendix H.

Reviewer_NaM9OpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces POLARIS, a framework that injects four-angle polarization into a pre-trained geometry backbone using a spin-2 equivariant harmonic encoder with prior-preserving, position-aligned tokens. Showing improvement over the MoGE v2 backbone and DSINE on both the depth and surface normal estimation tasks.

Critical strengths
  1. The insight is novel and physically grounded with an analysis of the basics of polarization. The idea identifies AoLP as a spin-2 field under analyser/camera roll, which provides a novel angle on the polarization vision.
  2. The circular-harmonic convolution design with magnitude readout is novel and consistent with the previous physics-based analysis, providing an in-depth comparison against the conventional CNN.
  3. The experimental results are comprehensive, with both depth and surface normal estimation showing improvement with the proposed method.
Critical weaknesses
  1. I suppose the module could be added to other similar baselines. The author should demonstrate more x+Polaris designs and evaluations to show the improvement from adding Polaris to other baselines.
  2. It would be better to add PPFT results for comparison, although I know that PPFT is for depth completion. The experiment will be more robust with a comparison to PPFT.
  3. Some metrics are worse than the RGB fine-tuned version of MoGE v2. The authors should provide more detailed justification.
Questions for authors

I have no further questions.

4 comments

Overall Response: We thank the reviewer for the positive assessment and the constructive review, and we are grateful that the reviewer recognized the spin-2 insight, the circular-harmonic encoder and its comparison against a conventional CNN, and the comprehensive experiments on depth and surface normals. We attach POLARIS to two further networks, Metric3D v2 and VGGT (W1), compare POLARIS on the HAMMER test frames of Table 1 with the released PPFT model, which additionally takes active-stereo depth as input (W2), and explain the few metrics in which the RGB-only fine-tune of MoGe-2 leads, adding a zero-shot evaluation on HouseCat6D (W3). All changes will be highlighted in blue in the revision, and the table, figure and appendix numbers below refer to the revised paper.


W1. X + POLARIS on other backbones

We sincerely thank the reviewer for this constructive suggestion. As the reviewer points out, our module is not tied to MoGe-2. Because the polarimetric channels are pixel-aligned with the RGB image, the output of the module can be added to the features of other ViT or CNN backbones at each spatial position. In addition to MoGe-2 (Tables 1 and 5) and the convolutional DSINE (Table 5) in the paper, we have now attached the same module to Metric3D v2 and VGGT.

HAMMERAbsRel ↓AbsRel change vs. zeroed inputRMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑e-AbsRel ↓
MoGe-2 (RGB-only fine-tune)0.0471–38.10.99810.02600.01580.10650.0427
MoGe-2 + POLARIS0.0396−11.0%36.20.99800.02090.01480.09660.0417
MoGe-2 + POLARIS, polarimetric input zeroed0.0445–37.60.99800.02230.01510.11010.0415
Metric3D v2 (RGB-only fine-tune)0.0749–67.60.98060.04070.02550.03470.0870
Metric3D v2 + POLARIS0.0712−7.8%53.50.98770.03000.02260.03860.0762
Metric3D v2 + POLARIS, polarimetric input zeroed0.0772–57.50.97400.03750.02440.04160.0817
VGGT (RGB-only fine-tune)0.1089–83.30.94430.03510.02680.05340.1115
VGGT + POLARIS0.0934−11.4%74.30.96310.03200.02540.05310.0963
VGGT + POLARIS, polarimetric input zeroed0.1054–82.10.94460.03430.02770.05150.1072

The module adds about 1.07M parameters and at most 0.21% more FLOPs to each backbone. Setting the polarimetric input of the same weights to zero raises AbsRel and RMSE on every backbone, so each model draws on polarization. We will add these results to the main text (Table 4), with the training recipes and computational cost in Appendix L, and mention them in contribution (iii).


W2. Comparison with PPFT

We thank the reviewer for raising this important comparison. PPFT motivated our work and, in particular, inspired the idea of injecting polarization into a pre-trained depth network. We left it out of Table 1 because PPFT also takes a sensor depth map at test time, whereas POLARIS uses no depth sensor.

HAMMERTest-time inputAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑e-AbsRel ↓
Without a depth sensor
MoGe-2 (RGB-only fine-tune)RGB0.047138.10.99810.02600.01580.10650.0427
POLARIS (ours)RGB + polarization0.039636.20.99800.02090.01480.09660.0417
With a depth sensor
PPFT, active-stereo depth inputRGB + polarization + sensor depth0.017729.40.99540.01800.01700.03520.0290

PPFT predicts at quarter resolution, and we upsampled its predictions bilinearly to full resolution, so its B-F1 is not directly comparable with that of the other models. PPFT benefits from the sensor depth, but its accuracy varies with the type of depth sensor (Appendix B, Table 7). We therefore regard the two approaches as complementary, with POLARIS serving settings in which no depth sensor is available. We will add PPFT to Table 1 as a separate, sensor-assisted block.

W3. Metrics in which the RGB-only fine-tune leads

We are grateful to the reviewer for this fair and careful observation. We believe that polarization provides a strong cue in indoor scenes with few light sources, such as those of HAMMER and HouseCat6D.

HAMMERAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑e-AbsRel ↓
MoGe-2 (RGB-only fine-tune)0.047138.10.99810.02600.01580.10650.0427
POLARIS (ours)0.039636.20.99800.02090.01480.09660.0417
POLARIS vs. MoGe-215.9% better5.0% better0.01% worse19.6% better6.3% better9.3% worse2.3% better

Boundary F1. POLARIS encodes polarization per 14-pixel patch and uses it mainly for the metric scale rather than to sharpen depth edges. Its lower B-F1 is not accompanied by a larger depth error near edges (e-AbsRel), and on HAMMER the gap is only 1.4% at strongly polarized boundaries (DoLP ≥ 0.15) and largest at weakly polarized ones (DoLP < 0.05).

δ1 and other datasets. The δ1 values of both models are saturated on HAMMER. On HouseCat6D, which was never used in training, they are not, and POLARIS also leads in δ1.

HouseCat6DAbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑
Marigold†0.7234378.40.00200.17500.04800.0144
ZoeDepth0.6387402.00.35910.15930.07060.0014
VGGT0.3950255.40.39610.10980.07080.0034
Depth Pro†0.2188125.90.75000.03530.04050.0090
Metric3D v20.1937159.80.73310.10930.06420.0019
MoGe-2 (RGB-only fine-tune)0.100660.50.93290.02710.02790.0118
POLARIS (ours)0.088353.60.94710.02700.02720.0074

On HouseCat6D, whose reference depth covers only objects, B-F1 stays below 0.015 for every model and carries little information. On GM, whose test split is a single outdoor drive, a small offset in the global scale shifts all unaligned metrics of Table 2. On RPS, the two metrics in which MoGe-2 leads, RMSE and s-AbsRel, do not differ significantly over the 50 test frames (paired bootstrap). We will report the B-F1 breakdown by DoLP in Appendix O (Table 18), add the HouseCat6D evaluation to the main text (Table 3), and note in Section 6, as a general limitation, that the DoLP and the AoLP are noisier than intensity, particularly in dark regions.

Thank you for the detailed response. The authors have satisfactorily addressed my concerns, and I have no further questions.

Thank you very much for your thoughtful review and for raising your score. We truly appreciate the time and effort you dedicated to evaluating our work, and your feedback has been very helpful in improving the paper.

Best regards, The Authors

Reviewer_UGpDOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for improving monocular metric depth estimation by incorporating polarisation measurements into a pretrained geometric foundation model. The rationale is that existing monodepth models are less accurate in regions of high linear degree of polarisation. Yet it is exactly in those regions where the estimate of the angle of polarisation is most robust and this provides a cue to surface normal direction and, in turn, depth. Hence the motivation for incorporating polarisation cues into monodepth.

However, polarisation measurements are dependent on the local frame of reference of the polariser orientation and therefore not rotation invariant or equivariant. AOLP also has an awkward discontinuity which makes it hard to represent when providing as input to a network. This work proposes to address this using a spin-2 rotation-equivariant polarisation encoder, whose features are injected into a largely frozen MoGe-2 backbone.

The result is a lightweight adaptation that improves metric depth accuracy by approximately 16% on the primary benchmark while maintaining invariance to the arbitrary orientation of the camera's polarisation analyser. This is a technically well motivated paper combining a principled representation of polarisation with a practical foundation model adaptation. The experimental results are encouraging, although the improvements in depth accuracy are not uniformly strong across datasets.

I am on the borderline with this paper. I have leaned very slightly towards accepting but am interested to hear the other reviewer's views.

Critical strengths

I see two contributions of somewhat different significance.

The first is demonstrating that polarisation can improve the predictions of a strong pretrained geometry model. This is practically valuable, but unsurprising given the extensive shape-from-polarisation literature.

The second is the proposed use of spin-2 equivariant feature extraction to construct analyser-axis-invariant geometric conditioning for a conventional transformer. This is the more interesting contribution, and the paper provides a clear mathematical justification and unusually direct tests of the resulting invariance.

Critical weaknesses

I disagree with part of the motivation:

"Strong polarization indicates specular reflection or transmission, which violates the Lambertian assumptions on which these models implicitly rely"

Modern monocular depth networks are not explicitly solving a Lambertian image formation model. They learn statistical relationships between images and geometry using large datasets. Their predictions exploit numerous cues, including perspective, texture, object recognition, occlusion, shading and semantic context. There is no inherent reason why these models cannot also learn to interpret specular highlights, reflections and transparent surfaces correctly, provided their training data adequately represents these phenomena. Indeed, specular highlights can themselves be useful cues to surface geometry.

I think the authors could reasonably argue that modern monocular depth networks are biased towards the appearance statistics of predominantly diffuse, opaque surfaces. Perhaps because training datasets underrepresent reflective and transparent objects or because depth supervision for such objects may be unreliable, particularly when obtained from active depth sensors or stereo reconstruction. These problems don't require an implicit Lambertian assumption. They arise from the distribution and quality of training data and from fundamental ambiguities in monocular image interpretation.

They show that MoGe-2's depth error increases with DoLP. This is interesting, but it does not establish that the cause is a Lambertian assumption. In particular, high DoLP could correlate with challenging materials or scene structures that are systematically underrepresented in training data. Moreover, high DoLP does not necessarily imply non-Lambertian reflectance. Diffuse reflection from dielectric surfaces can itself be partially polarised, depending on the viewing angle and material properties. The observed relationship between DoLP and depth error therefore supports the claim that polarisation identifies challenging regions, but not necessarily the proposed explanation for why those regions are challenging.

The numerical gains attributable specifically to equivariance are relatively small. On HAMMER, the conventional CNN polarisation adapter already improves AbsRel from 0.0471 to 0.0442. POLARIS further reduces this to 0.0396. Although this is a meaningful improvement, it means part of the headline 16% gain comes simply from adding polarisation. The incremental contribution of the spin-2 architecture needs to be assessed separately from the benefit of the additional modality.

Generalisation is not convincingly demonstrated. HAMMER is the main source of quantitative improvement. GM and RPS produce mixed or marginal gains, and the small real-world dataset does not demonstrate an unequivocal advantage over competing foundation models. The authors acknowledge that their results are predominantly indoor and in-domain.

Questions for authors

Can the authors compare POLARIS against a conventional CNN polarisation encoder trained with random analyser-axis rotations, with the corresponding transformations applied to AoLP? How much of the performance gap remains after this augmentation? If augmentation closes most of the gap, the empirical contribution becomes less substantial, although exact invariance and parameter efficiency remain valuable. Conversely, if POLARIS retains a meaningful advantage, it would considerably strengthen the case for acceptance.

The improvement in metric depth on HAMMER is convincing, but the results on GM, RPS and your own captures are considerably less compelling. Can you explain the inconsistent gains across datasets? In particular, is there evidence that the benefits depend on particular materials, degrees of polarisation, camera characteristics or training-domain similarity?

The AoLP randomisation experiment convincingly establishes that POLARIS uses polarisation orientation information. However, could the improvements arise primarily from material recognition or other appearance-dependent cues rather than geometric constraints? Could the authors provide further evidence distinguishing these explanations, perhaps through a controlled analysis of surface orientation or performance on materials unseen during training?

4 comments

Overall Response: We sincerely thank the reviewer for the thorough and thoughtful review, and we are grateful that the reviewer recognized the clear mathematical justification of the spin-2 encoder and the direct tests of its invariance. We compare POLARIS with a conventional CNN encoder trained with rotation augmentation (W2/Q1), clarify the role of each training dataset and evaluate all models on HouseCat6D, which was never used in training (W3/Q2), revise the motivation sentence as the reviewer suggested (W1), and separate the contribution of orientation cues from that of material cues (Q3). We will also define the scope of this work more precisely as passive metric depth estimation from polarization, built on a largely frozen foundation model and requiring no second depth source. All changes will be highlighted in blue in the revision, and the table, figure and appendix numbers below refer to the revised paper.


W2/Q1. Rotation augmentation and equivariance

We are grateful to the reviewer for this valid and important suggestion, which Reviewer AZTC also raised. It is a central control that we had missed, and we have now carried it out. Even with rotation augmentation, the conventional CNN encoder remains less accurate than POLARIS in AbsRel and RMSE and makes little use of the AoLP content.

Unlike a surface normal, the AoLP changes even when the geometry does not. Turning the analyser axis by α rotates (cos 2θ, sin 2θ) by −2α (Section 3.1), and the link between the AoLP and the geometry is indirect, since the azimuth is ambiguous by π and diffuse and specular reflection give AoLPs that differ by 90°. An augmented CNN therefore has to learn this order-2 rotation law from examples, which is difficult, whereas the spin-2 encoder satisfies it exactly by construction.

We trained the conventional CNN encoder of Tables 6 and 9, which has the same size as ours, with the analyser axis rotated at random, drawing φ ~ U[0°, 180°) for each sample and rotating only (cos 2θ, sin 2θ) by 2φ.

EncoderRotation augmentationAbsRel ↓RMSE (mm) ↓δ1 ↑AbsRel, polarimetric input set to zeroPolarization gainGain kept under AoLP noise ↓
Conventional CNNno0.044237.70.99810.051414.0%99%
Conventional CNNyes0.046137.60.99810.04895.6%108%
POLARIS (ours)not needed0.039636.20.99800.044511.0%2%

The augmented CNN gains less from polarization and is unaffected by AoLP noise, so augmentation makes it rely even less on the AoLP content. As the reviewer notes, part of the headline gain comes from adding polarization. Beyond this modality gain, the spin-2 encoder is the only encoder examined here that is exactly invariant (Table 6) while also making use of the AoLP content. We will add these controls to Tables 6 and 9 and to a new Appendix K, and we will state clearly which augmentation each encoder was trained with (Section 3.5).

W3/Q2. Generalization

We appreciate the reviewer's important comment on generalization, and we apologize for not explaining clearly why we composed the training data as we did.

Training data. Polarimetric depth data remain scarce, with 23,803 training frames in total across the three public datasets that we use, compared with 8.9M frames for MoGe-2. Previous comparisons, such as those of PPFT, relied mainly on HAMMER, whereas we added GM and RPS to increase the diversity of the training scenes. As benchmarks, however, the two datasets have limitations.

  • GM consists of outdoor driving scenes, and its test split is a single drive, so a small offset in the global scale shifts all unaligned metrics.
  • RPS provides disparity without the metric constant (focal length × baseline), which we fitted to the released MoGe-2, so its metric values are only indicative.
  • Both release only processed AoLP and DoLP maps, from which we re-synthesized the analyser images.

Table 2 is therefore best read as a supplementary result.

HouseCat6D. To test generalization beyond the training data, we evaluated the Table 1 models on HouseCat6D (CVPR 2024), an indoor household dataset that was never used in training.

MethodHAMMER AbsRel ↓ (Table 1)HouseCat6D AbsRel ↓RMSE (mm) ↓δ1 ↑s-AbsRel ↓st-AbsRel ↓B-F1 ↑
Marigold†0.56260.7234378.40.00200.17500.04800.0144
ZoeDepth0.13840.6387402.00.35910.15930.07060.0014
VGGT0.10890.3950255.40.39610.10980.07080.0034
Depth Pro†0.11680.2188125.90.75000.03530.04050.0090
Metric3D v20.07490.1937159.80.73310.10930.06420.0019
MoGe-2 (RGB-only fine-tune)0.04710.100660.50.93290.02710.02790.0118
POLARIS (ours)0.03960.088353.60.94710.02700.02720.0074

Since the reference depth of HouseCat6D covers only objects, B-F1 carries little information on this dataset. The other metrics indicate that the advantage of POLARIS is not tied to its training scenes, and our answer to Q3 examines how the gain depends on materials. We will mark Table 2 as a supplementary result, add the HouseCat6D evaluation to the main text (Section 4.1, Table 3 and Figure 7), and update the abstract accordingly.


W1. Motivation sentence on reflective surfaces

We greatly appreciate this thoughtful explanation and fully agree with it. Following the reviewer's suggestion, we will replace the sentence with the following one.

This trend is consistent with a bias of these models toward diffuse, opaque surfaces, since reflective and transparent objects are rare in pre-training data and their sensor depth is often unreliable (Jung et al., 2023).

In Section 3.2, we will also present Figure 4(b) only as a correlation. The method itself is unaffected, as it learns from the raw polarization field rather than inverting a reflection model. We would welcome any further suggestions.


Q3. Orientation information or material recognition?

We thank the reviewer for this insightful question. Polarization carries both kinds of information, as the DoLP mainly reflects the material and the viewing angle, whereas the AoLP mainly reflects the surface azimuth. To separate the two, we randomized only the AoLP per pixel, as in the noise test of Table 6, kept the intensity and the DoLP unchanged, and evaluated POLARIS on each material group of Table 10 and on HouseCat6D.

AbsRel ↓GlassMetalCeramicOther objectsTable and backgroundAll (HAMMER)HouseCat6D (120 frames)
POLARIS, polarimetric input set to zero0.05690.03940.05330.04680.04360.04450.1041
POLARIS, AoLP replaced by noise0.05690.03960.05340.04680.04350.04440.1037
POLARIS0.05130.03420.04090.04170.03870.03960.0987

Once only the AoLP is randomized, the gain almost entirely disappears for every material, including glass and metal, and also on HouseCat6D, which was never used in training. The gain therefore does not come from recognizing materials through the intensity and the DoLP, which is consistent with the design in which these inputs only scale the magnitude of the orientation features (Section 3.4). Unlike the conventional CNN encoders in our answer to W2/Q1, POLARIS draws on the AoLP itself to predict metric depth. We will add the HouseCat6D evaluation to the main text (Table 3) and report this AoLP-noise analysis in Section 5.1 and Appendix E (Table 10).

Hi authors. Thanks for your detailed response to my review and the additional experimental results. My questions have been satisfactorily answered. I was already positive about this paper and incorporating the updates/additional results from the rebuttal will make it stronger for the final version. I am now above the borderline and retain my weak accept rating.

Thank you very much for your positive assessment and continued support of our work. We are glad that our responses and additional results have addressed your questions. We have already incorporated these updates into the revised manuscript, and we will make sure they are fully reflected in the final version as well.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.