- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper injects four-angle polarization into a frozen MoGe-2 backbone for monocular metric depth. AoLP is treated as a spin-2 field and encoded by a circular-harmonic encoder whose rotation-invariant outputs are added to the ViT patch tokens. With polarization zeroed, the model reduces exactly to an RGB-only network. On HAMMER, metric AbsRel drops from 0.0471 to 0.0396 compared with MoGe-2 fine-tuned on the same data.
Critical strengths
-
Principled encoding with an exact fallback. Treating AoLP as a spin-2 field is well motivated, and the encoder follows the right transformation law. Since E(0) = 0, the same checkpoint also works on RGB-only input, and polarization can be ablated without retraining.
-
Low overhead. The branch is cheap: 1.07M extra parameters, about 1 ms latency and 0.04 GB memory (Table 9).
Critical weaknesses
Major weaknesses
-
Scale prediction is not analysed. The gain appears to come largely from correcting the global scale: in Fig. 2 the baseline point cloud sits behind the ground truth, and in Table 7 "table and background" (88% of pixels) improves from 0.0436 to 0.0387. Yet the scale prediction, inherited from MoGe-2, is never described, and Eq. (3) leaves the class token untouched. It is unclear how polarization reaches the scale estimate.
-
Limited gains outside HAMMER. On HAMMER, the polarization-off checkpoint already reaches 0.0445, so polarization accounts for about 11%, not 16%. Elsewhere the gains are unclear. On GM, POLARIS is worse than MoGe-2 in AbsRel (0.0690 vs. 0.0617). On the authors' captures it is well behind Metric3D v2 (0.143 vs. 0.090). RPS is too small to be conclusive (50 frames, with a scale fitted to MoGe-2).
-
Missing controls for the equivariance claim. Two controls are missing for the main claim that equivariance matters. (a) A conventional CNN trained with rotation augmentation: augmentation is not mentioned anywhere, so the gap between 0.0442 and 0.0396 cannot be attributed to equivariance. (b) An S0/DoLP-only branch: in Table 4, replacing AoLP with noise does not change the conventional encoder's error (0.04267 vs. 0.04268), which suggests its gain comes from S0/DoLP rather than AoLP.
Minor weaknesses
-
Fine-structure claim not supported. The abstract claims that POLARIS recovers fine structures on the authors' captures. In Table 8, however, its B-F1 (0.036) is below that of MoGe-2 (0.048), Metric3D v2 (0.056) and Depth Pro (0.168). B-F1 also drops on HAMMER. The claim rests on Fig. 10 alone.
-
Insufficient discussion of illumination. The Fresnel link between polarization and surface orientation assumes unpolarized incident light, but the paper never states or examines this assumption. Skylight, light reflected off glass or roads, and displays can all be polarized before they reach the surface, which may explain the weaker outdoor results. The screen experiment (Fig. 9) illustrates the issue. In a dark room, the strong and uniform polarization most likely comes from the display's own emission rather than from reflection. Because the screen happens to be planar, the experiment cannot distinguish geometric reasoning from a simpler rule such as "a uniform, highly polarized region is flat".
Questions for authors
-
Scale and intrinsics. How is metric scale predicted, and how does polarization reach it if the class token is not injected? Did all baselines receive ground-truth intrinsics at inference?
-
Hyperparameter sensitivity. How sensitive are the results to the zeroing probability (0.3) and to the number and placement of injection blocks?
-
Through the window. When depth estimation is used for 3D reconstruction, windows are a common annoyance: the scene outside shows up as protrusions extending out from the window, breaking an otherwise clean indoor structure. How does your work handle scenes seen through glass? Does it still predict depth beyond the window, or does it truncate the depth at the window plane?
5 comments
W2. Polarization share and results beyond HAMMER
We greatly appreciate the reviewer's close examination of our numbers, and we sincerely apologize for our error. The reviewer is correct. Section 4.1 credited polarization with the whole 15.9% gain over MoGe-2 fine-tuned on the same data (RGB only), whereas the polarization gain within the same weights is 11.0%.
| HAMMER | AbsRel ↓ | RMSE (mm) ↓ | δ1 ↑ | s-AbsRel ↓ | st-AbsRel ↓ | B-F1 ↑ | e-AbsRel ↓ | AbsRel change of POLARIS vs. this row |
|---|---|---|---|---|---|---|---|---|
| MoGe-2 (RGB-only fine-tune) | 0.0471 | 38.1 | 0.9981 | 0.0260 | 0.0158 | 0.1065 | 0.0427 | −15.9% |
| POLARIS, same weights, polarimetric input set to zero | 0.0445 | 37.6 | 0.9980 | 0.0223 | 0.0151 | 0.1101 | 0.0415 | −11.0% |
| POLARIS (ours) | 0.0396 | 36.2 | 0.9980 | 0.0209 | 0.0148 | 0.0966 | 0.0417 | – |
Our captures and HouseCat6D. On our captures (Table 11), the AbsRel gap to Metric3D v2 (0.0903 against 0.1431 for POLARIS) lies in the per-frame metric scale, and the two models perform on par once each frame is aligned by its median depth ratio (0.0740 and 0.0734). Since MoGe-2 fine-tuned on the same data (RGB only) trails Metric3D v2 there by an even larger margin (AbsRel 0.1495), we attribute this lead most likely to the data on which each backbone was pre-trained rather than to polarization. Attached to Metric3D v2, our module also draws on polarization.
| HAMMER | AbsRel ↓ | RMSE (mm) ↓ | δ1 ↑ |
|---|---|---|---|
| Metric3D v2 (RGB-only fine-tune) | 0.0749 | 67.6 | 0.9806 |
| Metric3D v2 + POLARIS, polarimetric input set to zero | 0.0772 | 57.5 | 0.9740 |
| Metric3D v2 + POLARIS | 0.0712 | 53.5 | 0.9877 |
On HouseCat6D, which is not part of our training data, Metric3D v2 trails POLARIS, as it does on HAMMER and GM.
| HouseCat6D | AbsRel ↓ | RMSE (mm) ↓ | δ1 ↑ | s-AbsRel ↓ | st-AbsRel ↓ | B-F1 ↑ |
|---|---|---|---|---|---|---|
| Marigold† | 0.7234 | 378.4 | 0.0020 | 0.1750 | 0.0480 | 0.0144 |
| ZoeDepth | 0.6387 | 402.0 | 0.3591 | 0.1593 | 0.0706 | 0.0014 |
| VGGT | 0.3950 | 255.4 | 0.3961 | 0.1098 | 0.0708 | 0.0034 |
| Depth Pro† | 0.2188 | 125.9 | 0.7500 | 0.0353 | 0.0405 | 0.0090 |
| Metric3D v2 | 0.1937 | 159.8 | 0.7331 | 0.1093 | 0.0642 | 0.0019 |
| MoGe-2 (RGB-only fine-tune) | 0.1006 | 60.5 | 0.9329 | 0.0271 | 0.0279 | 0.0118 |
| POLARIS (ours) | 0.0883 | 53.6 | 0.9471 | 0.0270 | 0.0272 | 0.0074 |
GM and RPS. The GM test split is a single outdoor drive, so a small offset in the global scale shifts all unaligned metrics. The depth scale of RPS was fitted to the released MoGe-2, which favours MoGe-2-based models, as the caption of Table 2 notes. We will define the 15.9% and 11.0% gains in Section 4.1 and the abstract, describe these characteristics with Table 2, and add the HouseCat6D evaluation and Metric3D v2 + POLARIS to the main text (Tables 3 and 4).
W3(a). Conventional CNN with rotation augmentation
We thank the reviewer for this important suggestion, which Reviewer UGpD also raised. The AoLP is not a plain normal map. It shifts with the analyser axis, differs between diffuse and specular reflection, and depends on the lighting. Augmentation only makes the absolute angle uninformative, so a CNN must still learn the relative-angle cues from data, whereas POLARIS discards the absolute angle by construction. We have now retrained the conventional CNN of Tables 6 and 9, which matches our encoder in size, with random rotations of the analyser axis.
| Encoder | Rotation augmentation | AbsRel ↓ | RMSE (mm) ↓ | δ1 ↑ | AbsRel, polarimetric input set to zero | Polarization gain | Gain kept under AoLP noise ↓ |
|---|---|---|---|---|---|---|---|
| Conventional CNN | no | 0.0442 | 37.7 | 0.9981 | 0.0514 | 14.0% | 99% |
| Conventional CNN | yes | 0.0461 | 37.6 | 0.9981 | 0.0489 | 5.6% | 108% |
| POLARIS (ours) | not needed | 0.0396 | 36.2 | 0.9980 | 0.0445 | 11.0% | 2% |
Augmentation thus does not account for the gap between the conventional CNN and POLARIS. The augmented CNN gains less from polarization and is unaffected by AoLP noise, so augmentation makes it rely even less on the measured AoLP. POLARIS, in contrast, is exactly invariant without augmentation (Table 6) and is the only encoder here whose gain depends on the AoLP content. We will add this CNN to Tables 6 and 9 and to a new Appendix K, and we will state clearly which augmentation each encoder was trained with.
W3(b). Conventional CNN fed only S0 and DoLP
We sincerely thank the reviewer for this perceptive observation. As the reviewer notes, the conventional CNN barely reacts to AoLP noise. To examine whether its gain comes from S0 and DoLP, we trained a variant of this CNN that receives only S0 and DoLP, with the same training recipe as in Table 9.
| HAMMER | AbsRel ↓ | RMSE (mm) ↓ | δ1 ↑ | AbsRel, polarimetric input set to zero | Polarization gain | Gain kept under AoLP noise ↓ |
|---|---|---|---|---|---|---|
| MoGe-2 (RGB-only fine-tune) | 0.0471 | 38.1 | 0.9981 | – | – | – |
| Conventional CNN, S0 and DoLP only (new) | 0.0477 | 38.2 | 0.9982 | 0.0475 | −0.3% | – |
| Conventional CNN, S0, DoLP and AoLP | 0.0442 | 37.7 | 0.9981 | 0.0514 | 14.0% | 99% |
| POLARIS (ours) | 0.0396 | 36.2 | 0.9980 | 0.0445 | 11.0% | 2% |
The variant gains nothing from polarization, as its branch changes the predictions but these changes cancel out across scenes. The conventional CNN therefore gains from polarization only when the AoLP channels are part of training, although it does not draw on their content at inference. Since the CNN of Table 9 that receives only the AoLP axis field also improves on the RGB-only MoGe-2, S0 and DoLP alone do not fully account for the gain of the conventional CNN. POLARIS, in contrast, cannot draw on S0 and DoLP alone, because they only scale the magnitude of its orientation features (Section 3.4). We will add this variant to Table 9 and revise Section 5.1 accordingly.
m1. Fine-structure claim
We are thankful to the reviewer for pointing this out, and we agree that B-F1 does not support the claim and that Figure 11 shows a single frame. On our captures, the strength of POLARIS lies instead in RMSE, s-AbsRel and st-AbsRel (Table 11). We will remove the sentence from the abstract and present Figure 11 only as a qualitative example.
m2. Illumination and the screen experiment
We appreciate these two points and agree with both. The Fresnel relation assumes unpolarized incident light, whereas skylight, light reflected by glass or roads, and the light of displays can already be polarized when they reach a surface. Such light can weaken or bias the cue and may contribute to the weaker results outdoors. In the screen experiment, the display itself polarizes the light (median DoLP 0.91 with a single AoLP direction) and the screen is planar, so the experiment cannot separate geometric reasoning from a simpler rule. It does show, however, that the polarimetric input separates the screen from the scene it displays, which MoGe-2 cannot do once the margin around the screen is small. We will state the illumination assumption in Section 3.1 and in the limitations, and limit Section 4.3, Figure 10 and the abstract to this claim.
Q2. Injection blocks and zeroing probability
We thank the reviewer for these questions about our design choices. We first removed single injections of POLARIS, or kept only one, at inference on HAMMER, and measured the share of the polarization gain that remains.
| Injection after block | 6 | 12 | 18 | 24 |
|---|---|---|---|---|
| Gain kept with this injection removed | 65% | 39% | 55% | 100% |
| Gain kept with only this injection | 20% | 25% | 18% | −6% |
The gain comes mostly from the injections after blocks 6, 12 and 18 acting together, whereas the injection after block 24, which cannot reach the scale head (our answer to W1/Q1), does not contribute. Retraining POLARIS with injections after blocks 12 and 24 only leaves a much smaller gain, in line with this ablation.
| HAMMER | AbsRel ↓ | AbsRel, polarimetric input set to zero | Polarization gain |
|---|---|---|---|
| Four injection blocks (paper) | 0.0396 | 0.0445 | 11.0% |
| Two injection blocks (after blocks 12 and 24) | 0.0424 | 0.0437 | 3.0% |
| Four injection blocks, zeroing probability 0 | 0.0466 | 0.0475 | 1.9% |
During training, all runs in the paper set the polarimetric input to zero with a probability of 0.3, so that the same weights also learn the RGB-only pass. Without zeroing, the model is less accurate and its polarization gain nearly vanishes, so zeroing does more than provide the RGB-only pass and also helps the branch draw on polarization. We will report these results in a new Appendix N (Table 17).
Q3. Scenes seen through glass
We are grateful for this interesting question. Our captures include an indoor scene that faces a large window, and this scene is part of Table 11. Its stereo reference depth (Appendix H) measures the scene behind the glass, and the evaluation uses no window mask, so our setting rewards depth beyond the window rather than depth truncated at the pane. As Figure 11 shows, POLARIS places the street beyond the pane and farther away than MoGe-2 fine-tuned on the same data (RGB only). We will add this discussion to Appendix H.
W1/Q1. Scale prediction and intrinsics
We are grateful to the reviewer for this thorough analysis. As the reviewer observes, a large part of the gain is in the metric scale, the core ambiguity of monocular depth estimation. On HAMMER, 72.6% of the polarization gain of POLARIS, measured against the same weights with the polarimetric input set to zero, lies in the global metric scale. Both the gain and its scale share are larger on strongly polarized surfaces, where an intensity-only model errs most (Figure 4(b)).
How polarization reaches the scale. We apologize that the sentence after Eq. (3), "The class token is left unchanged", caused confusion. As in MoGe-2, the scale head, a fine-tuned MLP, reads the class token of the last block and predicts the scale of the point map. The polarimetric tokens are added to the patch tokens after blocks 6, 12, 18 and 24 of the 24 blocks, and self-attention in the following blocks passes this information to the class token.
Intrinsics. All models that accept intrinsics (MoGe-2, POLARIS, Metric3D v2 and Depth Pro†) received the calibrated focal length, whereas ZoeDepth, VGGT and Marigold† take none. In POLARIS, the focal length is used only after the network to recover the shift of the point map, so neither the scale head nor the polarimetric branch sees it. We also evaluated the models without intrinsics. In this setting, MoGe-2 and POLARIS recover the focal length from the predicted point map, Depth Pro† uses its own estimate, and Metric3D v2 uses the focal length of its canonical camera.
The advantage of POLARIS and its gain from polarization therefore do not depend on the given focal length. We will make the path to the scale head explicit in Eq. (3), Section 3.3 and Figure 3, state in Section 4.1 which models receive the focal length, and report the evaluation without intrinsics in Appendix M (Table 16).