- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper addresses RGB-to-polarization estimation, i.e., predicting AoLP and DoLP from a single RGB image. It argues that appearance-only models suffer from a diffuse–specular branch ambiguity (a 90° AoLP offset). To address this, it converts normals from a frozen monocular geometry model (MoGe-2 used in the paper) into a polarization-coordinate prior. This prior conditions a lightweight four-step pixel-space residual-shifting diffusion model GeoPolar, which outperforms latent-diffusion baselines on a combined benchmark and an unseen-camera set, at much lower cost.
Critical strengths
The analysis of the form of the geometry condition (Sec. 5.3, Table 4, Appendix G) is helpful. By rewriting the condition at test time, the authors show that the azimuth carries most of the useful information. A raw normal map is worth no more than g = 0, so the polarization-coordinate encoding matters, not just the presence of geometry. Replacing g with another image’s geometry hurts further, showing that the model follows the content of the prior.
Critical weaknesses
-
The motivation for the task is unclear. It is not clear why polarization information needs to be estimated from RGB images in the first place. The introduction cites several downstream applications of polarization images to motivate the task. However, the paper provides no experiment showing that the polarization predicted by the proposed method is more faithful and actually benefits any of these downstream tasks.
-
The reliance on estimated normals is not adequately justified. Shape from polarization is typically used precisely to recover surface normals where RGB alone struggles, such as texture-less or reflective surfaces. The proposed method reverses this logic and depends on an input normal map. This dependency directly limits the method. Table 5 shows that once the normal error exceeds 20° (std of 30° and 45°), both AoLP and DoLP errors become worse than the w/o-geometry model. Monocular normal estimators are least reliable on exactly the surfaces where polarization is most useful, such as glass and specular objects (Appendix I). This raises further questions the paper leaves unanswered. What is the purpose of using a geometry prior to estimate polarization? For which downstream polarization applications does geometry-assisted, more accurate polarization actually help, and for which is it redundant? For example, if a normal map is already available, recovering normals from the predicted polarization seems circular. The paper lacks both the discussion and the experiments needed to answer these questions.
-
The comparison with baselines is not fully convincing. All baselines in Tables 1–2 are retrained by the authors on their own split rather than evaluated with released weights, and the retrained models appear substantially weaker than the originals.
- On the unseen-camera set, which no model has seen, the released PolarAnything (T=20) reaches 38.6° AoLP_m, versus 47.2° for the retrained version (Table 11).
- On the combined benchmark, the released GenPolar is far stronger than the retrained one: 16.5° vs. 33.4° on RLP and 4.4° vs. 21.9° on RSP (Table 10). The authors attribute this to possible train–test overlap, but this is not verified.
As a result, the claimed margins over prior methods may partly reflect suboptimal reproduction. The most reliable evidence in the paper is the comparison against its own w/o-geometry control, not against the state of the art.
Questions for authors
- Clarify the practical motivation for RGB-to-polarization estimation. Evaluate on at least one or two downstream tasks cited in the introduction, using each method’s predicted polarization.
- Justify relying on estimated normals, given that SfP is mainly needed where monocular normals fail. Discuss and test which downstream applications benefit from geometry-assisted polarization and which become redundant.
- Strengthen the baseline comparison:
- Verify the retraining of PolarAnything and GenPolar against their original training settings.
- Report results with released weights on data confirmed to be disjoint from their training sets.
- Explain why the retrained models are much weaker even on the unseen-camera set.
9 comments
W2 and Q2. Role of the estimated normals
We are grateful for this insightful comment, which raises a point that we have also considered carefully. When a normal map is available, recovering normals from our polarization is indeed circular, and we do not claim gains in normal estimation (Sec. 5.5, last paragraph). The predicted polarization is, however, not a re-encoding of the normals, because it contains information that the normals alone cannot provide.
Polarization rendered from the normals does not reproduce the measured DoLP. Rendering the same MoGe-2 normals with the diffuse or the specular dielectric model for every pixel gives the following errors on DoFP, RLP and RSP (Table 18, Appendix I).
| AoLP$_m$ (°) | DoLP$_m$ | |
|---|---|---|
| diffuse model | 64.2 / 44.3 / 54.0 | 0.13 / 0.15 / 0.12 |
| specular model | 25.8 / 45.7 / 36.0 | 0.41 / 0.39 / 0.43 |
| oracle (better model per pixel, chosen with the ground truth) | 17.2 / 15.8 / 11.6 | 0.39 / 0.24 / 0.35 |
| GeoPolar | 11.7 / 28.2 / 13.4 | 0.055 / 0.105 / 0.065 |
The normals do not determine the material response. On NeRSP, whose objects are all dielectric, the material head of SfPUEL classifies 73–84% of the pixels of the polarization rendered from the MoGe-2 normals as metal, but only 18% of those of GeoPolar. The DoLP depends on the material, which the normals do not carry.
Real estimator errors do not make the prior harmful. On HAMMER, we grouped the pixels by the actual azimuth error of MoGe-2 (Table 9, Appendix M; AoLP$_m$, °).
| MoGe-2 azimuth error | pixels | GeoPolar | w/o geometry | GenPolar |
|---|---|---|---|---|
| 0–5° | 69.5% | 20.6 | 33.4 | 20.9 |
| 5–10° | 15.0% | 25.8 | 33.9 | 26.4 |
| 10–20° | 7.5% | 29.9 | 36.4 | 33.7 |
| 20–30° | 3.0% | 34.5 | 38.7 | 39.2 |
| 30–45° | 2.2% | 36.3 | 39.9 | 39.1 |
| 45–90° | 2.8% | 37.0 | 39.3 | 38.3 |
GeoPolar is more accurate than the model without geometry in every bin, including the pixels where the prior deviates by 45–90°. Unlike these real errors, the σ ≥ 30° rows of Table 6 add content-independent noise to every pixel. Fig. 16 shows examples of the real errors of the estimator, and Fig. 15 compares them with the synthetic perturbations (Appendix M).
Where geometry-assisted polarization helps and where it is redundant. As stated in Sec. 5.5, geometry-assisted polarization is redundant for recovering normals when the normals are already available. It is beneficial for tasks that require the polarization itself beyond the surface orientation, namely the DoLP and the material response, as in material classification and reflection removal.
W3 and Q3. Baseline retraining and released weights
We sincerely thank the reviewer for examining our reproduction so carefully, and we acknowledge this concern. We intended to follow the setting of GenPolar exactly, but this was not possible, because two of its training datasets (OPID and IPID) were not public at the time of our submission and its exact train/test split is still not specified. We therefore retrained GenPolar with the public code, following the setting of the paper as closely as possible. In our submission, we trained Stage I for 258 epochs, compared with 100 epochs in the GenPolar paper. Since the model still appeared under-trained, we continued Stage I to 800 epochs. This improved GenPolar on all three subsets of our test split (RLP from 33.4° to 31.5°, DoFP from 27.2° to 21.3° and RSP from 21.9° to 18.5°), while GeoPolar remains more accurate on each subset (28.2°, 11.7° and 13.4°). The revision reports this model in Table 2 and all other tables (Appendix F), and Table 14 compares the two training settings.
| GenPolar paper | ours, submission | ours, revision | |
|---|---|---|---|
| training data | public sources and OPID, IPID (not public at our submission) | our training split (8 public sources) | same |
| train / test split | 50 random images per group held out, list not released | our split, identical for all compared models | same |
| Stage I / Stage II epochs | 100 / 30 | 258 / 30 | 800 / 30 |
| prediction target | , Stokes ×1 (paper text, code default) | , Stokes ×5 (recipe of the released weights) | same |
| effective batch, GPUs | 4, 4 × A40 | 1, 1 × RTX PRO 6000 | same |
| GPU-hours | – | 159 | 401 |
All compared models, namely GeoPolar, the model without geometry and the retrained GenPolar, were trained on the same training split. The released weights were trained with -prediction and a 5× Stokes target, as stated in the inference code of the authors ("Legacy release weights used a 5x Stokes target") and confirmed by reproducing their quick test.
Released weights on fully disjoint data. We evaluated all models on three datasets that none of them was trained on (Table 20, Appendix L; AoLP$_m$, °).
| dataset (images) | GeoPolar | GenPolar retrained | GenPolar released | PolarAnything retrained | PolarAnything released |
|---|---|---|---|---|---|
| HAMMER (146) | 23.9 | 24.9 | 50.8 | 32.1 | 36.5 |
| MCubeS (102) | 33.9 | 38.3 | 49.1 | 38.7 | 36.2 |
| NeRSP (258) | 18.8 | 23.8 | 25.5 | 29.8 | 31.4 |
On every disjoint dataset, our retrained GenPolar is more accurate than the released weights, which indicates that our retraining is not weaker. GenPolar holds out only 50 random images per group from the same public pool from which our test split is drawn, so a large part of our test split is expected to be in its training set. The strength of the released model on our test split (16.5° on RLP and 4.4° on RSP; Table 15, Appendix G) is therefore likely to result from this overlap.
Our prior also improves GenPolar. We added our prior to GenPolar as an additional ControlNet condition, using the same code, data and recipe (Sec. 4.1, Appendix F). GenPolar with our prior is more accurate in AoLP on all three subsets (Table 2, Sec. 4.2), and its Stage I training loss is lower.
| Stage I | Stage II evaluation, AoLP$_m$ (°) / DoLP$_m$ | |||
|---|---|---|---|---|
| training loss (AoP / total) | RLP | DoFP | RSP | |
| GenPolar | 0.110 / 0.094 | 31.5 / 0.126 | 21.3 / 0.095 | 18.5 / 0.087 |
| GenPolar + our prior | 0.106 / 0.091 | 28.6 / 0.121 | 15.2 / 0.080 | 17.3 / 0.090 |
The polarization-coordinate prior is therefore a general component rather than a property of our backbone.
Overall response
We thank the reviewer for the careful and constructive review, and we appreciate the recognition that our analysis of the form of the condition shows that the polarization-coordinate encoding, rather than the mere presence of geometry, is what matters. In this response, we address each concern as follows.
All changes are highlighted in blue in the revised PDF, and all table, figure, section and appendix numbers below refer to the revision.
W1 and Q1. Motivation and downstream evaluation
This valuable comment is well taken, and we fully agree that the task should be supported by downstream evidence. Polarization cameras are rare compared with RGB cameras, and polarization datasets are small. Estimating polarization from RGB makes polarization-based methods applicable to images and cameras without a polarizer, which is also the goal of PolarAnything and GenPolar. We evaluated four downstream tasks cited in Sec. 1 and replaced the captured polarization by the prediction of each method (Sec. 5.5, Table 10; protocols in Appendix N). The tasks include the protocols of PolarAnything (multi-view reconstruction with PISR on NeRSP) and GenPolar (material classification with SfPUEL and reflection removal with PolarFree).
Normal errors are given in degrees, and bold marks the best result among the predicted polarizations. The balanced accuracy averages the recall of metals and the accuracy of dielectrics, since PolarAnything classifies nearly every pixel as dielectric.
GeoPolar also performs well in these downstream tasks. Among the predicted polarizations, it yields the most accurate normals in single-view and multi-view shape from polarization and the highest balanced accuracy in material classification, and it remains close to captured polarization in reflection removal. Figs. 8, 17 and 18 show examples of material classification and reflection removal.