- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work targets outdoor cast‑shadow generation for composited foreground objects under sun‑sky illumination. The proposed pipeline injects explicit physical reasoning into a diffusion generator. It first leverages the monocular geometry estimator MoGe‑2 to obtain 3D point maps, then predicts a dominant 3D directional light direction, computes a coarse geometric shadow prior via ray alignment, and refines it into a full‑resolution shadow mask. The results on DESOBAV2 show that the method surpasses prior state‑of‑the‑art GPSDiffusion. Ablation studies validate the necessity of geometry input, predictor modules, and confidence gating.
Critical strengths
-
Physically grounded, confidence‑aware conditioning that addresses the ill‑posed nature of single‑image physical inference. This paper combines explicit monocular 3D geometry + geometric ray‑casting to produce a physical shadow anchor. This is a meaningful methodological advance for physically‑aware generation task.
-
Strong quantitative, qualitative, and ablation evidence on DESOBAV2 benchmark. The paper provides comprehensive comparisons against GAN and diffusion baselines. Targeted ablations isolate the contribution of point‑map guidance, light/mask predictors, and quality gating.
Critical weaknesses
-
Heavy dependency on an external off‑the‑shell monocular metric‑geometry model (MoGe‑2), with limited analysis of geometry‑estimation failure. All geometric shadow priors rest entirely on MoGe‑2 point‑maps. This paper does not systematically quantify how geometry quality degrades end‑to‑end shadow performance. There is no experiment replacing MoGe‑2 with alternative depth/geometry estimators to test generalisability of the proposed framework.
-
Only models a single dominant directional light and lacks explicit shadow‑attenuation modelling. The whole pipeline assumes one dominant outdoor directional sunlight. It cannot natively handle complex spatially‑varying multi‑light or soft ambient‑dominated illumination scenarios. While reasonable for sunny outdoor scenes, there is no study quantifying performance degradation under non‑single‑dominant‑light conditions.
Questions for authors
- How robust is your full pipeline to noisy or erroneous outputs from MoGe‑2? Could you provide quantitative results where you substitute MoGe‑2 with alternative monocular geometry/depth models?
- In your oracle ablation, ground‑truth shadow mask and pseudo‑label light direction yield a large performance jump. What are the primary failure modes of your learned light predictor and mask corrector that prevent approaching this oracle upper‑bound on real data?
- Future work may extend beyond single dominant directional light to handle spatially‑varying illumination and incorporate physical shadow‑attenuation modelling.
2 comments
With less than one day remaining in the discussion period, we would appreciate it if you could review our responses and additional results. Please let us know whether they address your concerns or whether anything remains that we can clarify in the time available. If your concerns have been addressed, we would be grateful if you would consider updating your assessment accordingly.
Thank you again for your time and consideration.
Response to Reviewer rVjN
Thank you for the positive assessment of our geometric guidance and experimental results. The two main questions and our brief responses are: 1) Geometry robustness. We provide point-map perturbation tests and replacement with UniDepthV2, without retraining. 2) The oracle gap. Our additional oracle comparisons identify mask accuracy and training-time adaptation as major factors behind the gap. We also agree that a more general, physics-based model of illumination and shadow attenuation is a valuable direction for future work.
[W1: Geometry robustness] [Q1] How robust is the pipeline to geometry errors?
Final generation degrades moderately under the tested geometry perturbations, and the pipeline can work with UniDepthV2 without retraining. We add spatially smooth relative-depth noise to MoGe-2 outputs at two strengths and separately replace MoGe-2 with UniDepthV2. We evaluate the unchanged pipeline on all 750 test cases (Appendix C.4, Table 9):
Generation performance changes little between the two noise strengths, while the intermediate mask prediction is more sensitive: its Dice decreases from 0.4487 to 0.3602, and predicted mask quality decreases from 0.4818 to 0.3996. Replacing MoGe-2 with UniDepthV2 increases LRMSE by 8.2% and LBER by 8.1%, but both remain below GPSDiffusion's 44.993 and 0.2721, respectively.
[Q2: Oracle gap] What accounts for the gap to oracle conditioning?
Our additional experiments point to mask accuracy and training-time adaptation as major factors in the oracle gap. Under predicted-light conditioning, using the GT mask only at inference gives 37.044 LRMSE; training with the GT mask reduces it to 21.906 and LBER to 0.0315, approaching the both-cue oracle. Pseudo-light replacement alone does not close the gap, with LRMSE of 38.446 for inference-only replacement and 43.855 when also used during training.
We compare inference-time substitutions on our final configuration's frozen checkpoint with separately trained oracle variants (Appendix C.2, Tables 6 and 7). Light replacement uses the pseudo-label and recomputes the light-dependent predicted mask. All results use the same 235 valid-light BOS-free cases, with both quality gates fixed to 1:
These results suggest that accurate masks simplify the generator's task during training, allowing it to focus more on shadow appearance rather than correcting uncertain shape and placement. The lower LRMSE of the combined oracle configuration suggests a complementary benefit from light-direction guidance (Table 6, Row 7).
[W2: Lighting and attenuation] [Q3] Future work
We thank the reviewer for highlighting this valuable future direction. Under our single-dominant-light assumption, we do not explicitly model spatially varying illumination or shadow attenuation (Appendix D), and the existing benchmark also does not provide enough multi-light or ambient-dominated scenes for a meaningful separate evaluation. With more diverse data, a more general, physics-based model of illumination and shadow attenuation would be a natural next step.