acceptodds
Under review as a conference paper at ICLR 2027

Embedding Physical Reasoning into Diffusion-Based Shadow Generation Under the Sun and Sky

Abstract

Generating realistic cast shadows for inserted foreground objects requires reasoning about scene geometry and illumination. However, most learning-based approaches treat shadow generation as an image translation problem and capture these physical relationships only implicitly. This often results in misaligned or implausible shadows. Motivated by the physics of shadow formation, we introduce explicit geometric guidance for outdoor shadow generation. Given a composite image and a foreground object mask, we recover approximate scene geometry and estimate a dominant light direction to derive a coarse shadow estimate via geometric reasoning. While coarse, this estimate provides a spatial anchor for shadow placement. Because illumination cannot always be uniquely inferred from a single image, we predict confidence scores for both lighting and shadow cues and use them to regulate their influence during generation. These cues (shadow mask, light direction, and their confidence scores) condition a diffusion-based generator that refines the estimate into a realistic shadow. Experiments on DESOBAV2 show substantially improved shadow-region fidelity and localization, with an overall 23% lower shadow-region RMSE and 30% lower shadow-mask BER than the prior state-of-the-art method.

Primary area:
generative models
Keywords:
Shadow Generation, Image Composition, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rVjNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work targets outdoor cast‑shadow generation for composited foreground objects under sun‑sky illumination. The proposed pipeline injects explicit physical reasoning into a diffusion generator. It first leverages the monocular geometry estimator MoGe‑2 to obtain 3D point maps, then predicts a dominant 3D directional light direction, computes a coarse geometric shadow prior via ray alignment, and refines it into a full‑resolution shadow mask. The results on DESOBAV2 show that the method surpasses prior state‑of‑the‑art GPSDiffusion. Ablation studies validate the necessity of geometry input, predictor modules, and confidence gating.

Critical strengths
  1. Physically grounded, confidence‑aware conditioning that addresses the ill‑posed nature of single‑image physical inference. This paper combines explicit monocular 3D geometry + geometric ray‑casting to produce a physical shadow anchor. This is a meaningful methodological advance for physically‑aware generation task.

  2. Strong quantitative, qualitative, and ablation evidence on DESOBAV2 benchmark. The paper provides comprehensive comparisons against GAN and diffusion baselines. Targeted ablations isolate the contribution of point‑map guidance, light/mask predictors, and quality gating.

Critical weaknesses
  1. Heavy dependency on an external off‑the‑shell monocular metric‑geometry model (MoGe‑2), with limited analysis of geometry‑estimation failure. All geometric shadow priors rest entirely on MoGe‑2 point‑maps. This paper does not systematically quantify how geometry quality degrades end‑to‑end shadow performance. There is no experiment replacing MoGe‑2 with alternative depth/geometry estimators to test generalisability of the proposed framework.

  2. Only models a single dominant directional light and lacks explicit shadow‑attenuation modelling. The whole pipeline assumes one dominant outdoor directional sunlight. It cannot natively handle complex spatially‑varying multi‑light or soft ambient‑dominated illumination scenarios. While reasonable for sunny outdoor scenes, there is no study quantifying performance degradation under non‑single‑dominant‑light conditions.

Questions for authors
  1. How robust is your full pipeline to noisy or erroneous outputs from MoGe‑2? Could you provide quantitative results where you substitute MoGe‑2 with alternative monocular geometry/depth models?
  2. In your oracle ablation, ground‑truth shadow mask and pseudo‑label light direction yield a large performance jump. What are the primary failure modes of your learned light predictor and mask corrector that prevent approaching this oracle upper‑bound on real data?
  3. Future work may extend beyond single dominant directional light to handle spatially‑varying illumination and incorporate physical shadow‑attenuation modelling.
2 comments
AuthorsOpenReview ↗

Response to Reviewer rVjN

Thank you for the positive assessment of our geometric guidance and experimental results. The two main questions and our brief responses are: 1) Geometry robustness. We provide point-map perturbation tests and replacement with UniDepthV2, without retraining. 2) The oracle gap. Our additional oracle comparisons identify mask accuracy and training-time adaptation as major factors behind the gap. We also agree that a more general, physics-based model of illumination and shadow attenuation is a valuable direction for future work.

[W1: Geometry robustness] [Q1] How robust is the pipeline to geometry errors?

Final generation degrades moderately under the tested geometry perturbations, and the pipeline can work with UniDepthV2 without retraining. We add spatially smooth relative-depth noise to MoGe-2 outputs at two strengths and separately replace MoGe-2 with UniDepthV2. We evaluate the unchanged pipeline on all 750 test cases (Appendix C.4, Table 9):

Geometry inputLRMSE ↓LBER ↓
Saved MoGe-2 reference34.3950.1915
5% noise36.4060.1967
15% noise36.4460.1988
UniDepthV237.2270.2070

Generation performance changes little between the two noise strengths, while the intermediate mask prediction is more sensitive: its Dice decreases from 0.4487 to 0.3602, and predicted mask quality decreases from 0.4818 to 0.3996. Replacing MoGe-2 with UniDepthV2 increases LRMSE by 8.2% and LBER by 8.1%, but both remain below GPSDiffusion's 44.993 and 0.2721, respectively.

[Q2: Oracle gap] What accounts for the gap to oracle conditioning?

Our additional experiments point to mask accuracy and training-time adaptation as major factors in the oracle gap. Under predicted-light conditioning, using the GT mask only at inference gives 37.044 LRMSE; training with the GT mask reduces it to 21.906 and LBER to 0.0315, approaching the both-cue oracle. Pseudo-light replacement alone does not close the gap, with LRMSE of 38.446 for inference-only replacement and 43.855 when also used during training.

We compare inference-time substitutions on our final configuration's frozen checkpoint with separately trained oracle variants (Appendix C.2, Tables 6 and 7). Light replacement uses the pseudo-label and recomputes the light-dependent predicted mask. All results use the same 235 valid-light BOS-free cases, with both quality gates fixed to 1:

Oracle useLight / mask conditioningLRMSE ↓LSSIM ↑LBER ↓
Inference onlyPseudo-light / predicted mask38.4460.30640.1478
Inference onlyPredicted light / GT mask37.0440.32320.1320
Inference onlyPseudo-light / GT mask36.8660.32330.1298
Training + inferencePseudo-light / predicted mask43.8550.24740.2017
Training + inferencePredicted light / GT mask21.9060.54540.0315
Training + inferencePseudo-light / GT mask18.3620.58340.0316

These results suggest that accurate masks simplify the generator's task during training, allowing it to focus more on shadow appearance rather than correcting uncertain shape and placement. The lower LRMSE of the combined oracle configuration suggests a complementary benefit from light-direction guidance (Table 6, Row 7).

[W2: Lighting and attenuation] [Q3] Future work

We thank the reviewer for highlighting this valuable future direction. Under our single-dominant-light assumption, we do not explicitly model spatially varying illumination or shadow attenuation (Appendix D), and the existing benchmark also does not provide enough multi-light or ambient-dominated scenes for a meaningful separate evaluation. With more diverse data, a more general, physics-based model of illumination and shadow attenuation would be a natural next step.

With less than one day remaining in the discussion period, we would appreciate it if you could review our responses and additional results. Please let us know whether they address your concerns or whether anything remains that we can clarify in the time available. If your concerns have been addressed, we would be grateful if you would consider updating your assessment accordingly.

Thank you again for your time and consideration.

Reviewer_wEviOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes physics-guided cast-shadow generation for outdoor image composites. Given a composite image and foreground mask, it recovers a monocular point map, predicts a dominant light direction, and computes a coarse geometric shadow prior. A mask corrector refines the prior, while quality estimates regulate the light and mask cues supplied to a diffusion generator. DESOBAV2 experiments show improved shadow fidelity and localization in BOS and BOS-free settings. The submission also reports evaluation without best-of-five selection, fixed-quality sensitivity analysis, background-mask conditioning, and performance on samples without valid light-direction pseudo-labels.

Critical strengths
  • Appropriate treatment of unreliable physical guidance. Single-image illumination and monocular geometry are inherently uncertain. Estimating the reliability of light and mask predictions and adjusting their conditioning strengths is a well-motivated design. The confidence-error correlations and conditioning-sensitivity results support the usefulness of these reliability signals.

  • Encouraging benchmark results with useful additional diagnostics. The method improves shadow-region fidelity and localization over GPSDiffusion in both BOS and BOS-free settings. Gains persist without ground-truth-based best-of-five selection, and the 140-sample subset without valid light-direction pseudo-labels provides evidence that improvements are not confined to cases where the offline labeling pipeline succeeds.

Critical weaknesses
  • Light-estimation information sources and robustness are insufficiently disentangled. The light predictor sees the composite RGB image, foreground mask, and point map, so it can use foreground shading, appearance, and shape. These are legitimate illumination cues when foreground and background lighting agree. However, the experiments do not establish robustness when an inserted object carries different lighting. The BOS/BOS-free split tests reference-shadow availability rather than separating foreground and background evidence. Appendix D replaces the foreground mask with a background mask, but does not state that foreground RGB or point-map information is removed. If the masks are complements, this transformation retains their information. Table 6 reports final generation metrics rather than the light-direction accuracy of this variant. Angular errors against geometry-derived pseudo-labels also do not resolve the information-source question. Consequently, the evidence does not distinguish scene-light estimation from reliance on foreground illumination cues that may be specific to the benchmark.

  • The specific benefit and geometric scope of physical conditioning remain incompletely validated. In Table 4, point-map conditioning alone reduces BOS-free LRMSE from 52.602 to 37.010; the full model reaches 36.126. Thus, much of the total improvement is already obtained without the explicit light and shadow-mask predictors. The full configuration improves further, but the ungated predictor configuration also changes whether predictors are updated during generator training, complicating attribution specifically to quality gating. Appendix C appropriately describes fixed-quality replacement as inference-time sensitivity analysis; it does not establish the same advantage over matched models trained with fixed conditioning. A cleaner separation of these factors is important for assessing the physical-reasoning contribution. Geometric claims should also reflect the tested scope: uneven or textured surfaces do not establish performance when shadows cross distinct receiver surfaces, and light pseudo-label acquisition uses a planar receiver approximation.

Questions for authors
  • Inputs and robustness of light prediction. In the background-mask variant, is only the mask channel changed, or are foreground RGB and point-map values also removed? Please define the background mask and report any available light-direction accuracy analysis. What evidence separates the influence of foreground appearance from background lighting, especially when their illumination differs?

  • Attribution of the physical-conditioning gains. Is there a matched comparison isolating the geometric shadow prior and quality gating while keeping predictor freezing and other training choices fixed? What evidence establishes their contribution beyond point-map conditioning and the inference-time sensitivity analysis?

  • Scope of receiver geometry. What receiver configurations are represented in the test set, and are there evaluated cases where shadows cross wall-floor boundaries, steps, or other depth discontinuities? Please clarify how such cases are handled by planar pseudo-label fitting and which geometric generalization claims the reported evidence supports.

6 comments

We thank the reviewer for their feedback. We would like to briefly discuss three major concerns upfront. 1) Input cues and lighting robustness. We clarify that the same input setting and task scope apply to all compared shadow-generation methods, and provide additional analyses of light-direction accuracy and foreground sensitivity. 2) Insufficient evidence beyond point-map conditioning. We have provided new matched ablations to establish the additional contributions of the alignment prior and quality gating. 3) Receiver-geometry scope. We clarify that the planar approximation applies to offline pseudo-label acquisition, not generation-time guidance, and have narrowed our receiver-generalization claims to reflect the available evidence.

Our detailed responses to all questions follow.

AuthorsOpenReview ↗

Response to Reviewer wEvi

[W1: Input cues and lighting robustness] [Q1] Light-prediction inputs and foreground illumination

We agree that our framework uses foreground illumination cues. This is the same input setting used across all compared shadow-generation methods, where foreground appearance provides useful illumination information. Its reliability can decrease when foreground appearance is corrupted by noise or when foreground and background lighting differ. Harmonizing foreground and background illumination is beyond the scope of this task (Appendix D).

1. Background-mask inputs

Only the mask channel changes from the foreground mask MfoM_{fo} to its complement 1−Mfo1-M_{fo}. Foreground RGB and point-map values remain available (Appendix C.2). This is a change in mask conditioning, not an exclusion of foreground input information.

2. Light-direction accuracy

The mean angular errors of the foreground- and background-mask variants are:

Light-predictor conditioningBOS (375 samples) ↓BOS-free (235 samples) ↓
Foreground mask6.78°10.11°
Background mask4.17°9.20°

Although the background-mask variant has lower angular error against the pseudo-labels, its BOS-free generation results are slightly worse (Table 6). Angular error therefore serves as an intermediate diagnostic rather than a measure of final shadow generation quality.

3. Foreground sensitivity and conflicting lighting

Foreground corruption reduces prediction accuracy, while our method retains an advantage over GPSDiffusion under the same perturbation:

Method / inputLRMSE ↓LSSIM ↑LBER ↓
Ours — clean34.400.46260.1915
Ours — noise + jitter38.320.41960.2104
Ours — 2× noise + jitter41.040.38170.2389
GPSDiffusion — clean44.990.35910.2721
GPSDiffusion — noise + jitter48.120.33420.3058

These tests perturb foreground appearance while retaining cached geometry. On valid-light BOS-free samples, mean angular error also increases from 10.11° to 11.04° and 13.25° under the two corruption strengths, confirming sensitivity to foreground appearance.

On another set of 100 cross-image composites, where foreground and background illumination are not guaranteed to match, median qlightq_{\mathrm{light}} decreases from 0.710 to 0.628 and median qmaskq_{\mathrm{mask}} from 0.563 to 0.487 relative to the standard test set. These lower scores are consistent with reduced confidence under changed compositing conditions. Without paired ground-truth shadows, quantitative evaluation of generation quality is unavailable.

[W2: Insufficient evidence] [Q2] Contributions of the alignment prior and quality gating

We agree that the point map provides the major improvement in generation quality by supplying explicit geometric information for coarse shadow placement. To assess the additional contributions of the alignment prior and quality gating, we ran two matched comparisons.

First, under matched mask-predictor training, adding the direction-alignment prior improves full-resolution mask Dice from 0.5138 to 0.5479 across all 750 test cases (0.5024 → 0.5497 in BOS; 0.5364 → 0.5442 in BOS-free). Both variants receive the same image, object mask, point map, and frozen light prediction; only the prior channel is zeroed in the comparison variant. This matched ablation is included in Appendix C.3 (Table 8).

Second, we have added the PointMap ✓, Light ✓, Mask ✓, Quality — configuration as Configuration 5 in Table 4. Its predictors remain frozen, while both quality scores are fixed to 1 throughout training and inference:

Quality gatesBOS LRMSE ↓BOS LBER ↓BOS-free LRMSE ↓BOS-free LBER ↓
Fixed gates (q=1q=1)40.8990.287045.8280.2323
Predicted quality gates33.5300.211536.1260.1514

These comparisons show that the explicit prior improves mask prediction, while adaptive quality weighting improves generation with the predictors frozen in both models.

[W2: Receiver-geometry scope] [Q3] Receiver geometry and the planar approximation

We acknowledge that our experiments do not establish generalization to multi-plane receivers, such as shadows crossing wall–floor boundaries or steps. The planar receiver assumption applies only to offline light pseudo-label acquisition. During generation, the soft shadow prior scores direction alignment between object points and candidate non-object points in the point map, without fitting a receiver plane. However, DESOBAV2 does not provide receiver-type annotations, so we do not report separate quantitative results for these complex configurations. We have clarified this distinction in Section 3.1 and Appendix G, narrowed our qualitative claims to the illustrated examples, and stated the receiver-generalization limitation in Appendix D.

Thank you for the detailed responses and additional analyses. The matched ablations provide stronger evidence for the contributions of the alignment prior and quality gating, addressing the earlier confounding factors. I also appreciate the clarification of the planar approximation and the narrowing of the receiver-generalization claims.

However, my concern about the robustness of light estimation is only partially addressed. Foreground appearance is a legitimate illumination cue when foreground and background lighting are consistent. The unresolved question is how the method handles conflicting illumination, which is relevant to object insertion. Noise and color perturbations demonstrate foreground sensitivity, but do not directly test this conflict. Likewise, lower confidence scores on cross-image composites do not establish that the scores identify lighting-estimation errors or that the resulting shadows remain consistent with background illumination.

Although foreground–background harmonization may be outside the task scope, selecting reliable lighting cues for shadow generation remains relevant to the proposed framework. Correcting foreground appearance and discounting misleading foreground illumination are distinct issues. This concern is particularly consequential because reliability-aware use of uncertain physical guidance is a central contribution. A controlled comparison that fixes the background, object geometry, and placement while varying foreground illumination would provide more direct evidence. If consistent foreground–background lighting is instead a prerequisite, this assumption and its implications for applicability should be stated explicitly.

The rebuttal strengthens the component-level validation, but the remaining evidence does not sufficiently establish robustness to conflicting illumination. Given the importance of this issue to the intended application and the proposed reliability mechanism, I maintain my Weak Reject recommendation.

Reviewer_Lt8iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses shadow generation for image compositing, including scenes without informative background object–shadow pairs. It combines a pretrained monocular point-map estimator with learned dominant-light prediction, a coarse shadow prior based on three-dimensional direction alignment, and learned mask refinement. Predicted cue-quality scores modulate spatial and lighting conditions supplied to a frozen diffusion backbone through trainable conditioning branches. The proposed method is best understood as geometry-conditioned diffusion with adaptive cue weighting in my view. Experiments on DESOBAv2 report improvements over existing shadow generators under both best-of-five and independently sampled single-output evaluation, with additional analyses of cue prediction and reliability.

Critical strengths
  1. The complete system provides substantial, practically relevant improvements. In the BOS-free setting, single-output evaluation averaged over independent draws reduces LRMSE from 61.920 for GPSDiffusion to 43.330. Thus, the gains do not depend entirely on ground-truth-based best-of-five selection. The diagnostic evaluation on test examples without valid light pseudo-labels further supports coverage beyond the easiest labeling cases.

  2. The treatment of imperfect intermediate predictions is sensible. The geometric prior is used as a soft condition, and predicted quality scores correlate with cue errors. Combining spatial guidance with scene-level lighting conditioning is a reasonable design for ambiguous compositing inputs.

Critical weaknesses
  1. The evidence does not sufficiently establish the incremental value of the central proposed mechanisms. In the BOS-free ablation, point-map conditioning alone changes LRMSE from 52.602 to 37.010, while the full system reaches 36.126. Thus, most of the observed gap to GPSDiffusion is already present before the full proposed mechanism. Moreover, the ungated predictor configuration updates predictors during generator training, whereas the complete configuration freezes them. The fixed-gate experiment changes gates only at inference in a model trained with predicted gates. As the appendix correctly notes, this is a sensitivity analysis; it does not establish superiority over independently trained fixed-gate models. These limitations matter because the paper's contribution rests on explicit geometric reasoning and reliability weighting beyond a strong geometry-conditioned diffusion baseline.

  2. The distinction from existing geometry-and-learning approaches is narrower than the framing suggests. Pixel-height-based shadow generation (ECCV 2022) already supports learned height prediction as well as manual editing, and PixHt-Lab (CVPR 2023) combines explicit geometry with learned lighting effects. The present contribution is more specifically automatic cue estimation and conditioning for this task. Direction errors measured against internally generated pseudo-labels establish agreement with that labeling process, rather than independent physical-light accuracy. The current experiments do not yet demonstrate a distinctive physical-modeling advantage beyond better geometric conditioning.

  3. It is not ‌convincing‌ that the method is a physical decomposition of shadow formation. A single light direction provides orientation but not irradiance, angular source extent, contact darkening, material response, ambient sky, or visibility. The paper’s own limitation section concedes that attenuation is not represented. The core novelty is conditional feature gating, which is a standard reliability-aware fusion pattern.

Questions for authors
  1. What evidence isolates the benefit of the explicit direction-alignment prior from the point-map information itself? In particular, is there an existing matched comparison against a learned mask predictor receiving the same image, object mask, and point map but no explicit ray-direction prior? This would materially affect my assessment of the proposed physical reasoning contribution.

  2. Can you separate the effect of quality weighting from the frozen-versus-updated predictor difference in the conditioning ablation? If matched frozen-predictor or independently trained constant-gate results already exist, please report them; otherwise, please delimit the conclusions supported by the current sensitivity analysis.

  3. What exact angular search grid, density-threshold percentile, logistic slope, and acceptance rules were used for the Appendix G light pseudo-labels? A concrete configuration and manual-rejection rubric would clarify how reproducible the supervision and angular-error evaluation are.

  4. Can the authors compare against GPSDiffusion using the same diffusion backbone, training data, sampling steps, and post-processing, with only the point-map/light cues and confidence gating varied? This would show whether gains come from the proposed cues or from pipeline differences.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Lt8i

We thank the reviewer for their feedback. We would like to briefly discuss two major concerns upfront that we believe are most central to the rating. 1) Insufficient evidence for the central contributions beyond point-map conditioning: For this, we have provided new matched ablations and explanations to establish the additional value of the alignment prior and quality gating. 2) Our method is not a complete physical model of shadow formation: The reviewer is correct, but we have not claimed this. Our framework is physics-guided and not a physical model of shadow decomposition.

Our detailed responses to all questions follow.

[W1: Insufficient evidence] [Q1] Is there an existing matched comparison against a learned mask predictor receiving the same image, object mask, and point map but no explicit ray-direction prior?

We ran this exact experiment: two mask predictors trained under identical settings, differing only in whether the geometry-based ray-direction prior is provided as an input channel, which is zeroed out in the no-prior variant. Results on the DESOBAV2 test set:

SettingWithout explicit prior, Dice ↑With explicit prior, Dice ↑
All0.51380.5479
BOS0.50240.5497
BOS-free0.53640.5442

The prior improves mask prediction in both settings, with a larger gain in BOS. This analysis is included in Appendix C.3 (Table 8).

[W1: Insufficient evidence] [Q2] Does adaptive quality gating outperform fixed-gate conditioning?

Yes. With predictors frozen in both models, adaptive quality weighting outperforms a control trained and evaluated with both gates fixed to 1 on all six BOS-free metrics. We agree that the original ablation did not isolate quality weighting from predictor updates, and have extended Table 4 with this matched control:

Quality settingGRMSE ↓LRMSE ↓GSSIM ↑LSSIM ↑GBER ↓LBER ↓
Fixed 1 (training and inference)12.7445.830.91840.23680.12060.2323
Fixed 0 (inference only)11.0038.840.92440.29390.08030.1535
Fixed 0.5 (inference only)10.9738.690.92460.29640.07990.1528
Fixed 1 (inference only)11.2439.260.92400.29500.08240.1573
Predicted9.7836.130.93350.32250.07930.1514

The first row corresponds to the PointMap ✓, Light ✓, Mask ✓, Quality — configuration now included as Configuration 5 in Table 4, with predictors frozen and both quality scores fixed to 1 throughout training and inference. The remaining rows are from Table 5 (Appendix C.1).

[W1: Insufficient evidence] Is the point map the major contributor to the improvement?

Yes. The point map provides the major improvement in generation quality by supplying explicit geometric information for coarse shadow placement. Improving from coarse shadow placement to more realistic shadow shapes and boundaries also benefits from the other components, as shown in our ablation studies.

AuthorsOpenReview ↗

Response to Reviewer Lt8i (cont’d)

[W2, W3: Physical-modeling scope] Contribution and distinction from prior work

Sheng et al. (2022) and PixHt-Lab (Sheng et al., 2023) are important precedents for combining explicit geometry with learned shadow synthesis, and we have revised our discussion in Section 2. Both directly use predicted or manually provided geometry and take specified lighting parameters as input for rendering. Our setting is different in two ways: light direction is estimated from the scene, and we treat the geometry-derived shadow estimate and predicted lighting as uncertain guidance, with predicted quality scores regulating their influence during diffusion generation.

Our contribution is therefore the explicit construction and reliability-aware use of geometry and illumination guidance for outdoor cast-shadow generation, and we do not claim a complete physical decomposition of shadow formation. The coarse shadow estimate explicitly relates approximate monocular geometry to the predicted light direction. Light direction provides orientation, while learned refinement and diffusion synthesis implicitly handle aspects of shadow detail and appearance that a single directional cue does not capture, including attenuation, softness, and surface interaction (Appendix D). The reliability-aware conditioning is not incidental to our approach. It allows imperfect physical guidance to help rather than hurt, as the comparison between Configurations 5 and 6 in Table 4 demonstrates.

[Q3] Pseudo-label reproducibility and interpretation

The concrete configuration used for light-direction pseudo-label acquisition is:

  • Coarse angular search: Azimuth from −180° to 180° in 20° increments; elevation from −90° to 90° in 10° increments, including endpoints.
  • Refinement: Adam for up to 1,000 iterations, with cosine learning-rate decay from 0.1 to 0.01; early stopping after 100 iterations without improvement of at least 10−610^{-6}.
  • Density threshold: 0.2Dmax⁡0.2D_{\max}, where Dmax⁡D_{\max} is the maximum density of the corresponding map.
  • Logistic flattening: R=sigmoid⁡((D−0.2Dmax⁡)/(0.1Dmax⁡+ϵ))R=\operatorname{sigmoid}((D-0.2D_{\max})/(0.1D_{\max}+\epsilon)), corresponding to a sigmoid slope of 10 after peak normalization.

For automatic screening, scenes whose selected shadow occupies less than 0.4% of the image are skipped. Estimates with a fitting objective above 0.3 or negative elevation are flagged for manual inspection, not automatically rejected. Manual verification compares the rendered and observed shadow support: estimates are rejected when shadow location or shape is inconsistent, or when incomplete shadow observations prevent reliable light estimation. This is a qualitative acceptance check rather than an additional numerical cutoff.

The reported angular errors quantify agreement with the derived pseudo-labels and serve as an intermediate diagnostic, while generation quality is evaluated separately against ground-truth images and shadow masks. We have clarified this evaluation reference and specified the acquisition settings and manual-verification criteria in Appendix G.

[Q4] Do the gains come from the proposed conditioning?

Yes. GPSDiffusion and our method are trained on DESOBAV2 and use the same frozen diffusion backbone, sampling steps, and post-processing. The differences lie in the conditioning cues and pathways. Tables 4 and 6 compare GPSDiffusion (Configuration 1) with our ablated and full versions.

With less than one day remaining in the discussion period, we would appreciate it if you could review our responses and additional results. Please let us know whether they address your concerns or whether anything remains that we can clarify in the time available. If your concerns have been addressed, we would be grateful if you would consider updating your assessment accordingly.

Thank you again for your time and consideration.

I appreciate the concrete reply from the authors, which have addressed my partial concerns by more experiments and analysis. However, after carefully thinking the novelty and feasibility of this study and considering the comments from other reviewers, I decide to retain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.