- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents Physical-Anchor Diffusion (PAD), a framework for foreground harmonization and cast-shadow generation in image compositing. The first stage adapts foreground appearance and illumination through intrinsic decomposition, albedo harmonization, and spherical-harmonic relighting. Then the second stage uses estimated geometry and lighting to compute a coarse shadow mask, which guides a ControlNet-based diffusion model in generating the final composite. The proposed framework connects foreground relighting with geometry-guided shadow generation. Experiments on DESOBAv2, real-world examples, and a user study evaluate shadow quality and overall visual consistency, reporting improvements over the compared methods.
Critical strengths
The paper proposes a unified pipeline that first harmonizes and relights the inserted foreground, then constructs a coarse shadow prior from estimated geometry and illumination before diffusion refinement. The authors aim to use geometric projection to anchor shadow direction, contact relationships, and global layout, while assigning soft boundaries and local appearance to diffusion refinement, with the goal of improving consistency between foreground lighting and cast shadows. The controlled comparison in Table 3 supports the usefulness of the geometry-based prior, showing improvements in local reconstruction and structural similarity over the tested alternatives.
Critical weaknesses
-
Insufficient justification and validation of the foreground adaptation stage.
Relationship to prior work. The first stage follows the intrinsic decomposition, albedo harmonization, and foreground relighting pipeline of Careaga et al. [1]. PAD replaces several components, but does not clearly explain what problems these changes address or demonstrate their benefits over the original pipeline.
Input to intrinsic decomposition. Careaga et al. [1] formulate the decomposition separately for the foreground source image and the background image. PAD applies it directly to the unharmonized composite. The inserted foreground still contains shading from its source scene, and decomposing the composite may incorrectly encode some of this shading in the foreground albedo. Since subsequent relighting replaces the foreground shading while retaining the harmonized albedo, such errors would carry into the relit result. The paper does not explain how this issue is addressed.
Normal estimation. PAD uses MoGe-2 for foreground geometry and Omnidata for background normals. However, the paper does not explain why different models are used for the two regions or specify how their outputs are aligned and combined into the normal map used for relighting.
Compatibility between albedo, normals, and illumination. Careaga et al. [1] fit lighting parameters to the estimated background shading and normals, explicitly connecting these estimates. PAD instead combines independently estimated albedo, geometry, and EMLight illumination. The paper does not explain how the normal and lighting coordinate conventions are aligned, or how the brightness scales and color representations of the albedo and rendered shading are matched before multiplication.
Validation of the intermediate result. Figure 3 provides a qualitative component ablation, but does not establish whether these modifications improve foreground relighting. A direct comparison of the stage-one output against Careaga et al. [1] is missing. Since subsequent geometry estimation and shadow generation use this intermediate image, its quality is consequential to the reliability of the full pipeline.
-
The benefit of diffusion refinement is not established. Given estimated geometry and lighting, conventional shadow rendering is a natural alternative for fixed-view compositing. The paper provides no quantitative ablation removing diffusion refinement, and Figure 3 does not demonstrate a clear improvement over the corresponding variant. Table 3 keeps the diffusion model fixed and therefore cannot establish its contribution. This leaves the benefit of a central component of the proposed coarse-to-fine framework insufficiently supported.
[1] Chris Careaga, S. Mahdi H. Miangoleh, and Yağız Aksoy. Intrinsic Harmonization for Illumination-Aware Compositing. SIGGRAPH Asia, 2023.
Questions for authors
-
Clarification of normal estimation: How are foreground normals obtained from MoGe-2, and how are they aligned and combined with the Omnidata estimates? Why are both models needed?
-
Evidence for the changes to intrinsic decomposition: Why is intrinsic decomposition applied directly to the unharmonized composite? Careaga et al. [1] formulate this step separately for the foreground source image and the background image. How do the authors verify that the lighting mismatch in the composite does not bias the estimated foreground albedo? Can the authors provide a direct comparison of the stage-one relighting results against Careaga et al. [1] under matched input conditions?
-
Evidence for diffusion refinement: What quantitative improvement does diffusion refinement provide over both the variant without refinement and conventional shadow rendering using the same estimated geometry and lighting? Can the authors provide clearer visual comparisons demonstrating the improvement that is difficult to discern in Figure 3?