Physical-Anchor Diffusion: A Coarse-to-Fine Framework for Shadow-Aware Image Harmonization

Realistic image compositing requires jointly resolving foreground appearance, illumination, and cast-shadow consistency. These factors are mutually constrained: appearance adjustment alone cannot correct lighting mismatch, while unconstrained shadow generation may improve local realism but violate scene geometry. We therefore propose Physical-Anchor Diffusion (PAD), a unified coarse-to-fine framework for shadow-aware image compositing. PAD first harmonizes the foreground in intrinsic albedo space and estimates background illumination using an environment-aware spherical-harmonic representation to relight the foreground consistently with the scene. It then combines the reconstructed scene geometry with the estimated illumination direction to construct a physically grounded coarse shadow prior that encodes shadow direction, contact region, and global layout. Conditioned on this prior, a diffusion model refines the shadow by recovering soft boundaries and realistic local appearance. By constraining global shadow structure with geometry while delegating fine-scale appearance synthesis to diffusion, PAD balances physical plausibility and visual realism. Experiments on the DESOBAv2 benchmark and real-world compositing cases demonstrate that PAD achieves stronger shadow structural fidelity while improving the perceived coherence of foreground appearance, illumination, and cast shadows in the final composites.