To Inpaint, or Not to Inpaint: Auxiliary-Free Shadow Removal via Inpainting and Post Fusion

Shadows degrade downstream vision tasks, yet most shadow removal methods depend on auxiliary inputs such as shadow masks, surface normals, or depth maps, which are often unavailable or inaccurate at deployment. We propose Inpaint2Free (I2F), an auxiliary-free shadow removal framework that exploits the mask conditioning of a pretrained inpainting diffusion model. I2F is fine-tuned in two stages: first with ground-truth shadow masks to learn localized shadow removal, then with Full-one masks so that inference requires only the shadow image. We further propose Post Fusion, which estimates a high-recall pseudo mask from the difference between the removal result and the input, and uses it to restore non-shadow details from the original image. On ADSP, the two-stage fine-tuning substantially improves shadow restoration, while Post Fusion markedly improves non-shadow and full-image fidelity. When compared with mask-dependent baselines using FDRNet-predicted masks on ADSP, auxiliary-free I2F achieves a shadow-region PSNR of 41.178 dB, outperforming all baselines. The framework thus removes the dependence on both auxiliary inputs and upstream prediction models, providing a practical standalone solution for real-world shadow removal.