PhysLayer: Learning Object–Effect Separation through Physics-Grounded Layer Decomposition

Recent diffusion-based approaches have advanced image layer decomposition, enabling flexible extraction and editing of individual image components. However, complex natural scenes remain challenging when a target is partially occluded, covers previously invisible content, or induces effects such as shadows and reflections. Existing representations often entangle these effects with the foreground or background, while natural 2D imagery rarely provides aligned supervision for the complete target, the target-free scene, and their interactions. We propose PhysLayer, a target-conditioned framework that decomposes a composite image, given a target bounding box, into three complementary components: an amodal foreground RGBA layer, a clean target-removed background, and a signed residual capturing target-induced scene interactions. To supervise this formulation, we construct PhysLayer-30K through controllable 3D scene manipulation and counterfactual rendering, providing matched foreground, background, visibility, and interaction annotations from the same scene. A joint diffusion transformer predicts the three layers within a shared denoising process using layer-aware spatial encoding, targeted foreground and background supervision, and residual attention alignment. We evaluate PhysLayer on foreground matting, object removal, and layer decomposition. Compared with the box-conditioned RevealLayer baseline, PhysLayer improves object-removal PSNR by 2.51 dB on OBER-Test and reduces background LPIPS by 47.4% on PhysLayer-Test. It also achieves alpha SoftIoU scores of 0.9333 on RefMatte-RW100 and 0.945 on PhysLayer-Test, demonstrating improved foreground recovery and target-free scene reconstruction.