PhysInsert: Physically Plausible Object Insertion with Geometric Priors and Bidirectional Scene Interactions

Reference-based object insertion requires preserving an object's identity while adapting its appearance and geometry to a target scene. Differences in viewpoint and illumination make this task challenging, while physical plausibility additionally requires bidirectional interactions: the scene affects the object's lighting and visibility, and the inserted object alters the scene through shadows, reflections, and occlusions. These coupled requirements call for supervision and evaluation that jointly capture object adaptation and its effects beyond the object region. We propose a framework for physically plausible object insertion that combines controlled rendering data, geometric priors, and spatially guided generation. We construct a dataset by rendering complete scenes with and without selected objects while keeping other scene configurations fixed, capturing the visual consequences of object insertion. Each object is additionally rendered from multiple viewpoints under varied illumination, providing reference images that differ from the target in both viewing and lighting conditions. Building on this dataset, we incorporate features from a pretrained geometric foundation model into the editing backbone and introduce soft spatial attention guidance to support placement control and object–scene interaction modeling. We further develop a benchmark that evaluates geometric and optical plausibility alongside identity preservation, fine-grained instruction following, and preservation of unaffected scene content. The benchmark combines controlled synthetic cases with in-the-wild tests supported by multiview imagery. Together, these components establish a framework for learning and evaluating object insertion as a coordinated transformation of the object and its surrounding scene.