acceptodds
Under review as a conference paper at ICLR 2027

PhysInsert: Physically Plausible Object Insertion with Geometric Priors and Bidirectional Scene Interactions

Abstract

Reference-based object insertion requires preserving an object's identity while adapting its appearance and geometry to a target scene. Differences in viewpoint and illumination make this task challenging, while physical plausibility additionally requires bidirectional interactions: the scene affects the object's lighting and visibility, and the inserted object alters the scene through shadows, reflections, and occlusions. These coupled requirements call for supervision and evaluation that jointly capture object adaptation and its effects beyond the object region. We propose a framework for physically plausible object insertion that combines controlled rendering data, geometric priors, and spatially guided generation. We construct a dataset by rendering complete scenes with and without selected objects while keeping other scene configurations fixed, capturing the visual consequences of object insertion. Each object is additionally rendered from multiple viewpoints under varied illumination, providing reference images that differ from the target in both viewing and lighting conditions. Building on this dataset, we incorporate features from a pretrained geometric foundation model into the editing backbone and introduce soft spatial attention guidance to support placement control and object–scene interaction modeling. We further develop a benchmark that evaluates geometric and optical plausibility alongside identity preservation, fine-grained instruction following, and preservation of unaffected scene content. The benchmark combines controlled synthetic cases with in-the-wild tests supported by multiview imagery. Together, these components establish a framework for learning and evaluating object insertion as a coordinated transformation of the object and its surrounding scene.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Object Insertion, Image Editing, Physical Plausibility

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CT2cOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PhysInsert, a framework for physically plausible reference-guided object insertion that focuses on bidirectional object–scene interactions and cross-view adaptation. The authors construct InsertWorld, a synthetic dataset based on BlenderKit 3D assets, which provides strictly aligned images with and without the target object, along with reference images under multiple viewpoints and lighting conditions. Building on Qwen-Image-Edit-2511, the method introduces Geometry-Guided Adaptation (GGA), Counterfactual Object–Scene Supervision (COSS), and Soft-Mask Spatial Grounding (SSG). The paper evaluates object insertion quality, geometric adaptation, and physical plausibility through the synthetic PhysInsert-Bench benchmark, real-image test sets, and a human evaluation.

Critical strengths
  • The paper addresses lighting, shadows, reflections, and viewpoint adaptation in object insertion, making the task practically relevant.
  • The dataset provides strictly aligned images with and without the target object, as well as reference images captured from multiple viewpoints and under different lighting conditions.
  • The experiments show that the proposed method outperforms existing models.
Critical weaknesses

InsertWorld is entirely synthetic, and all of its assets come from BlenderKit. Compared with real photographs, these rendered images lack realism and cover a relatively limited range of object categories and scenes. Reliance on a single asset source further limits the diversity of the dataset, making it difficult to adequately capture the complexity of real-world objects, materials, lighting, and interactions.

The ablation study shows that most of the performance gains come from fine-tuning on InsertWorld. Compared with PhysW., the additional improvements introduced by GGA, COSS, and SSG are not substantial, yet they incur considerable computational overhead. The full method increases inference time by approximately 51% relative to the original model.

Phy-VLM is an important metric for evaluating physical plausibility in this paper, but it is not reported in Tables 3, 4, or 5, leaving the evaluation of key experiments incomplete.

Questions for authors
  1. What object categories are included in InsertWorld, and how many samples does each category contain?
  2. Most of the presented examples appear close to the scene distribution of BlenderKit. Can the method generalize to inserting subjects from categories such as people and animals?
3 comments
AuthorsOpenReview ↗

Response to Reviewer CT2c

We sincerely thank the reviewer for recognizing the practical relevance of our task, the value of our dataset, and our experimental gains. We address concerns regarding dataset diversity and generalization, the benefits and computational costs of our method, and evaluation completeness.

W1. Synthetic data and diversity

We appreciate the reviewer’s concern regarding the realism and diversity of synthetic data. While synthetic rendering cannot fully capture real-world complexity, InsertWorld covers 18 object-category subgroups and diverse material properties (Appendix C, Figs. S2 and S4).

More importantly, synthetic rendering enables systematic variation of lighting and viewpoint for the same object, providing reference diversity that is difficult to obtain through real-world collection (Fig. S3).

DatasetObjectsCounterfactual pairsDistinct environments across object referencesMultiview referencesSystematic light × view coverage
ORIDa [1]200Yes≈179*Not systematically providedNo systematic joint coverage
NAVI [2]36No≈50*≈25 views/captureNo systematic joint coverage
InsertWorld1,710Yes97723 views/setupYes, 7 × 23 = 161 refs/object
  • The approximate scene count was estimated by clustering DINOv2 image features and cross-checking the resulting scene groups with GPT-6.1-Sol.

Furthermore, PhysInsert also generalizes to real-world images despite being trained on synthetic data. It achieves the best PSNR and FID across the evaluated settings on 50 PhysInsert-Bench-Wild and 40 DreamBooth cases (Table 4), and receives the highest overall rating in our real-image user study (Fig. 4; Appendix R).

We acknowledge that relying on BlenderKit as our sole asset source may limit coverage of some real-world object and scene types. Incorporating additional asset sources could broaden this coverage.

References

[1] Kim et al. ORIDa: Object-centric Real-world Image Composition Dataset. CVPR 2025.

[2] Jampani et al. NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations. NeurIPS 2023.

W2-a. Improvements beyond dataset fine-tuning

We thank the reviewer for highlighting the substantial gains from fine-tuning on InsertWorld. These gains highlight the importance of controlled training data for physically plausible insertion, and InsertWorld itself is a key contribution. At the same time, dataset-only fine-tuning (PhysW.) can still produce incorrect object scale, perspective, and scene framing, as illustrated in the new Fig. S8 (Appendix H). The original Table 5 already shows additional gains from our full method: PSNR improves from 22.53 to 24.40 dB, and FID decreases from 48.61 to 41.96; newly added Phy-VLM scores also improve from 3.48 to 3.75. Together with the original qualitative ablations in Fig. 6(b), these results show the complementary contributions of the dataset and method.

W2-b. Computational overhead

We appreciate the reviewer’s concern about inference efficiency. COSS and SSG are used only during training and introduce no additional inference-time cost. The inference overhead comes from GGA, which requires online VGGT feature extraction and an additional geometry stream. We report the detailed efficiency measurements in Appendix N (Table S6) and acknowledge this limitation in Appendix A of the revised manuscript.

W3. Completeness of Phy-VLM evaluation

Thank you for noting the missing Phy-VLM scores. Our initial evaluation used GPT-5.5, whose API cost constrained its scale. The more cost-efficient GPT-6.1-Sol enabled broader evaluation. We therefore upgraded the evaluator while retaining the original scoring prompt and averaging three ratings per output. We re-evaluated Table 2 and completed the previously missing Phy-VLM results in Tables 3–5. The new entries are marked in blue and labeled Phy-VLM⋆; Table 2 retains the earlier GPT-5.5 scores for reference. Appendix B, Fig. S1 further analyzes agreement with human ratings.

Q1. Object categories and sample counts

Appendix C now organizes 1,710 insertion targets into 18 category–rigidity subgroups (Fig. S2), covering rigid/non-rigid objects, metallic/non-metallic materials, opaque/translucent/transparent surfaces, and emissive/non-emissive properties. We also report category-wise counts for 12,951 training scene pairs (Fig. S4) and the diversity of 977 reference HDRIs (Fig. S3).

Q2. Transfer to people and animals

We appreciate this question. We added qualitative examples of people and animals inserted into varied environments using the existing checkpoint, without category-specific fine-tuning (Appendix D, Fig. S5). Our DreamBooth qualitative results also include animal examples (Appendix G, FigS7; FigS10, FigS11). Together, these examples illustrate potential cross-category generalization.

I thank the authors for their response, and my concerns have been addressed.

Thank you for your thoughtful review and constructive feedback. We appreciate your time and are glad that our response has addressed your concerns.

Reviewer_nm5XOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PhysInsert, a novel framework for physically plausible image object insertion, considering the bidirectional interaction between objects and the environments, including shadows, reflections, and geometric structure. This paper does not explicitly model the above factors. Instead, this paper sovles this problem via a data-driven approach, involving three contributions claimed by the authors:

  1. A new dataset, InsertWorld, constructed from Blender via a complete pipeline. The dataset covers strictly aligned counterfactual pairs, multiview and multilight references.
  2. A novel object insertion framework, PhysInsert, which combines a multi-model Diffusion Transformer (Qwen-Image-Edit) and a geometry stream (VGGT) to improve object-background-environment interaction.
  3. A corresponding benchmark, PhysInsert-Bench, which evaluates illumination and pose adaptation of object insertion through cross-environment, cross-veiw scenarios.
Critical strengths
  1. Good motivation. Different from previous object insertion methods, this paper considers the cross-environment consistency, bidirectional object–scene interactions, and cross-view consistency. This aligns with the intuition of the object insertion task, which is more challanging but also practically significant. The dataset construction is also reasonable.
  2. Comprehensive method design and pleasable results. The proposed method integrates geometric priors, spatial supervision of objects and scenes, and textual spatial constraints into a unified generative framework. The experiments covers different challenging scnarios with qualitative and quantitative ablations, demonstrating the effectivenss of the proposed mehod.
  3. The presentation of this paper is good. The illustrations are clear and the paper is well-written.
Critical weaknesses
  1. This paper relys on the construction of training data and the design of loss functions to encourage the model to learn physical plausibility during the fine-tuning process. There is no explicit phsical laws modeled. Therefore, while the data collection, model construction, and loss function design are reasonable, it seems difficult to derive new insights from this paper.
  2. Reliability of quantitative evaluation on physical plausibility. Most of the evaluation metrics are 2D-image-level. The Phy-VLM socre, which asks a VLM o assign scores to generated images and also provide reasoning for the scores, is the only specific metric for physical plausibility evaluation. Since the proposed dataset is constructed using Blender, I wonder whether more physics-oriented evaluations could be introduced. At least, such metric should be included in the tables in the main paper, but for now, the Phy-VLM is only presented in the appendix.
Questions for authors
  1. The proposed GGC, COSS, and SSG apply supervision at specific and different MMDiT layers, i.e., COSS and SSG at blocks 35, 47, and 56; GGA at blocks 14, 29, 44, 59. How does these layers are selected?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nm5X

We sincerely thank the reviewer for the constructive feedback and recognition of the paper's motivation and framework. We clarify the scope of our physical modeling, provide further evaluation evidence, and explain the supervision-layer choices.

W1. Physical scope and methodological contribution

We appreciate the reviewer’s concern about the absence of explicit physical laws. PhysInsert aims to learn physical plausibility rather than enforce hard physical constraints. Our key methodological insight is that object insertion benefits from jointly supervising post-insertion geometry and the effects induced in the surrounding scene, rather than only the inserted object's appearance. We combine counterfactual data, geometric priors, and spatial supervision to improve cross-lighting, cross-view, and bidirectional adaptation. Beyond dataset-only fine-tuning, Table 5 shows improvements in PSNR (22.53→24.40 dB), FID (48.61→41.96), and Phy-VLM (3.48→3.75). We acknowledge this limitation in Appendix A of the revised manuscript and view incorporating explicit physical constraints as a promising direction for future work.

W2a. Agreement between Phy-VLM and human ratings

We thank the reviewer for raising the important question of evaluation reliability. We compared Phy-VLM scores with human physical ratings on 100 outputs from the real-image user study (25 cases × 4 methods, rated by 27 participants). The output-level Spearman correlation is ρ = 0.772 (95% scene-bootstrap CI: [0.679, 0.840]), with 80.1% tie-adjusted within-case preference agreement across 148 eligible pairs. The method-level mean-score rankings also agree exactly (Spearman ρ = 1.00: Ours > Nano Banana Pro > Insert-Anything > DIRECT). While Phy-VLM tends to score higher in absolute terms (mean signed difference +0.89), these results support its use as a complementary metric for relative physical-quality evaluation (Appendix B, Fig. S1 of the revised manuscript).

W2b. Evaluation against Blender ground-truth depth

We appreciate the suggestion to add a more geometry-oriented evaluation. We have added a depth comparison against the original Blender scenes using renderer-exported ground-truth depth (Appendix E, Table S1 of the revised manuscript). DA3 predicts depth from generated images, with scale calibrated on the unchanged background while excluding the inserted-object region. PhysInsert achieves the best object-region AbsRel and δ₁.₂₅ among generation methods under both text and box guidance, providing complementary evidence of object–scene depth consistency.

W2c. Phy-VLM evaluation in the main tables

We appreciate the request for more complete physical evaluation. The original submission already reported Phy-VLM in Table 2. In the revised manuscript, we re-evaluated Table 2 using GPT-6.1-Sol and completed the previously missing results for Tables 3–5, retaining the original scoring prompt and three-rating averaging procedure. The updated scores are labeled Phy-VLM⋆ and highlighted in blue, while Table 2 also retains its earlier GPT-5.5 results for reference.

Q1. Selection of supervision layers

Thank you for raising the question of layer selection. GGA reads out geometry states at four depths in the 60-block DiT (blocks 14, 29, 44, and 59, using zero-based indexing), spanning intermediate through final representations. For COSS and SSG, we favor later blocks to capture object occupancy, scene effects, and spatial-text grounding, including fine visual details. This choice was motivated in part by previous work on diffusion representations and frequency components [1, 2]. During early method development, we conducted a preliminary study on 250 cases comparing shared later supervision with a split configuration moving object and spatial-text supervision to earlier blocks:

ConfigurationSSG / COSS-object blocksCOSS-effect blocksBackground PSNR ↑Background LPIPS ↓Full-image PSNR ↑
Shared later35, 47, 5635, 47, 5625.240.070119.73
Split earlier12, 24, 3636, 47, 5624.210.084319.28

Shared later supervision gave better background preservation and full-image PSNR in this preliminary study, informing our selected COSS/SSG layers (35, 47, 56).

References

[1] Sihyun Yu et al. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. ICLR, 2025.

[2] Jiahui Sun et al. W-Edit: A Wavelet-Based Frequency-Aware Framework for Text-Driven Image Editing. ICLR, 2026.

Dear Authors,

Thank you for your clarifications and experiments. My concerns have been addressed, and I have no further questions.

Thank you for your positive assessment and constructive feedback. We appreciate your time and are pleased that our responses have addressed your concerns.

Reviewer_H3mTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a model for plausible reference-based object insertion into a given background, guided by text instruction or bounding box placement. The authors leverage Qwen-Image-Edit-2511 as base model, and enrich it with several insertion specific components: 1) Geometry-Guided Adaptation, lifting the scene to 3D features via VGGT and distilling them with the GT ones, 2) Counterfactual Object-Scene Supervision (COSS), predicting and regularizing masks for the inserted object and for the region it affects indirectly (e.g. by shading or reflection) and 3) Soft-Mask Spatial Grounding, aligning attention masks between text and image to the target mask. The auhtors propose a paired insertion dataset created with renders from Blender, and build an insertion benchmark around it. Comparisons with some recent competing methods on the benchmark testifies for the suitability of the insertion method.

Critical strengths
  • The manuscript describes a very thorough research on object insertion into images, comprising a dataset acquisition, a methodology for solving the task and an evaluation benchmark to compare different models. The paper provides the full package and it is of great value for the community. The proposed dataset pipeline (rendering scenes with and without rendered objects) is reasonable and allows to have paired examples of both plausible placement of objects and second-orded interactions with the scene (e.g. shadows, mirroring etc.)
  • The paper is very clear at a first read and the proposed method is sound. Although the pipeline is fairly complex comprising several additions to the Qwen base model, each one of these modifications is well motivated, illustrated, and its validity is testified by appropriate user studies.
  • Besides the proposed benchmark, the paper reports a clear user study, where human observers assign a rating to insertion results. The study seems well conducted, and it uses real images rather than blender renders. Also the results assess several aspects of the insertion, such as lighting and plausubility.
  • The method, despite the fact it has been trained on synthetized renders, showcases good generalization when inserting objects in real images. For instance, Figure S5 shows examples from the user study, where the proposed method shows plausible insertions.
Critical weaknesses
  • My main concern is about the multimodal nature of object insertion, that is completely overlooked in this paper. Indeed, even when guiding insertion with bounding boxes or text prompts, the task naturally allows for multiple solutions, for instance by changing object orientation or scaling.
    • To my understanding, the DiT backbone of the PhysInsert model should allow for sampling multiple solutions given the same background, reference objects and prompts. However, the paper doesn't showcase any analysis of how variable the variability of the model's insertions. Figure 6c shows some examples of different placement options, but I understand they come from different prompts.
    • The dataset proposes pairs of images pre and post insertion, and only exposes a single possible solution for every example. This is a legit artifact of the dataset creation process. However, in the model formulation and optimization, there isn't any awareness of the fact that the presented GT is only one of the possible realizations.
    • More critically, the benchmark itself is based on the paired InsertWorld dataset, and the used metrics only reward compliance with the monotonic groundtruth. In fact, most metrics (PSNR, SSIM, LPIPS, DIoU, Mask IoU) require a reference realization and penalize any deviation from it, even if the proposed insertion is plausible. This is evident in several qualitative results figures, where it is clear that many results from competitors are plausible, yet do not comply with the single provided GT, and will therefore be penalized for it.
  • The paper proposes both the dataset and a benchmark based on it, which are surely valuable contributions. However, the fact that the benchmark is derived from the dataset might give some competitive advantage to the proposed model, that is to my understanding the only model trained on the dataset itself. A more fair comparison could be achieved by finetining some competing methods on InsertWorld.
Questions for authors
  • What does "counterfactual" mean whenever describing the InsertWorld dataset? Is it a synonym of "paired", as in describing that every example comes with images pre and after insertion?
  • I find reconstruction scores in Tab. 4 rather low. PSNRs <= 20 Db generally mean very poor fidelity. Why is that the case?
  • What does "real world" mean on line 415? That the images are not blender renders?
4 comments
AuthorsOpenReview ↗

Response to Reviewer H3mT (1/2)

We sincerely thank the reviewer for the thoughtful assessment and for recognizing the value of our dataset, method, and real-image user study. We appreciate the questions concerning multiple valid solutions and fair evaluation and respond below.

W1a. Variability under fixed conditioning

We appreciate the reviewer’s suggestion to examine sampling variability. We added controlled comparisons across four examples using the four random seeds. Within each example, the background, object reference, complete prompt, checkpoint, and inference settings are held fixed, with only the random seed varied. Fig. S6(a–d) in Appendix F of the revised manuscript shows variations in pose, scale, and appearance, illustrating that the model can generate different outputs under identical conditioning.

W1b. Single-target supervision

We agree that each paired target represents one valid realization, but this does not restrict the stochastic generator to a single inference-time output, as the fixed-conditioning examples in Fig. S6(a–d) illustrate. Our paired supervision learns from observed plausible insertions without explicitly enumerating all valid solutions. We therefore present these comparisons as evidence of sampling diversity, not exhaustive coverage of the conditional solution space.

W1c. Evaluation with multiple valid outputs

We agree that single-reference metrics may penalize plausible alternative insertions. They nevertheless follow established evaluation practice: Insert Anything [1] uses PSNR, SSIM, LPIPS, and FID; OmniPaint [2] uses DINOv2-based object identity evaluation; and PhyEdit [3] employs spatial and depth metrics. These metrics provide complementary information: FID compares image distributions, while AbsRel and δ₁.₂₅ measure object-region depth agreement with the reference target using DA3-estimated depths. We recognize that they do not fully resolve the single-reference limitation.

To complement these metrics, our original submission included a blinded human study on real images, evaluating physical plausibility without a unique recorded insertion target (Fig. 4; Appendix R). Our method received the highest mean human ratings for lighting, perspective, pose, and overall quality (overall: 4.04). In the revised manuscript, we have also completed the Phy-VLM evaluations in Tables 3–5 and re-evaluated Table 2. Fig. S1 in Appendix B examines agreement with human ratings, providing complementary evidence for relative physical-quality assessment.

References

[1] Wensong Song, Hong Jiang, Zongxing Yang, Zheqiao Cheng, Ruijie Quan, and Yi Yang. Insert Anything: Image Insertion via In-Context Editing in DiT. AAAI, 2026.

[2] Yongsheng Yu, Ziyun Zeng, Haitian Zheng, and Jiebo Luo. OmniPaint: Mastering Object-Oriented Editing via Disentangled Insertion-Removal Inpainting. ICCV, 2025.

[3] Ruihang Xu, Dewei Zhou, Xiaolong Shen, Fan Ma, and Yi Yang. PhyEdit: Towards Real-World Object Manipulation via Physically-Grounded Image Editing. ACM MM, 2026.

W2. Comparison with baselines trained on InsertWorld

We appreciate this fairness concern. The originally submitted Table 3 already compares against Insert-Anything and OmniPaint fine-tuned on InsertWorld under matched training budgets (+PhysW.). Their PSNR improves from 22.70 to 24.38 dB and from 20.60 to 22.49 dB, respectively. Our full model reaches 25.81 dB PSNR and 37.50 FID, compared with 24.38 dB and 43.80 FID for the stronger fine-tuned baseline. Notably, our base model Qwen-Image-Edit-2511 initialization has a higher (worse) FID (65.27) than the original Insert-Anything (52.23) and OmniPaint (59.95). These comparisons support both the benefit of InsertWorld for existing models and the additional gains of our framework.

AuthorsOpenReview ↗

Response to Reviewer H3mT (2/2)

Q1. Meaning of counterfactual

Thank you for pointing out the need for clarification. Here, counterfactual refers to rendering the same scene with and without the target object, while keeping the camera, lighting, and all other scene elements fixed. Unlike arbitrary image pairs, these renders differ only in the target object's presence and its induced environmental effects, such as shadows and reflections, which are recomputed through ray tracing (Fig. 2; Appendix P of the revised manuscript). Comparing the two renders provides supervision for both object occupancy and object-induced scene changes, enabling the modeling of bidirectional object–scene interactions.

Q2. Low PSNR on real-image test sets

We agree that PSNR values below 20 dB warrant explanation. PSNR measures pixelwise agreement with a single recorded target, so plausible differences in object pose, scale, and appearance may lower the score. We added six DreamBooth comparisons of GT, Ours, and Insert-Anything (Fig. S7, Appendix G of the revised manuscript); all displayed generated results score below 20 dB, although some insertions remain visually plausible. This does not imply that every discrepancy is valid, but supports interpreting PSNR alongside perceptual, identity, and physical-plausibility metrics.

Q3. Definition of real world

We appreciate the request for a more precise description. In this paper, “real-world results” referred to test images collected outside the InsertWorld synthetic rendering pipeline, including 40 DreamBooth cases and 50 PhysInsert-Bench-Wild cases (Appendix P.2). DreamBooth, NAVI, and ORIDa provide real photographs; ABO contributes product imagery. To avoid overstating this scope, we have renamed the corresponding main-paper subsection “In-the-Wild Results” (§5.2) in the revised manuscript.

Reviewer_H3mTOpenReview ↗

Response to rebuttal

Dear Authors,

Thank you for your response to my review. I am generally satisfied with the answer and I don't have any major remaining concern.

I acknowledge the superior performance of the proposed model with respect to competitors fine-tuned on InsertWorld.

I appreciate the variability of insertion showcased in Fig. S6. Especially the shadow modeling in the armchair example seems impressive. It seems that the model does showcase good variation in pose, orientation and scale, yet it seems to be somewhat biased about the location. Is this a consequence of the textual instruction? Is this something that the authors also notice at scale? And if so, what could be the reason for it?

Finally, I still consider pairwise metrics comparing to a GT insertion to be unsuitable to evaluate this task, regardless of how established they are in the community. This appears even more evident in Fig. S7, that showcases plausible insertions resulting in uncorrelated PSNR values w.r.t. GT. However, I acknowledge that a clean and conclusive evaluation is difficult in general, and the paper already showcases sufficient empirical evidence in the form of the user study or the Phy-VLM metric, that appears to be strongly correlated with human judgement.

Thank you for your encouraging feedback and thoughtful follow-up. We are pleased that our response has addressed your main concerns.

Response to the question on spatial placement: We further visualize the relationships among spatial text prompts, the corresponding attention-based image–text relevance maps, and the resulting insertions. As shown in the updated Fig. S6, final insertions tend to fall within regions of higher relevance. When the prompt does not specify a precise insertion location, the maps show higher relevance in central regions across random seeds, while the resulting insertions remain prompt-consistent and physically plausible. We hypothesize that this spatial preference may be shaped by the distribution of target-object regions in the training data. Adding more specific spatial instructions (e.g., left/right) shifts both the relevance distributions and the final insertion locations accordingly. These observations indicate that spatial requirements in the text can adjust our model’s spatial preference, consistent with SSG’s goal of aligning spatial instructions with target-object regions. Detailed examples and analysis are provided in the updated Fig. S6 and accompanying appendix text.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank all reviewers for their thoughtful feedback. In response, we have expanded the evaluation, added qualitative analyses, and clarified our dataset and method. Changes in the revised manuscript are highlighted in blue.

  1. Updated Phy-VLM evaluation and human agreement. Our initial Phy-VLM evaluation used GPT-5.5, whose API cost limited its scale. The later availability of the newer, more cost-efficient GPT-6.1-Sol made comprehensive evaluation feasible. We re-evaluated Table 2 and completed the previously missing evaluations in Tables 3–5, while retaining the original scoring prompt and three-rating averaging procedure. Fig. S1 in Appendix B of the revised manuscript analyzes agreement with human physical ratings.
  2. Dataset diversity. We added object-category counts and material-property distributions (Appendix C, Fig. S2), the hue and relative-luminance coverage of 977 reference HDRIs (Fig. S3), and category-wise counts for all 12,951 training scene pairs (Fig. S4).
  3. Generalization to people and animals. Additional qualitative examples illustrate cross-category generalization using the existing checkpoint without category-specific fine-tuning (Appendix D, Fig. S5).
  4. Depth consistency with Blender ground truth. We added depth evaluation against renderer-exported ground-truth depth from the original Blender scenes (Appendix E, Table S1).
  5. Sampling diversity. Four controlled examples compare outputs across the four random seeds, holding all other inference conditions fixed within each case (Appendix F, Fig. S6(a–d)).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.