- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Depth-to-RGB (D2R) for reference-based object compositing. D2R predicts completed-scene relative depth from a background image, reference image, and compositing mask using encoder-feature supervision, then uses this depth as a fixed condition for RGB synthesis. The paper also introduces AnyInsertion++ to evaluate generalization beyond compositing training categories.
Critical strengths
- Explicit geometric conditioning. D2R predicts the completed scene’s depth before RGB synthesis and uses it as a fixed geometric condition, explicitly addressing geometry that is typically left implicit in reference-based compositing.
- Extensive evaluation. Broad comparisons and ablation studies support the method’s effectiveness and generalization beyond compositing training categories.
Critical weaknesses
- It remains unclear whether D2R retains its geometric advantages with coarse masks rather than precise silhouettes, as users often provide only rough object outlines in practice.
- The paper lacks evaluation assessing foreground–background integration. Depth consistency and identity similarity do not sufficiently capture plausible contact, occlusion, lighting, or boundary transitions, leaving the perceptual quality of scene integration insufficiently validated.
Questions for authors
- Evaluate robustness to mask precision using silhouettes, dilated masks, and bounding boxes on the same paired cases.
- Provide a user study comparing spatial plausibility and appearance harmonization against baselines.
- Evaluate fine-grained spatial relationships in multi-object scenes, including contact and occlusion, while preserving non-target objects.
- Compare with the matched RGB-only baseline on geometrically simple scenes with challenging illumination to assess the benefit of depth guidance for harmonization.
6 comments
[2/2] Response to Reviewer UEjZ
(Continued from Previous Response)
# [Q3] Fine-Grained Spatial Relationships in Multi-Object Scenes (Contact and Occlusion) and Non-Target Objects
On scenes with multiple objects, contact, and occlusion, D2R has the lowest AbsRel and the highest PSNR of all 16 methods, and its AbsRel is 46% to 67% lower than that of the matched RGB-only baseline (Tab. C10). From the 139 OOD cases, we select three subgroups, which can overlap and together cover 66 cases: Multi-Object (two or more non-target objects next to the inserted object), Contact (the inserted object touches a non-target object other than its support or wearer), and Occlusion (a non-target object partly occludes it). The group labels use the target image and the target mask only (App. C.4.2 and rows five to seven and nine of Fig. D2).
- Full-Image Results (Table R3). Against the matched RGB-only baseline (Insert-Anything*), the AbsRel of D2R is 46% lower on Multi-Object (0.149 to 0.080), 67% lower on Contact (0.224 to 0.073), and 61% lower on Occlusion (0.168 to 0.065). The baseline is worse in these subgroups than on the full split (0.119), whereas D2R stays close to its full-split error (0.067).
- Boundary Band of Contact and Occlusion. Contact and occlusion occur at the boundary of the object. There, the AbsRel of D2R is 21% lower on Multi-Object (0.257 to 0.202), 30% lower on Contact (0.201 to 0.140), and 31% lower on Occlusion (0.188 to 0.129).
- Preservation of Non-Target Objects. An inpainting backbone is conditioned on the input background and should reproduce it up to the encoding and decoding error of the VAE. On the background, which holds the non-target objects, D2R has the lowest AbsRel of all methods in all three subgroups (0.080, 0.074, and 0.065 vs. 0.158, 0.245, and 0.178 for the RGB-only baseline). The RGB-only baseline on the same backbone sometimes redraws this region (Fig. D7).
- Human Judgment. These scores measure geometric agreement and do not label an individual contact or occlusion as correct. This perceptual judgment is left to the user study, which includes such scenes (App. C.4.5).
Table R3 (i.e., Tab. C10; 8 of 16 methods). Multi-Object Scenes, Contact, and Occlusion on the Category-Disjoint Split.
| Method | Multi-Object | (52) | Contact | (27) | Occlusion | (25) |
|---|---|---|---|---|---|---|
| AbsRel ↓ | PSNR ↑ | AbsRel ↓ | PSNR ↑ | AbsRel ↓ | PSNR ↑ | |
| MimicBrush (NeurIPS'24) | 0.105 | 19.2 | 0.110 | 19.9 | 0.089 | 21.4 |
| Insert-Anything* (AAAI'26) | 0.149 | 20.1 | 0.224 | 19.4 | 0.168 | 20.7 |
| SHINE (ICLR'26) | 0.212 | 16.6 | 0.235 | 17.1 | 0.226 | 17.7 |
| HiFi-Inpaint (CVPR'26) | 0.100 | 18.8 | 0.116 | 19.1 | 0.120 | 19.3 |
| DIRECT (ICML'26) | 0.157 | 18.7 | 0.137 | 19.1 | 0.113 | 19.9 |
| GPT-Image-2 (Tested 2026-08) | 0.377 | 12.7 | 0.377 | 12.8 | 0.316 | 15.4 |
| Nano Banana 2 (Tested 2026-08) | 0.295 | 14.5 | 0.273 | 14.4 | 0.235 | 16.8 |
| D2R (Predicted Depth) | 0.080 | 22.8 | 0.073 | 23.9 | 0.065 | 24.9 |
# [Q4] Matched RGB-Only Baseline on Geometrically Simple Scenes with Challenging Illumination
Fig. D7 adds the requested matched comparison: in a low-light scene, "RGB Only" redraws the surroundings and keeps a bright exposure, whereas D2R preserves them and follows the local light. In a low-light room lit through one window, we fix the background and the bright reference and place the same bounding box on a dark surface (table) and on a bright surface (floor) (see App. D).
- Observation (Fig. D7). "RGB Only" redraws the surroundings of the object (background hallucination): the table and the magazine on it change, and the object grows beyond the box. It also keeps a bright exposure on the dark table, which leaves a visible seam. With the composite depth, D2R keeps the object inside the box, preserves the non-target objects, and takes the exposure and color tone of each location.
- Scope and Future Work. This example covers two placements in one scene, and depth guidance does not replace harmonization (App. C.5). Existing harmonization benchmarks (e.g., iHarmony4) provide an object that is already placed with plausible geometry, so they evaluate appearance only. Once a benchmark that covers both geometry and harmonization is available, we expect it to allow an accurate evaluation of illumination-aware compositing. We leave extending the depth condition with illumination to future work.
We hope these additional evaluations address your concerns. We will post the user study results in this thread once they are complete, and we welcome any further questions or requests in the meantime.
Follow-up to Reviewer UEjZ: User Study Results [Q2] and Revision Summary
We would like to kindly follow up on our response. The user study that you proposed [Q2] is now complete, and we share its results below. We acknowledge that depth and identity scores cannot capture whether people perceive an inserted object as part of the scene, and that perception determines whether a compositing method is useful in practice. After the results, we list where each point of your review is addressed in the revised paper.
# [Q2/W2] User Study on Spatial Plausibility and Appearance Harmonization
With 51 participants and 30 cases, D2R receives the highest selection rate on both criteria: 91.4% for spatial plausibility and 87.5% for appearance harmonization, compared with 80.5% and 79.2% for the previous state-of-the-art, matched RGB-only baseline Insert-Anything* (Table R4 and Fig. C7).
- [Protocol] (App. C.4.5 and Fig. C6). Each case shows the results of seven methods with anonymized names in a randomized order for each participant. Since multiple outputs can be valid composites for a given scene, we use a multi-select setup: a participant marks every result that is spatially plausible and, separately, every result whose appearance blends with the scene. The selection rate of a method is the proportion of responses in which it is selected (1,530 answers per criterion).
- [Cases] (Tab. C14). Since 30 cases already take a participant about 20 minutes, we evaluate a diverse subset instead of the full benchmarks, which keeps the study short and helps recruit more participants: 12 OOD and 9 ID cases from AnyInsertion++ (18 object categories) and 9 cases of three unpaired benchmarks. They include scenes with multiple objects, contact, and occlusion, and no case is repeated.
- [Spatial Plausibility]. D2R (91.4%) and Insert-Anything* (80.5%) are marked far more often than other methods (9.3% to 26.0%).
- [Appearance Harmonization]. D2R is marked in 87.5% of all responses, followed by Insert-Anything* (79.2%) and the two proprietary models (58.6% and 51.8%).
Table R4 (i.e., Fig. C7). Selection Rate (%) of Each Method in the Multi-Select User Study (51 Participants, 30 Cases), Sorted by Appearance Harmonization.
| Method | Spatial Plausibility ↑ | Appearance Harmonization ↑ |
|---|---|---|
| D2R (Predicted Depth) | 91.4 | 87.5 |
| Insert-Anything* (AAAI'26) | 80.5 | 79.2 |
| Nano Banana 2 (Tested 2026-08) | 17.5 | 58.6 |
| GPT-Image-2 (Tested 2026-08) | 9.3 | 51.8 |
| DIRECT (ICML'26) | 26.0 | 30.1 |
| HiFi-Inpaint (CVPR'26) | 26.0 | 15.4 |
| SHINE (ICLR'26) | 15.1 | 4.3 |
Also, we add the plot of the selection rates (Fig. C7) and the list of the 30 evaluated cases (Tab. C14), and we replace the placeholder sentence for the user study with the number of participants and the selection rates.
With this study, every evaluation requested in your review is in the revised manuscript: mask precision ([W1/Q1]; Fig. C4), boundary transitions ([W2]; Tab. C11), contact and occlusion ([Q3]; Tab. C10), challenging illumination ([Q4]; Fig. D7), and the user study ([Q2]; Fig. C7). Your review raised an important question of how a composite should be evaluated beyond quantitative depth and identity scores, and these results are our answer. We would be grateful for your view on whether they address your concerns, and on what a paper in this area should report so that later work can be compared fairly.
One study does not settle how scene integration should be evaluated. We hope that the two criteria you proposed, together with our curated dataset (i.e., AnyInsertion++) and the codebase that we will release, give future work a great starting point. If you see a comparison or a breakdown that would make this evaluation more useful to the research community, please let us know. We will do our best to add it within the discussion period.
Thank you again for the suggestion and for your time.
[1/2] Response to Reviewer UEjZ
We deeply appreciate Reviewer UEjZ's thoughtful review and the recognition of the explicit geometric conditioning and the extensive evaluation that supports the generalization beyond compositing training categories. In what follows, we address each of your comments, grouping related weaknesses and questions together. All changes are marked in blue in the revised manuscript.
# [W1/Q1] Robustness to Mask Precision: Silhouettes, Dilated Masks, and Boxes
With coarse masks, D2R has the highest identity scores of all 16 methods on three unpaired benchmarks, which provide only bounding boxes (Tab. 4). On the same paired cases, D2R also retains its geometric advantage: with the same bounding box, its OOD AbsRel (↓) is 52% lower than that of the matched RGB-only baseline (0.115 vs. 0.237; Fig. C4 (a)). For the paired evaluation, we change only the input mask on the 139 OOD cases of AnyInsertion++ and keep the background, the reference, and the paired target fixed (new App. C.3.5).
Table R1 (values of Fig. C4). Mask Sensitivity on the OOD Split of AnyInsertion++ (AbsRel ↓).
# [W2/Q2] Evaluation of Foreground–Background Integration and User Study
At the boundary transition between the object and the background, D2R has the lowest depth error of all 16 methods on both splits (Tab. C11). Contact and occlusion are evaluated under [Q3], lighting under [Q4], and the perceptual quality of scene integration in the user study below.
Table R2 (i.e., Tab. C11; AbsRel ↓ of 8 of 16 methods). Depth Error on the Full Image and Inside the Boundary Band on AnyInsertion++. ID = In-Distribution (287 cases), OOD = Category-Disjoint (139 cases).
(Continued in Next Response)