- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces JointEdit3D, a 3D editing method which jointly generates edited appearence and geometry rather than editing views first and reconstructing afterward. The input of this method is a source video and a single edited reference frame. Then, it uses an asymmetric RGB-geometry latent inpainting process to obtain edited appearance and geometry of the scene. This paper also introduces SceneEdit3D-15K, a dataset of 15K paired scene edits with renderer-provided 3D annotations, and a benchmark SceneEdit3D-Bench for evaluation. Results show that the proposed method achieves better results than exisiting methods, in terms of PSNR and LPIPS.
Critical strengths
-
The idea of editing the appearence and geometry in one single feed-forward pass is much better then previous optimization-based methods.
-
This paper features a video for introduction, the results are clearly demonstrated, good results.
-
The contribution of SceneEdit3D-15K dataset could be beneficial to the community.
Critical weaknesses
-
All of the editing shown in the paper focuses on object deletion, add or move, which makes the paper more similar to a 3D inpainting paper rather than an editing paper. Results involve color-changing, like day to night, spring to fall, or changing from one object to another, would be interested to see in the paper.
-
The results only provides PSNR and LPIPS. More metrics, including CLIP metrics to see if the editing is aligned with text prompt, FID, time, etc, are expected. Meanwhile, qualitative results are only on the self-constructed SceneEdit3D-Bench, with some inpanting benchmark in the supplementary.
-
Given that many of test cases in this paper are inpainting, the following paper would be interesting to compared with:
You, Junqi, et al. "Instainpaint: Instant 3d-scene inpainting with masked large reconstruction model." Advances in Neural Information Processing Systems 38 (2026): 153574-153594.
4.It's not critical, but the reference annotation in this paper is black, the same color as text, which makes it difficult to read.
Questions for authors
-
I would be interested to know the (1) time (2) results on large appearance related editing for this work.
-
The pipeline includes the editing with one reference image and a source unedited video. What if we use a video editing model to edit the entire video?
-
Since the input of the model is source video, it doesn't perfectly aligned with the idea of "3D editing". What if the given images are not in a time order?
5 comments
Response to Reviewer qXVa (2/2): InstaInpaint, Video Editing Models, and Input Format
W3. Comparison with InstaInpaint.
We evaluated InstaInpaint on SPIn-NeRF, giving all methods the same real object-free reference (Table R1, in our response to Reviewer rarN). JointEdit3D is better on all metrics except SSIM (e.g., M-PSNR 23.12 vs. 21.73, M-LPIPS 0.266 vs. 0.285), while requiring neither masks nor camera poses. InstaInpaint is designed mainly for object removal. On the more complex addition, relocation, appearance, and multi-operation edits of SceneEdit3D-Bench, it fails on a large number of edits, with Edit PSNR of 13.27 to 16.91 compared with 23.19 to 27.67 for JointEdit3D (Table R7), and it does not place new content in other views (Figure R4). Changing or adding InstaInpaint's input views does not close the gap (Edit PSNR at most 25.87 vs. 31.92, Table R6).
W4. Citation color.
Thank you for pointing this out. The default hyperref link boxes may not be displayed in some PDF viewers, so we have switched to colored citation text in the revision.
Q2. Editing the entire video with a video editing model.
Editing the video first and reconstructing it afterwards is the cascaded paradigm that JointEdit3D is designed to avoid. We added two first-frame-guided video editors, I2VEdit and AnyV2V, followed by VGGT reconstruction (Table R3, in our response to Reviewer rarN). Errors from the video editing stage carry into the geometry and raise CD to 0.1254 or higher (vs. 0.0976). Since these editors have no training mechanism for preserving unedited content, the overall color and style of the edited videos also drift from the source as the video progresses, and local errors such as a ghost of the removed object appear (Figure R6).
Even without its 3D output, JointEdit3D is a strong reference-guided video editor. Its RGB outputs in Table R3 are directly comparable with those of the video editors, and JointEdit3D achieves higher edit and background quality than the better of the two (Edit PSNR 26.54 vs. 22.99, Bg. PSNR 32.73 vs. 23.99), without the per-clip LoRA training that I2VEdit requires. The Table 5 variant "w/o GEO in Wan Flow (+VGGT)" is an RGB-only video editor trained on the same data, and it is worse in appearance as well as geometry (Edit PSNR 24.75 vs. 26.82, Bg. PSNR 27.85 vs. 30.69, CD 0.1506 vs. 0.1053), suggesting that joint geometry generation also benefits appearance editing. JointEdit3D is also more flexible in how the edit is given. First-frame-guided video editors such as I2VEdit and AnyV2V take the edit only at the first frame, so a reference elsewhere requires reordering the video (Figure R6), whereas JointEdit3D accepts the edited reference at any frame of the clip (Appendix B.1) and multiple edited references at inference (Figure 11), without fine-tuning.
Q3. Video input and unordered images.
The "video" here denotes ordered multi-view observations of a static scene, and the output is 3D geometry. JointEdit3D is designed for such input, which lets it build on a video generation prior and operate without camera poses. Our current scope is views organized along a camera trajectory. Fully unordered image collections would need to be ordered first, and we have not evaluated such reordering or very sparse, wide-baseline inputs.
We hope these clarifications address the reviewer's concerns. If so, we would be grateful if the reviewer would consider updating the score, and we are happy to discuss any remaining questions.
Follow-up on our response
Dear Reviewer,
We would like to follow up on our response above. We clarified the distinction between object-level editing and global stylization, added zero-shot stylization results and further editing examples, and provided CLIP/FID evaluations and comparisons with InstaInpaint and video-edit-then-reconstruct baselines (Appendix F). We also highlighted the geometry, depth, runtime, and real-scene evaluations already included in the original manuscript.
We would appreciate your feedback on whether these address your concerns and would be happy to clarify any remaining issues. Thank you for your time and consideration.
Best regards,
The authors
Response to Reviewer qXVa (1/2): Editing Scope, Metrics, and Runtime
We thank the reviewer for the constructive feedback and for recognizing our joint appearance-geometry editing and SceneEdit3D-15K. All new results have been added to Appendix F of the revised PDF.
W1 & Q1. Editing scope and large appearance changes.
We respectfully clarify that JointEdit3D targets object-level 3D scene editing rather than 3D inpainting, which typically refers to filling the region of a removed object, as in SPIn-NeRF and AuraFusion360. Beyond removal, JointEdit3D handles addition, relocation, appearance change, and multi-operation edits, which require generating new content with consistent geometry and occlusion. This scope follows the closest feed-forward 3D editing methods, MVInpainter and Omni-3DEdit, which target the same family of object-level operations. Accordingly, appearance editing in our paper refers to object-level changes such as recoloring or retexturing a specific object (Table 3, Figure 9).
Following the reviewer's suggestion, we added object replacement and more object-level appearance edits (Figures R1 and R2). JointEdit3D keeps the new content consistent across views while preserving the surrounding scene.
Global changes such as day-to-night or seasonal transitions belong to scene stylization, a separate line of work represented by Instruct-NeRF2NeRF and EditCast3D. Our training data contains no such edits, yet zero-shot tests already transfer these changes across views, although coverage and lighting are not yet fully consistent (Figure R2(b)). Our Blender pipeline can render paired stylization data, and adding it to training is a natural direction for future work.
W2 & Q1. Metrics, runtime, and real data.
Several of these results are already in the paper. They include SSIM and depth metrics (Table 7), point-cloud and region-wise Chamfer metrics (Table 4), and runtime (Table 10). JointEdit3D takes 11.94 s per scene, compared with 43 s for MVInpainter, 237 s for Omni-3DEdit, and 285 to 1619 s for optimization-based methods. Every edit, including global changes such as the stylization in Figure R2(b), runs through the same feed-forward pass with a fixed number of denoising steps, so the runtime is essentially the same regardless of the edit type or how much of the scene changes.
Following Omni-3DEdit, we further report CLIP metrics and FID (Table R5). JointEdit3D performs best on all four metrics, with a much lower FID (14.27 vs. at least 29.45).
Table R5. CLIP metrics and FID on SceneEdit3D-Bench.
For real data, the paper already reports quantitative results on 360-USID (Table 2) and Inpaint360GS (Table 12), and we now add SPIn-NeRF (Table R1). Qualitative real-scene results are also part of the paper, on Instruct-NeRF2NeRF and DL3DV captures (Figure 4), real videos (Figure 5), and DAVIS, ScanNet++, and casual captures (Figures 9–12). We have added further real-scene examples in Figures R1, R2, R4, R5, and R6.