acceptodds
Withdrawn from ICLR 2027

Geometry-Consistent Video Editing for Appearance Controllable Gaussian Scenes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image editing models provide intuitive control over visual changes, but extending this control to 3D also requires edits to remain consistent across viewpoints. We present SceneWeaver, a two-stage framework that constructs an editable Gaussian scene from a single photograph. We progressively build a point cloud from generated source-appearance views, then fix it as a geometric prior for multi-view video generation under different appearance references.We distill these videos into a shared Gaussian representation controlled by edit latents extracted from source–edited image pairs. These latents drive the desired edits relative to the original appearance while Gaussian geometry remains shared across edits. The resulting representation further supports zero-shot transfer of appearance edits not used during scene fitting, without additional optimization.We also introduce a scene-editing benchmark spanning indoor and outdoor scenes to evaluate appearance agreement and structural consistency. Extensive experiments demonstrate that our method achieves strong performance in both reconstruction and editing across multiple dimensions.

Primary area:
generative models
Keywords:
Controllable Gaussian Scenes, Video Editing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_njmoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a two-stage framework named SceneWeaver, which constructs an edited Gaussian scene from a single image. The key idea is to generate multiple appearances under a fixed geometric prior and distill them into a shared Gaussian representation. Specifically, SceneWeaver progressively constructs a scene point cloud and then reuses its fixed geometry to generate multi-view videos to expand the scene geometry and appearance. In addition, the authors leverage a Gaussian representation to model residuals between the edits and the original appearance. Furthermore, the authors introduce a scene-editing benchmark and evaluate the reconstruction and editing quality of SceneWeaver through extensive experiments.

Critical strengths
  1. This paper proposes a 3D Gaussian editing method from a single image with a strategy that progressively expands the point cloud.
  2. This paper introduces a scene-editing benchmark with indoor and outdoor scenes.
Critical weaknesses
  1. In Section 3.1, the authors leverage Depth Anything to estimate monocular depth and progressively update the point cloud. Although the authors apply depth scale alignment, the relative depth estimates may still introduce geometric inconsistencies during iterative point cloud construction. Additional ablations using different depth estimators would help assess the robustness of this process.
  2. In Section 3.1, SceneWeaver leverages the depth estimated from the reference image for initialization. In Section 3.2, SceneWeaver mainly updates SH colors and opacity without updating other Gaussian attributes. When the authors edit scenes involving geometry changes (e.g., adding balloons in Figure 4), it is unclear whether such edits can maintain plausible 3D geometry and multi-view consistency.
  3. The method evaluates novel view synthesis on the Mip-NeRF 360 and Tanks and Temples datasets, but the appearance editing experiments are primarily conducted on the authors' constructed benchmark. Additional appearance editing experiments on public datasets (e.g., Mip-NeRF 360, Tanks and Temples, and Instruct-NeRF2NeRF) would be helpful.
  4. Although the related work discusses several 3D scene editing approaches, the experimental evaluation lacks direct comparisons with representative 3D editing methods, such as Instruct-NeRF2NeRF [1], GaussianEditor [2], and GaussCtrl [3].

[1] Instruct-nerf2nerf: Editing 3d scenes with instructions. ICCV 2023. [2] Gaussianeditor: Swift and controllable 3d editing with gaussian splatting. CVPR 2024. [3] Gaussctrl: Multi-view consistent text-driven 3d gaussian splatting editing. ECCV 2024.

Questions for authors

Please refer to the Critical Weaknesses.

Reviewer_6kXwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SceneWeaver, a two-stage framework for constructing an appearance controllable Gaussian scene from a single picture. The first stage progressively expands a point cloud using generated source-appearance views, then fixes this geometry to guide multi-view video generation under different appearance references. The second stage fits a shared Gaussian representation, where image-pair-derived edit codes and local Gaussian features predict residuals to the default SH coefficients and opacity. Gaussian positions, rotations, and scales are shared across appearances. After fitting, by adding a new edited image can control the scene without additional optimization. The authors also introduce a 60-scene benchmark and evaluate reconstruction, appearance agreement, structural consistency, and video quality.

Critical strengths
  1. Meaningful task and coherent design. The paper addresses the coupled challenges of completing unobserved regions from a single photograph and maintaining a consistent structure across appearance changes. Source-geometry construction, shared depth conditioning, and conditional Gaussian fitting are logically connected.

  2. A useful, reusable editing interface. Encoding the change between a source image and its edited counterpart is well motivated. Combining this global condition with local Gaussian features provides a mechanism for spatially varying appearance changes while retaining a shared scene representation. Applying new edits without refitting the scene has practical value.

  3. Encouraging results with appropriately reported limitations. The method achieves the best held-out PSNR, SSIM, and LPIPS among the evaluated methods in Table 1, alongside improvements in appearance agreement and unseen-lighting transfer.

Critical weaknesses
  1. TGenerating multi-view images and then fitting Gaussians follows existing methods. The main changes are using shared point-cloud geometry across appearances and an image-pair edit encoder. The paper needs to explain more clearly how these changes differ from existing 3D editing methods.

  2. The final model changes SH coefficients and opacity while keeping Gaussian positions, rotations, and scales shared across edits. This limits edits that require new geometry, such as adding objects(as author mentioned). Changing opacity also changes visibility, so shared Gaussian geometry does not guarantee unchanged geometry in the rendered images.

  3. Gaussian fitting uses generated video frames and point-cloud depth as supervision. Incorrect structures in these targets may therefore remain in the fitted scene. VGGT and camera refinement help align the observations, but do not directly correct the generated images.

  4. (Minors)Typos / Minor Issues: Several spelling inconsistencies were noted: "WorldStero" (appearing in Figure 5 and Table 1 on page 7, and Table 4 on page 14) should be corrected to "WorldStereo."

  5. The seasonal test uses the Colosseum, and the lighting test uses nine unseen anchors. authors also needs to clarify how test scenes and edits are separated from encoder pretraining.

Questions for authors
  1. What is the main technical difference between the proposed edit encoder and appearance conditioning in previous 3D editing methods? Can its benefit be shown using the same generated views and Gaussian initialization?

  2. How are the object-addition examples produced with shared Gaussian geometry? How do opacity changes affect rendered depth and visibility?

  3. How does Gaussian fitting handle incorrect structures in generated videos or errors in the point-cloud depth?

  4. How are test scenes and edits separated from encoder pretraining? Do the unseen-edit results hold across other scenes, and what explains the weaker seasonal results compared with DINO conditioning?

  5. Can you report the final SH-plus-opacity, fixed-scale configuration in Table 3(c)? The current combined configuration also changes scale.

I would consider raising my score if the authors address these concerns with clear explanations and targeted supporting results.

Reviewer_WGJUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper builds an editable 3DGS scene from a single image. It uses camera-controlled video generation to expand the scene into a default 3DGS. Shared geometry and learned representations then support consistent editing. The paper also introduces a scene-editing benchmark with 60 indoor and outdoor scenes.

Critical strengths
  1. The benchmark of 60 indoor and outdoor scenes is a useful resource for scene editing.

  2. The supplementary video shows visually better results than the alternatives.

Critical weaknesses
  1. Table 1 shows small gains and low absolute quality. The proposed method reaches 16.03 PSNR on generated-path views, against 15.45 for GEN3C. On held-out views it reaches 14.49, against 12.75 for the best baseline. These values are low in absolute terms. Single-image setups are harder than per-scene optimization, but the authors should explain what level of quality is practically usable. The method is also not best on the vis. column for generated frames, where GEN3C scores 15.38 against 14.63. This conflicts with the text claiming the best results in both settings.

  2. Table 2 does not clearly support the editing claim. VideoPainter has higher background consistency in both settings. UniVideo has higher motion smoothness for time-of-day edits. The gains in aesthetic and imaging quality are small, and no variance or significance is reported. VBench metrics also measure video quality, not edit fidelity or 3D consistency.

  3. Possible cherry-picking. The qualitative results look better than the numbers suggest. The paper should show random or failure cases, report per-scene results, and give a user study or an edit-specific metric.

Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.