acceptodds
Under review as a conference paper at ICLR 2027

SplatRefiner: Faithful and Efficient Post-Hoc Diffusion Refinement for Sparse-View Novel View Synthesis

Abstract

Sparse-view novel view synthesis requires inferring content that a few input images leave unresolved. Many diffusion-assisted pipelines supply this content through reconstruction–generation cycles along predefined camera trajectories, making novel views costly to obtain. Refining renderings of a fixed Gaussian reconstruction directly into final outputs avoids these cycles. However, these renderings mix useful scene content with artifacts, and without 3D re-optimization, the refiner alone must remain faithful to the observed scene. To address this, we exploit information from both the reference views and the target view, during training and at inference. During training, VGGT features of the reference views condition denoising, and those of the ground-truth target view supervise intermediate representations. At inference, dual-condition guidance strengthens the contributions of the reference views and of the rendering of the target view. Motivated by these, we present SplatRefiner, a post-hoc diffusion refiner that restores renderings at arbitrary requested poses without predefined trajectories. On identical renderings, SplatRefiner outperforms strong baselines in most experiments while in 3 input view settings it runs 6.9 faster end-to-end with comparable PSNR using 2DGS, and 3.7 faster with higher PSNR using FSGS. Trained only on 2DGS renderings from DL3DV-10K, it improves PSNR, LPIPS, and FID over the renderings of 2DGS, 3DGS, and FSGS across three datasets, two unseen during training, with 3, 6, and 9 input views.

Primary area:
generative models
Keywords:
Sparse-view novel view synthesis, Post-hoc refinement, 3D Gaussian splatting, diffusion models, representation alignment, geometry foundation models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CxFFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SplatRefiner, a diffusion-based post-hoc refinement method for sparse-view novel view synthesis. Instead of repeatedly generating novel views and re-optimizing the 3D representation, it directly refines renderings from a fixed Gaussian reconstruction. The method uses VGGT features for reference-view conditioning and target-view alignment, along with dual-condition guidance during inference. Experiments show improved rendering quality and reduced computational cost across multiple datasets and reconstruction methods.

Critical strengths
  1. The method avoids repeated reconstruction and generation cycles, which substantially reduces the overall processing time.

  2. The experiments cover multiple datasets, Gaussian reconstruction methods, and input-view counts. The results also show that a model trained on 2DGS renderings can generalize to 3DGS and FSGS.

  3. The paper examines different locations for VGGT feature injection and alignment. The results provide useful observations about how these components interact.

  4. SplatRefiner consistently improves the input renderings and outperforms DIFIX when applied to the same renderings in nearly all evaluated settings.

Critical weaknesses
  1. Feature injection, feature alignment, and classifier-free guidance are established techniques. The main contribution appears to be their combination and configuration for post-hoc refinement rather than a fundamentally new method.

  2. SplatRefiner substantially outperforms DIFIX, but the two methods use different pretrained diffusion backbones and training configurations. It is therefore difficult to determine how much of the improvement comes from the proposed components. A more controlled comparison would help clarify this.

  3. Although the paper provides consistency and correspondence evaluations, it remains unclear how reliably the method preserves the observed scene when the initial rendering contains significant errors. A more detailed analysis of such failure cases would strengthen the faithfulness claim.

  4. The comparison between consecutive and random view grouping is useful, but it does not sufficiently evaluate challenging target viewpoints. Additional results under large viewpoint changes or severe disocclusions would strengthen the trajectory-free claim.

Questions for authors
  1. Could you clarify the fundamental technical novelty of your method beyond the specific combination and configuration of established techniques (feature injection, alignment, CFG) for post-hoc refinement?

  2. Could you provide a controlled comparison or ablation study against DIFIX using the same pretrained diffusion backbone and training configuration to isolate the exact contribution of your proposed components?

  3. How reliably does the method preserve the observed scene when the initial rendering contains significant errors? Please provide an analysis of failure cases under these conditions to substantiate the faithfulness claim.

  4. To fully support the "trajectory-free" claim, could you provide additional evaluation results under more challenging target viewpoints, such as large viewpoint changes or severe disocclusions?

  5. A demo video is needed.

2 comments
Reviewer_CxFFOpenReview ↗

Official Comment

I still have some concerns about the novelty and the DIFIX comparison, but the authors have addressed most of my comments. I will keep my rating of weak accept.

Thank you for your careful review and for taking the time to consider our responses and additional experiments. Your constructive feedback has helped us strengthen our experimental validation.

We sincerely appreciate your time and consideration.

Reviewer_x5MuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SplatRefiner, a framework for sparse-view novel view synthesis that refines renderings from a fixed GS reconstruction without requiring a continuous camera trajectory or repeated cycles of image generation and scene optimization. The major contributions are:

  • The refiner is built on See3D, where continuous camera trajectory is not required, and can be used for various downstream reconstruction like 3DGS/2DGS/FSGS, etc.
  • The method incorporates features from a frozen VGGT model in two complementary ways: reference view features injection and joint target and reference view features alignment. A dual-condition classifier-free guidance scheme further controls the influence of reference images and Gaussian renderings at inference.

The reported results show consistent improvements in PSNR, LPIPS, and FID over the corresponding unrefined renderings on various datasets.

Critical strengths
  • A favorable quality–efficiency trade-off. SplatRefiner refines a fixed Gaussian reconstruction without repeated generation–optimization cycles or requiring target views to form a continuous trajectory.
  • Broad empirical support across reconstruction methods and datasets. Despite being fine-tuned only on 2DGS renderings from 300 DL3DV-10K scenes, the model improves PSNR, LPIPS, and FID over the corresponding unrefined 2DGS, 3DGS, and FSGS renderings across three benchmarks and multiple reference-view counts.
Critical weaknesses
  • The placment analysis conducted in Figure 3 and Figure 7 are thorough, but in the Figure 3 comparison, adding the selected alignment to Enc L2 injection improves PSNR by approximately 0.08 dB, while two versus four decoder alignment sites differ by only 0.03 dB. These results use one seed; multiple runs with different seeds are recommended to prove if the certain selections are stable.
  • There are a couple related works that align the intermediate diffusion representations with VGGT features, like Geometry Forcing [1]. GeoWorld [2] also combines VGGT-based geometric conditioning with an alignment loss between the VGGT features of generated and ground-truth images. Although GeoWorld's output-feature loss differs from SplatRefiner's direct supervision of denoiser hidden states, it establishes a relevant precedent for combining geometric conditioning and alignment. The authors should explain what these choices add beyond the existing approaches and, where feasible, compare alternative alignment objectives under the same backbone, conditioning setup, and training budget.
  • Minor presentation issues warrant proofreading and more precise labeling. Appendix A.5 contains subject–verb agreement errors: “The VGGT features improve all four metrics and adds another 0.34 dB” should use “add,” while “reference guidance add 0.56 dB” should use “adds.” The Figure 8 caption joins two independent clauses with a comma; a period or semicolon would improve readability. In addition, “unguided” in Appendix A.7 and the generic “Guidance” heading in Table 9 should explicitly refer to reference guidance, since rendering guidance remains active.

[1] Wu, Haoyu, et al. "Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling." International Conference on Learning Representations. Vol. 2026. 2026.

[2] Wan, Yuhao, et al. "GeoWorld: Providing Full-Frame Geometry Features to Facilitate 3D Scene Generation." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.

Questions for authors
  1. Clarification request: interpretation of the placement results. Decoder L1–L4 alignment performs best alone, whereas L1–L2 performs best when combined with Enc L2 injection. Should the conclusion be interpreted as an empirical choice among the tested configurations, or is there evidence supporting a broader explanation of the interaction between injection and alignment?
  2. Clarification request: novelty relative to prior geometric conditioning and alignment. Recognizing that methods like GeoWorld and Geomtry Forcing differ from SplatRefiner, what specific methodological advance does the proposed reference/target separation and joint placement strategy establish over these precedents?
Reviewer_DfVMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SplatRefiner refines sparse-view Gaussian renderings with a multi-view diffusion model while keeping the reconstruction fixed. It combines VGGT reference-feature injection, target-feature alignment during training, and dual-condition guidance. Experiments show improved rendering quality across reconstruction methods and datasets, at lower cost than iterative reconstruction–generation pipelines.

Critical strengths

The paper presents a practical approach to improving rendered images without repeated scene optimization. The evaluation covers multiple reconstruction methods and datasets. The component and placement ablations are informative, and the correspondence metrics provide useful evidence beyond image quality alone.

Critical weaknesses
  1. The comparison includes the full GSFixer reconstruction pipeline, but not its standalone restoration module applied to the same Gaussian renderings. GSFixer already uses reference-based VGGT features, and its restoration code and pretrained weights are public. This is a particularly relevant comparison: it would help establish whether SplatRefiner improves on an existing reference-guided refiner under the same input conditions. The standalone DIFIX comparison is useful but does not resolve this question.

  2. I could not find video results showing continuous camera motion in the provided material. Since the final images are diffusion-refined, consistency of the underlying Gaussian scene does not establish consistency of the outputs. Appendix A.9 provides useful correspondence metrics, but stability along a trajectory, especially across independently refined groups of views, remains insufficiently demonstrated. The random-versus-consecutive grouping experiment measures average image quality rather than continuity across groups.

Questions for authors
  1. How does SplatRefiner compare with GSFixer’s standalone restoration module on identical coarse renderings, using a view sequence supported by both methods? If this comparison is not feasible, please explain the relevant differences in their input requirements.

  2. Can you provide continuous-camera videos spanning multiple independently refined groups, with group boundaries marked? Does the same target view remain consistent when included in different groups?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.