- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces UniFill3D, a unified framework for 3D Gaussian Splatting (3DGS) reconstruction under diverse imperfect observation conditions framed collectively as information deficiency. The approach produces two target-aligned estimates for each camera: a scene-grounded Reconstruction Anchor rendered from the current 3D representation and a Prior Prediction generated by an external restoration or generative model; these are reconciled by a hierarchical fusion module that combines dual-latent joint attention in a single-step diffusion U-Net with multi-scale prior-guided VAE decoding, after which the restored views supervise 3D refinement.
Critical strengths
-
The target-aligned, two-branch abstraction is a clean, reusable interface that decouples deficiency-specific recovery from shared reconciliation.
-
The fusion mechanism combines a dual-latent joint-attention U-Net with single-step diffusion and multi-scale prior-feature injection into the VAE decoder, providing a principled, lightweight alternative to multi-step sampling.
Critical weaknesses
-
The core idea using external priors to refine 3D reconstructions via target-aligned fusions and feeding restored views back is close to several prior pipelines, making the incremental novelty of the proposed dual-latent single-step diffusion + VAE feature injection the main technical delta.
-
No explicit mechanism for reliability estimation or gating of the prior; if Pq hallucinates off-geometry content, the fusion may propagate artifacts into 3D without uncertainty-aware weighting.
Questions for authors
For sparse-view, how do you choose which cameras receive pseudo-observations and how many? Is performance sensitive to the schedule of adding restored views and to the relative weighting of restored vs. real views during 3D refinement?
5 comments
Response to Reviewer TRcd (Part 2/2)
Weaknesses
W1. Relationship to Existing Pipelines and Technical Contributions
UniFill3D follows the restore-then-refine paradigm and uses the DiFiX3D+ backbone and checkpoint initialization. Our focus is fusing the current reconstruction and external predictions within this procedure.
We construct a Reconstruction Anchor and a Prior Prediction for the same target camera, then combine them through dual-latent interaction and multi-scale feature injection during decoding. For sparse views, a video generation model supplies additional information at unobserved views, and the current Gaussian rendering provides the corresponding scene estimate. Fused views then supervise 3D refinement. The independently trained ablations in the submission show that full fusion outperforms either branch alone on both sparse-view reconstruction and super-resolution.
This target-aligned fusion interface, architecture, and training objective can be reused across tasks and are not tied to specific internal reconstruction or external prior models. A reconstruction model and a generative or restoration model can replace the respective branches if their predictions correspond to the same target camera and image plane. Replacement requires retraining the fusion weights, while the shared fusion architecture and training objective remain unchanged. Different model combinations can therefore fuse their predictions through the same interface and use the restored views for 3D optimization.
W2. Prior Reliability and Hallucination Propagation
During fusion, the updated Reconstruction Anchor latent and prior features jointly participate in decoding. During sparse-view refinement, real observations continue to constrain the Gaussian representation jointly optimized across views. We rely on these two stages to constrain prior use; joint 3D optimization is intended to suppress residual inconsistencies among restored views.
The current method has no separate confidence estimate, geometry-aware gate, or conflict-based view rejection. Adding reliability prediction and geometric checks is a possible extension for reducing prior-error propagation.
Please let us know if any point needs further clarification or if you have additional questions. We will respond promptly. Thank you again for your time and thoughtful feedback.
Follow-Up on Rebuttal Discussion
Dear Reviewer TRcd,
Thank you again for your time and valuable feedback. We have provided point-by-point responses, additional experimental results, and clarifications in the revised manuscript to address your concerns.
As the discussion period draws to a close, we would appreciate hearing whether any concerns remain unresolved or require further clarification. We would be happy to provide additional explanations during the remaining discussion period.
Once you have had an opportunity to consider our responses and additional results, we would also appreciate your updated assessment.
Thank you again for your time and consideration.
Best regards, Paper26205 Authors
Response to Reviewer TRcd (Part 1/2)
We thank the reviewer for recognizing the two-branch interface and fusion mechanism. We provide the pseudo-observation settings and additional tests of supervision weights and refresh schedules below.
Questions
Q1. Pseudo-Observation Cameras, Counts, Introduction Schedule, and Supervision Weights
Pseudo-observations follow the progressive camera path of DiFiX3D+: cameras advance from near the input views toward the target poses. Each round rerenders the scene, obtains external predictions, and performs fusion at those camera positions. This process uses target camera poses.
Each round contains one pseudo-observation per target camera. The count stays fixed within a scene, while camera positions and coverage change. The main settings are:
Sampling probability controls how often a supervision source participates; the loss weight controls its strength in that update.
Sensitivity to the Introduction Schedule and Relative Supervision Weight
We completed two sets of controls on three preselected DL3DV scenes with 148 target views. All other settings match the paper's progressive refinement configuration. All metrics are measured after final query-view fusion, matching the paper's sparse-view evaluation.
The weight test uses a uniform schedule and pseudo-view loss weights of 0.15, 0.30, and 0.60. The real-view weight remains 1.0, and the pseudo-view sampling probability remains 0.3. The schedule test fixes the original weights and 15 refreshes, comparing uniform spacing with schedules that refresh more frequently early or late in optimization. More frequent early refreshes use seven 1K intervals followed by seven 3K intervals; more frequent late refreshes reverse this order.
Relative to the original configuration, the absolute change in mean PSNR for final fusion is at most 0.035 dB. This small-scale test suggests that the method does not depend on a precise weight or uniform refresh intervals.