acceptodds
Under review as a conference paper at ICLR 2027

DensiTok: Making Feed-Forward 3D Gaussian Splatting See More Views Than It Is Given

Abstract

Feed-forward 3D Gaussian Splatting (3DGS) reconstructs a scene in a single forward pass, replacing per-scene optimization with a network trained across many scenes. Its quality, however, degrades sharply as the number of input images drops. The bottleneck is upstream of the reconstruction heads: from a few unposed views, the internal representation they read carries no evidence for unobserved regions, leaving holes, floaters, and blur. The common remedy supplies that evidence as pixels, synthesizing extra views with an image or video generator and re-encoding them, which is costly and not 3D-consistent by construction. We instead densify the evidence itself. We present DensiTok, a plug-in module for pretrained feed-forward 3DGS models that densifies their internal geometry tokens directly, making a frozen backbone behave as though it had observed many more views than it was given. DensiTok compresses those tokens into a compact latent space, completes the latents of the unobserved viewpoints in a single flow-matching step conditioned on camera geometry, and decodes them back into tokens that the original reconstruction heads. The same module design can be integrated into different pretrained predictors while keeping each backbone and its reconstruction heads frozen. Completion in a low-dimensional latent space requires no image synthesis or additional encoder passes. Across three pretrained backbones and two benchmarks, DensiTok consistently improves sparse-view reconstruction and recovers much of the gap to dense-view reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward Gaussian Splatting, Novelview Synthesis, Flow Matching, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hwHHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

THe paper introduces DensiTok, a plug-in latent completion module for pretrained unposed feed-forward 3DGS backbones that mitigates sparse-view degradation by synthesizing intermediate multi-level geometry tokens for unobserved viewpoints directly from sparse input views. DensiTok first uses a LAM-VAE to compress high-dimensional multi-level tokens into a compact latent space, and then trains a camera-conditioned DiT to complete missing view latents along an interpolated camera trajectory before decoding them back into full-view token sets for the reconstruction heads. Evaluated across three pretrained backbones, DensiTok consistently improves rendering, depth, and pose accuracy.

Critical strengths

[S1] Plug-in compatibility across different backbones: DensiTok consistently improves appearance, depth, and camera pose across AnySplat, WorldMirror, and DA3, while running faster than 16-view dense inference.

[S2] Compact multi-level token compression and one-step latent completion enable faster generative 3DGS: Rather than synthesizing novel RGB views and re-encoding them, or running cascaded multi-step diffusion on large feature channels as in GLD, DensiTok compresses backbone feature levels into a four-channel latent per token and completes missing views. As a result, DensiTok can produce improved quality with fast inference speed.

Critical weaknesses

[W1] Unexplained gap over the 16-view teacher: Sparse-view DensiTok substantially outperforms the 16-view teacher backbone. Because stage 2 and 3 supervise against ground-truth target images rather than teacher renderings, token completion is entangled with in-domain supervised adaptation on Re10K and DL3DV.

[W2] Ambiguity of the warped color prior and appearance adaptation: While the main paper frames DensiTok as a pure token-space module with frozen heads, AnySplat require per-view RGB inputs which DensiTok supplies for unobserved views using a depth-warped color prior, a learned hole embedding, and finetuning. It is unclear in the main text that which backbones use this color prior and finetuning, whether “Fully Frozen” in Table 12 still relies on depth-warped color prior or uses no RGB warping at all.

[W3] Observed-view latent mismatch (ziobsz_i^{obs} vs. ziz_i): Both the geometry encoder and LAM-VAE use global cross-view attention, so the observed latents ziobsz_i^{obs}(encoded from n views) differs from ziz_i(encoded from N views) and carry degraded cross-view features.

[W4] Trajectory interpolation: Fitting a cubic spline through predicted sparse-view poses assumes ordered inputs along a smooth 1D path (this is not general unordered collections) and is vulnerable to noisy initial pose estimates such as n=2.

Questions for authors

[Q1] Why does sparse DensiTok exceed the 16-view teacher? What is the performance of DensiTok at n=N=16n=N=16, or when stage 2 and 3 are supervised purely by teacher outputs rather than GT images?

[Q2] In Table 12, does “Fully frozen” still pass the warped color prior for unobserved views, or does it omit the warped color prior entirely? Also, do WorldMirror and DA3 use Appendix A.2?

[Q3] How much does performance improve if conditioned on oracle (scale-aligned GT) cameras at inference?

1 comment

Thank you for the clarification and the new experiments. All my concerns are resolved, I will maintain my positive score.

Reviewer_VtoDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes DensiTok, a method that generates geometry tokens for unobserved viewpoints from sparse input images. LAM-VAE compresses features from multiple encoder layers, and a camera-conditioned flow matching model completes the latents for missing views. These latents are decoded back into geometry tokens and passed to existing reconstruction heads to predict a Gaussian scene, cameras, and depth. The method is evaluated with AnySplat, WorldMirror, and DA3 across multiple input view counts and datasets.

Critical strengths
  • Directly augmenting geometry tokens without generating additional images and re-encoding them is an interesting approach. Applying the same module design to three backbones and reducing sampling costs through one-step generation are useful practical contributions.

  • The experiments are comprehensive. Evaluations with 2, 3, 5, and 7 input views generally show improvements in rendering quality, camera accuracy, and depth accuracy. The paper also provides component ablations and comparisons with sparse-view reconstruction and generative methods. Results on ETH3D, ScanNet, and Mip-NeRF 360 support its generalization ability.

Critical weaknesses
  • The geometric consistency of the generated views needs further analysis. Jointly processing multiple views to reconstruct a single Gaussian scene is a reasonable design. However, the paper does not directly evaluate whether the depth and camera predictions decoded from the generated tokens are geometrically consistent across views. This is particularly relevant in sparsely observed or occluded regions.

  • Sensitivity to camera errors is unclear. Sec. 3.4 uses scale-aligned ground-truth cameras during training but predicted and interpolated cameras during inference. Scale alignment does not address relative pose errors. The reported results show that the method works under this setup, but its behavior with low input overlap or inaccurate initial poses remains insufficiently analyzed.

  • The relationship to existing geometry token generation methods needs more discussion. VGGT-World [1] also uses flow matching to predict geometry features and decodes them through a frozen reconstruction model. Although it addresses temporal forecasting rather than sparse-view reconstruction, the shared idea of generating geometry features without synthesizing images is relevant. The paper should clarify its technical differences from this work.

[1] Sun et al. “VGGT-World: Transforming VGGT into an Autoregressive Geometry World Model.” ECCV 2026.

Questions for authors
  • Does training account for errors in the camera conditions? Please share any existing results broken down by the three input-overlap ranges, or failure cases involving inaccurate initial poses.

  • How do the gains differ between regions with and without valid RGB reprojections in Appendix A.2? If this analysis has already been conducted, please provide the results.

Reviewer_NPGUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper propose DensiTok, a diffusion-like pipeline for sparse-view 3D Gaussian reconstruction. The proposed method uses a level-adaptive VAE to compress multi-level geometry features and a camera-conditioned diffusion transformer to complete novel view latents. The transformer is trained with flow matching and subsequently fine-tuned for one-step inference.

Experiments with AnySplat, WorldMirror, and DA3 on RealEstate10K and DL3DV report improvements in sparse-view novel view synthesis.

Critical strengths

DensiTok addresses sparse-view reconstruction by generative models. This avoids generating additional RGB images with a generative model and re-encoding them, while retaining the pretrained geometry piror.

The ablations support the contributions of level-adaptive fusion and the one-step training strategy.

Critical weaknesses

DensiTok generates missing-view latents from random noise, but the effect of this randomness on reconstruction stability and accuracy is not quantified. Different samples could potentially produce different geometry or appearance from the same observations. With the input images and virtual cameras fixed, how much do the reconstructed scenes and NVS metrics vary across random seeds? Could the authors report repeated-sampling statistics and visual comparisons, particularly in poorly observed regions?

In the bottom-left example of Figure 3, DensiTok reduces artifacts present in the AnySplat baseline, but the reconstruction still appears blurred. This example does not clearly demonstrate whether the method recovers meaningful missing geometry and appearance, beyond improving the overall visual quality. Could the authors provide qualitative comparisons and quantitative evaluations specifically for regions unobserved in the input images, to demonstrate the accuracy and plausibility of the completed content?

Questions for authors

See Weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.