- Rating
- 4
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work presents LoCoSplat, a feed-forward 3D Gaussian Splatting framework that replaces a learned sparse 3D U-Net with a fixed two-scale grid average and small pointwise networks. The central claim is that, after depth prediction and voxelization, each Gaussian mainly needs low-order local geometric context rather than a high-capacity learned 3D module. On RealEstate10K, the method reports the best PSNR, SSIM, and LPIPS among the evaluated feed-forward methods at 6, 12, and 24 input views, while reconstructing a six-view scene in 33 ms and using substantially less memory than VolSplat. The paper also reports zero-shot ACID and fine-tuned ScanNet results, controlled ablations of the 3D stage, and an analysis attributing dense-view degradation to the plane-sweep candidate spacing in the depth branch rather than to 3D aggregation.
Critical strengths
-
Main design of proposed method is well motivated and practically important. Replacing dynamic sparse 3D computation with fixed grid averages and pointwise maps makes the encoder compatible with a single static CUDA graph, and the reported latency and memory gains are substantial.
-
Empirically, LoCoSplat performs strongly along several dimensions. It leads every evaluated image-quality metric at 6, 12, and 24 views on RealEstate10K, surpasses a retrained VolSplat baseline, and stays competitive or better when transferred zero-shot to ACID or fine-tuned on ScanNet.
-
Architectural and systems gains are disentangled effectively in the cumulative ablations. Removing the U-Net by itself improves both reconstruction quality and speed, after which the deployment stack cuts latency further with essentially no change in the reported metrics.
Critical weaknesses
-
Generality remains established only within a fairly narrow operating regime. The experiments use fixed voxel sizes of 0.001 and 0.128, posed inputs, and 256×256 images. As the paper itself notes, those scales may not carry over to 512×512 resolution, wider baselines, noisy poses, or poorer depth, all of which matter for deploying a fixed spatial aggregation rule.
-
Isolation of the proposed 3D mechanism is clearest only for 6 views RealEstate10K setting, primarily against VolSplat's U-Net. The headline end-to-end model simultaneously changes the plane-sweep candidate count, backbone interface, regressor width, prediction head, and deployment path. Consequently, the full quality and runtime improvements cannot be assigned solely to two-grid aggregation.
-
Qualitative examples are selected through a procedure that favors LoCoSplat: frames are removed when any baseline has higher PSNR or when LoCoSplat falls below 26 dB, and the remaining cases are ranked for inspection of local failures. The authors disclose this process, but the figures therefore emphasize selected successes rather than typical outputs or a balanced set of failure cases.
Questions for authors
-
How do reconstruction quality and memory respond to the fine and coarse voxel sizes as image resolution, focal length, or scene scale changes? Could the authors give a normalized rule for setting these values, or report experiments at 512×512 and on data with wider baselines?
-
For 12 and 24 views, what portion of the gain over retrained VolSplat comes from the two-grid 3D stage itself, as opposed to reducing the plane-sweep candidates from 128 to 64 and modifying the backbone? A factorial study at these view counts would make the attribution clearer.
-
Could the evaluation include randomly sampled visual results and representative cases in which a competing method outperforms LoCoSplat, alongside the existing diagnostic selection? Such examples would provide a more interpretable visual comparison.
-
Is the depth collapse at 24 views consistent across training seeds, and can different optimization or regularization prevent it while preserving 128 candidates? The current experiments connect the failure to disparity per candidate, but do not fully explain why the learned output becomes an input-independent one-hot distribution.
We thank the reviewer for recognizing our design, results and ablations, and for the constructive questions and suggestions. We have uploaded a revised paper with all changes marked in blue, and respond to each point below.
W1 (noisy poses, poorer depth). We added noise to the input poses (0.5-2∘ rotation, 1-5% translation) and to the predicted depth (3-5%) on all 709 RealEstate10K test scenes, without retraining. LoCoSplat stays ahead of VolSplat at every level (+0.25 to +0.61 PSNR); pose errors hurt both models equally, so the learned U-Net does not handle them better than our fixed average. Details are in Sec. 3.5 and App. C.6 (Tab. 15).
W1 / Q1 (voxel sizes vs. resolution, focal length, scene scale; 5122). v and H need no retuning: the rule is v sub-pixel (so anchors are pixels) and H≈4% of the median depth, which 0.001/0.128 satisfy. Across scene scales, the gain over VolSplat does not vary with H (App. C.4; see our response to Reviewer ttZu, Q4). Without retraining, from 2562 up to the native 352×640 and at 1.25×/1.5× focal length, v stays ≤0.12 px and H at 3.5-4.7% of the depth, and rescaling H at test time by 0.67-1.375× changes PSNR by ≤0.13 dB. Memory and time grow roughly linearly with pixel count: at every size up to 512×896, LoCoSplat's encoder is 1.9-2.4× faster than VolSplat's with 1.9× less memory, and our deployed path needs only 153 ms / 3.4 GB at 5122 (8× less memory than VolSplat). Details are in App. C.7 (Tab. 16).
W2 / Q2 (attribution at 12 and 24 views). As suggested, we added the factorial study (App. C.3, Tab. 10). The two-grid model changes only VolSplat's 3D stage, keeping its backbone and D=128. At 12 views, the 3D stage alone gives +0.58 of LoCoSplat's +0.61 PSNR. At 24 views, D decides the collapse, and the two-grid 3D stage stays ahead of the U-Net at either D (+0.70 at D=128, +0.17 at D=64). The speedup also comes from the 3D stage: at equal D=64 our encoder is still 3.7× faster.
W3 / Q3 (selected qualitative examples). As suggested, App. E.5 now adds randomly sampled results and failure cases where a baseline beats LoCoSplat (Figs. 14-15), plus per-baseline win rates over all test scenes (Tab. 21).
Q4 (collapse across seeds; prevention with 128 candidates). We retrained LoCoSplat with D=128 at 24 views with two new seeds: one collapses and one does not. In total, 4 of 5 D=128 trainings collapse, while every D=64 model is healthy. The collapse is therefore an optimization failure, decided within the first few hundred steps, when a collapsing run's matching distribution locks into a one-hot it never leaves. It can be prevented with all 128 candidates kept: matching each view against its second-nearest cameras restores the disparity per candidate. On the collapsing seed, this keeps the model healthy, as we predicted before training, and gains +5.1 PSNR (25.37 to 30.48). Details are in Sec. 3.5 and App. D.4 (Tab. 20).