acceptodds
Withdrawn from ICLR 2027

SpecGuard: A Camera-Manifold Gaussian Atlas for Radiance Field Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Radiance field reconstruction seeks to recover a renderable 3D scene from multi-view images. Monocular video captures changing views of scene structure and reflections, requiring a representation that preserves local detail while adapting to view-dependent appearance. We address this challenge with , a camera-manifold Gaussian atlas that combines scaffold-seeded local submaps with pose-supported soft composition. Given camera poses, a Gaussian scaffold seeds trajectory-local Gaussian submaps, each optimized for its local viewing distribution. To render across submaps, pose-supported soft composition suppresses submap switching artifacts, while rasterized opacity adjusts contributions according to ray-wise coverage. To reduce the storage overhead of overlapping submaps, shared low-rank bases and sparse residuals compress their non-DC spherical-harmonic coefficients. Full-session evaluations demonstrate improved aggregate reconstruction fidelity and reduced submap switching artifacts. The resulting representation reduces map storage with a small fidelity loss while supporting real-time rendering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Gaussian Splatting, wild scenes, SLAM backend, mapping

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1TZsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes SpecGuard to address redundant geometry initialization, artifacts caused by switching between local submaps, and storage redundancy in monocular reconstruction. It uses a shared Gaussian scaffold to initialize local submaps consistently, avoiding repeated geometric front-end processing. To reduce visual discontinuities during submap switching, it introduces a pose-supported soft composition mechanism. In addition, its query-aware appearance factorization compresses the non-DC spherical harmonic coefficients, reducing storage by 39.8% at a PSNR cost of 0.058 dB.

Critical strengths
  1. The method includes substantial engineering optimizations. It keeps only a limited number of local maps active, enabling interactive rendering. In addition to rasterization FPS, the paper reports loading time and GPU memory usage.
  2. The paper provides a reasonably comprehensive set of quantitative and qualitative experiments. In the initialization ablation in Table 2, the variants have the same final number of Gaussians, which makes the comparison fairer and helps rule out the possibility that the improvement merely comes from using more Gaussians. Figure 5 clearly illustrates the artifacts caused by hard assignment and the smoother transitions produced by soft composition. The paper also discusses its limitations, including the uneven gains across datasets.
Critical weaknesses
  1. The method appears to be more clearly beneficial for mapping and replay than for reconstructing a better radiance field. The reported 1.92 dB improvement is measured at previously observed viewpoints. Table S21 evaluates held-out views, but it includes only three scenes, and the test viewpoints remain close to the original trajectory.
  2. Opacity-aware blending provides little additional benefit. In Table 2(b), moving from pose blending to the opacity-aware variant changes the reported metrics only slightly.
Questions for authors
  1. Could the authors add a held-out evaluation whose test cameras are excluded from blend calibration? In particular, I would like a split with viewpoints that deviate substantially from the training trajectory.
  2. How many local submaps and Gaussians are stored for each scene? How do the total training time and peak training GPU memory compare with those of the global baseline?
  3. To reduce storage, the method groups similar local submaps into families and shares a set of basis features within each family. However, Table S19 shows that the PSNR loss is nearly the same with and without grouping. What practical benefit does this additional grouping mechanism provide?
Reviewer_cn9dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method to use local maps to improve 3D GS rendering quality. They split the scene into segments based on the timestamps of video frames and use the view frustums to select global primitives for local map initialization. Then each local map is optimized independently, followed by optimization of per-local-map blending calibration parameters. Finally, they propose factorization of the SH coefficients to reduce storage. Results show their method improves novel view synthesis on several datasets compared to previous methods.

Critical strengths
  • The paper presents a practical method for 3D GS reconstruction through local maps. They improved NVS quality over existing baselines.
  • The compression of SH coefficients is useful, reducing storage by 39.8% and losing only 0.06 PSNR.
Critical weaknesses
  • The novelty of the work is limited. Local radiance fields[2,3] and PCA-based appearance compression[3] already exist; the technical contribution needs further clarification.

  • The ablation studies have mixed signals regarding design choices. For starters, in Table 2, "Hard assignment" has the highest PSNR, while the more complicated composition does not improve LPIPS over "Pose Blend". In Table S19, "Single shared basis" has the same storage as "ours", but it has a smaller PSNR drop, meaning the proposed "query-aware factorization" is actually harmful. The ablation studies require further clarification in order to justify these design choices.

  • The training time is not reported. The end-to-end training time for scenes in each dataset cannot be found. This is important to help understand how much more or less training resources the method needs compared to baselines for the quality gain.

  • Experimental results are not comprehensive. The method is motivated by better modeling of reflective surfaces (Line 64), but it would benefit from evaluation on the reflective dataset RefNeRF. Also, existing qualitative results do not visualize the same reflective surface showing different reflections across different submaps, which is important to demonstrate the effectiveness of the proposed method.

Reference:

  1. Z. Zhu, S. Li, Y. Liao and L. Yu, "Low-Rank Approximation for Efficient Compression of Gaussian Splatting Spherical Harmonics," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 7, pp. 9225-9238, July 2026, doi: 10.1109/TCSVT.2026.3670178.
  2. Meuleman, Andreas, et al. "Progressively optimized local radiance fields for robust view synthesis." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
  3. Lin, Jiaqi, et al. "Vastgaussian: Vast 3d gaussians for large scene reconstruction." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
Questions for authors

Novelty, ablation studies, and training time need clarification; details can be found in the weaknesses.

Reviewer_eHWYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SpecGuard, a Gaussian atlas for reconstructing scenes from posed monocular videos. It first builds a global Gaussian scaffold to initialize trajectory-local submaps. At rendering time, it blends nearby submaps using camera-pose similarity and rasterized opacity to reduce switching artifacts. It also compresses spherical harmonic (SH) coefficients using shared low-rank bases and sparse residuals.

Experiments across four benchmarks show a 1.92 dB average PSNR improvement over RT-Splatting under shared poses, smoother transitions than hard submap assignment, and a 39.8% reduction in atlas storage with only a 0.058 dB PSNR loss.

Critical strengths
  1. A practical approach with good reconstruction and rendering results.

The paper addresses a useful problem: how to preserve local view-dependent appearance while rendering smoothly across a long camera trajectory. The overall system is well motivated, and the results are promising. In particular, pose-supported composition substantially reduces submap switching artifacts, while the system maintains real-time rendering at an average of 84 FPS.

  1. Fairly thorough and transparent experiments.

I appreciate the resource-matched global-map comparisons and component ablations. The authors also report cases where local submaps underperform a global map and where soft composition trades some fidelity for continuity. These results help clarify the strengths and limitations of the approach.

Critical weaknesses
  1. The gains on unseen views are much less convincing than the main results suggest.

The main evaluation focuses on observed views used for reconstruction and calibration. While this is useful for assessing map fitting, generalization to unseen viewpoints is important for radiance field reconstruction.

Appendix Table S21 evaluates withheld views on three scenes, but the average PSNR improvements over the global map are only 0.48 dB for interleaved holdout and 0.13 dB for contiguous holdout. These are much smaller than the gains highlighted in the main experiments.

The continuity evaluation also measures differences between adjacent rendered frames without ground-truth novel views. This demonstrates smoother transitions, but does not necessarily establish more accurate view synthesis. Overall, the evidence for improved novel-view reconstruction is still limited.

  1. The novelty of the individual components is not sufficiently established.

Trajectory-local radiance fields and partitioned Gaussian representations have been explored in LocalRF and VastGaussian. Pose-based blending and low-rank appearance compression also build on established techniques. SpecGuard offers a useful combination, but I am not fully convinced that its specific new mechanisms provide substantial improvements.

For example, Table 2(b) shows that most of the continuity improvement comes from basic pose blending, while ray-wise opacity adds only a small fidelity gain. More importantly, Appendix Table S19 shows that the proposed query-aware appearance grouping performs essentially the same as a single shared basis or pose-only grouping at the same storage size.

These results make it difficult to identify which parts of the proposed design are truly necessary.

Questions for authors
  1. Could you expand the held-out novel-view evaluation to the full benchmark, particularly for contiguous trajectory gaps? This would help establish whether local specialization improves generalization rather than mainly fitting observed views.

  2. Could you clarify the benefit of query-aware appearance grouping over a single shared basis? Table S19 shows nearly identical results, so it is unclear when the additional grouping complexity is useful.

  3. Could you provide a more direct measure of submap transition quality, ideally using ground-truth views or motion-aligned consistency? Adjacent-render differences alone may not distinguish smoother rendering from reduced visual detail.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.