acceptodds
Under review as a conference paper at ICLR 2027

ReLo-S4DGS: Reliability Modeling for Locality Optimization in Sparse-View Dynamic Gaussian Splatting

Abstract

Recent 4D Gaussian Splatting methods have achieved impressive quality and efficiency for dynamic scene reconstruction, yet sparse-view inputs severely underconstrain scene geometry, leading to geometric ambiguity and view-inconsistent reconstructions. We observe substantial locality degradation in anchor-based Gaussian representations under sparse supervision: to fit limited observations or compensate for insufficient spatial coverage of anchors, anchors tend to decode excessively large offsets, placing Gaussians far from their source anchors and undermining representation locality. Based on this observation, we propose ReLo-S4DGS, which, to the best of our knowledge, is the first anchor-based Gaussian Splatting framework for sparse-view dynamic scene reconstruction. To address locality degradation, ReLo-S4DGS models anchor reliability and suppresses low-reliability rendering contributions to favor reliable local representations, we further introduce dynamic-aware locality correction to rectify excessive offsets and geometric regularization to constrain local geometry, thereby mitigating sparse-view overfitting. Extensive experiments on standard dynamic scene benchmarks demonstrate state-of-the-art performance of our method.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Gaussian Splatting, Sparse-view Scene Reconstruction, Dynamic Scene Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YrEvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReLo-S4DGS, an anchor-based 4D Gaussian Splatting framework tailored to sparse-view dynamic scene reconstruction. The key idea is to explicitly model and optimize locality in anchor-based representations under sparse supervision by estimating per-anchor reliability from complementary spatial, temporal, and cross-view evidence via a Beta-evidence model, then using this reliability to modulate opacity, perform dynamic-aware locality correction, and add reliability-weighted locality regularization plus sparse keyframe depth supervision.

Critical strengths
  • Beta-evidence reliability that aggregates three complementary signals is a principled, interpretable design and novel in this anchor-based, dynamic, sparse-view context.

  • The diagnosis of locality degradation in anchor-based dynamic GS under sparse views is insightful and well-quantified.

Critical weaknesses
  • The reliance on external depth priors (DA3) for keyframes injects additional supervision that several baselines may not use, complicating attributions of gains purely to reliability/locality modeling.

  • The cross-view gradient agreement statistic may be computationally heavy; overhead and scalability to long sequences or high resolutions are not reported.

  • The dynamicity metric D_i based on feature norms may be sensitive to feature scaling; no normalization or calibration study is provided to show stability across scenes.

Questions for authors

How large is the computational overhead of reliability statistics, especially cross-view gradient agreement, per update interval? Please provide training time and memory breakdowns vs baselines.

Reviewer_XskkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ReLo-S4DGS for sparse-view dynamic reconstruction with an anchor-based Gaussian representation. It estimates anchor reliability from local geometric risk, temporal repeatability, and cross-view gradient agreement using a decayed Beta model. Reliability controls opacity and locality regularization, while dynamic-aware correction recenters low-dynamic anchors and splits high-dynamic anchors with large offsets. Depth Anything 3 supplies additional keyframe geometry.

Critical strengths
  1. The paper identifies a meaningful failure mode: large decoded offsets can indicate either locality degradation or genuine motion. This distinction is important because penalizing all large offsets would suppress valid dynamics.

  2. The modules match the anchor representation: opacity modulation, locality regularization, and dynamic-aware recentering or splitting address different stages of the reconstruction process.

  3. Reliability accumulates evidence over time and views. Decay and observation filtering can reduce the effect of a single noisy or weakly observed update.

Critical weaknesses
  1. The Beta evidence mapping is largely hand designed. Temporal repeatability is only positive evidence, local risk only negative evidence, and gradient agreement can have both signs. The statistical meaning of this asymmetry is not fully justified.

  2. The score compares the same indexed anchor and decoded slot at the same time across views. However, the decoder is also conditioned on viewing direction, and the paper does not establish that those view-conditioned outputs represent comparable geometry. The gradient cosine may therefore reflect decoder-induced changes as well as consistent optimization.

  3. The dynamicity measure is representation dependent. It uses the relative norms of dynamic and static features rather than direct motion or displacement, and its ability to separate genuine motion from locality drift across scenes is not established.

  4. The method depends on Depth Anything 3 and uniformly sampled keyframes. The paper does not establish how a different depth estimator, noisier predictions, or another sampling schedule would affect the method.

  5. The appendix shows that high-reliability anchors tend to occur in better-observed, structurally stable regions and that removing them affects rendering more strongly. This supports observability and rendering importance, but does not independently establish that the score measures geometric accuracy.

If these concerns are addressed, I would consider increasing the rating.

Questions for authors
  1. Since the decoder uses viewing direction, what supports treating the same indexed Gaussian slot across views as geometrically comparable, and how could view-conditioned changes affect the gradient-agreement score?

  2. What justifies assigning temporal repeatability only to positive evidence and local risk only to negative evidence? Would alternative mappings change the score’s interpretation?

  3. How dependent is the method on Depth Anything 3 and uniform keyframe sampling? Are they essential or replaceable sources of depth supervision?

Reviewer_PBRfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the sparse view challenge in dynamic novel view synthesis. Under the anchor-based methods, they observe that the locality decreases as the views become sparser, which weakens the association between anchors and local gaussians. They also find that it is necessary to determine the reliability of an anchor to adjust the locality correction. Based on these findings, the paper introduces a reliability modeling framework for locality optimization in sparse view dynamic reconstruction. The experimental results demonstrate the improvements over several existing methods.

Critical strengths
  • The findings on locality degradation are interesting and inspiring, showing quantitative evidence of the different effects of decoded offset in sparse and dense view reconstruction.
  • The method achieves competitive performance in three benchmarks, with an ablation study showing the effectiveness of adding different components.
Critical weaknesses
  • The novelty of the paper is limited, with too many heuristic module designs. The proposed reliability measure is not justified for distinguishing correct and incorrect geometry. The proposed local geometric risk, temporal repeatability, and cross-view gradient consistency are hand-crafted signals and assumed to correlate with reliability, without extensive validation of their actual effectiveness and correctness. While Table 2 shows the ablation study of removing each component, it shows that the largest improvement comes from DA3 depth supervision, not from the proposed reliability modeling.
  • The central hypothesis has insufficient validation. Although locality degradation motivates the method, the experiments only evaluate rendering quality. The paper does not directly demonstrate whether the proposed components restore anchor locality or improve geometric correctness. Quantitative comparisons of anchor-offset distributions and geometric consistency before and after optimization are necessary to justify the central claim.
  • For the dynamic-aware locality correction, the paper estimates the dynamicity from the relative magnitude of dynamic and static latent features. However, these magnitudes do not necessarily reflect physical motion, which requires more validation and justification.
  • The evaluation is incomplete. The paper should compare with more sparse-view dynamic approaches like 4D-Scaffold-GS [1], SparseCam4D [2], Sparse4DGS [3], and MonoFusion [4]. The geometric accuracy, computational cost, and robustness under different sparse views also need to be compared.
  • The paper uses depths from DA3 to supervise the rendered depth and achieves large improvement (see Table 2). However, since the method is based on LocalDyGS, it is hard to tell if the improvement is mostly from depth estimation or the designed modules. Therefore, it is necessary to add DA3 supervision for LocalDyGS as a comparison.
  • Inconsistency in equations. The equation for anchor splitting in Appendix Eq. 18 appears inconsistent with Eq. 12-13, which requires clarification.

[1] Cho, Woong Oh, et al. "4D scaffold Gaussian splatting with dynamic-aware anchor growing for efficient and high-fidelity dynamic scene reconstruction." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 5. 2026.

[2] Pan, Weihong, et al. "SparseCam4D: Spatio-Temporally Consistent 4D Reconstruction from Sparse Cameras." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

[3] Hu, Dongdong, et al. "Sparse4dgs: Flow-geometry assisted 4d gaussian splatting for dynamic sparse view synthesis." Proceedings of the 33rd ACM International Conference on Multimedia. 2025.

[4] Wang, Zihan, et al. "Monofusion: Sparse-view 4d reconstruction via monocular fusion." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.