acceptodds
Under review as a conference paper at ICLR 2027

FS-GStream: Distribution-Adaptive Motion Masking and Optimization for Flicker-Suppressed 3D Gaussian Streaming

Abstract

The online reconstruction of dynamic scenes from multi-view video streams enables fast free-viewpoint video synthesis, but existing streaming 3DGS methods often suffer from temporal flickering with geometry degradation. Temporal flickering partly arises from the difficulty of distinguishing true scene motion from static background during per-frame optimization, leading to unnecessary changes in static scene representations even when they do not change. In this paper, we propose FS-GStream, distribution-adaptive motion masking and optimization for flicker-suppressed 3D Gaussian streaming, which can be easily integrated as a generalizable plug-and-play module with two-stage streaming 3D Gaussian reconstruction models. Our FS-GStream significantly improves temporal consistency through two main components: Distribution-Adaptive Motion Masking (DAMM) and Mask-Guided Optimization (MGO). The DAMM first estimates a frame-wise motion threshold from the estimated distribution of log-transformed Gaussian translation magnitudes, which stably separates static and dynamic Gaussians in a statistical manner without manually setting a fixed displacement threshold. Guided by the masks that distinguish static and dynamic Gaussians from the DAMM, the MGO jointly refines the geometry and appearance of dynamic Gaussians while preserving static geometry and selectively adapting static appearance, and restricts densification and pruning to dynamic Gaussians. Extensive experiments on state-of-the-art two-stage streaming baselines demonstrate the generalizability of our plug-and-play design, which consistently alleviates temporal flickering and significantly improves qualitative and quantitative performance in highly dynamic scenes}.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Temporal Consistency, Online Dynamic Scene Reconstruction, 3D Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_q8XcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes FS-Gstream to address flicker and background degradation in two-stage streaming Gaussian reconstruction. DAMM estimates a frame-specific threshold from a KDE of log-transformed Gaussian translation magnitudes, and MGO uses the resulting masks to regulate motion application, parameter optimization, densification and pruning. Dynamic geometry and appearance are refined, while static geometry is protected and selected static appearance parameters remain adaptable.

Critical strengths
  1. This work proposes a plug-and-play module, which can be applied to some sotas directly, making it widely applicable.
  2. Table 1 shows lower Ewarp and higher PSNR for all twelve backbone–dataset pairs. These improvements should be credited.
  3. The ablations experiments are substantial, supporting a genuine quality and stability contribution beyond changing initialization and low-level Gaussian operations.
Critical weaknesses

W1. Missing comparisons with the closest alternatives is my main concern.

Table 1 mainly compares three backbones with and without FS-GStream, alongside IGS. These results demonstrate useful gains, but do not establish an advantage over existing selective-update and anti-flicker methods. In particular, CPOStream (AAAI 2026) and Yun et al.'s observation-restoration method (SIGGRAPH 2025) address closely related objectives and are cited but not evaluated. Comparisons under matched protocols, including a shared backbone where applicable, are needed to assess reconstruction quality, temporal consistency and computational cost. The existing ablations cannot substitute for these comparisons.

W2. The reliability of the translation-based separation is not established in important boundary cases.

The paper already supplies an EMA fallback and acknowledges dependence on the backbone's motion estimates. What remains unclear is its cold-start behavior, fallback frequency and parameterization, degenerate-KDE handling, and motion fidelity in these regimes. More specifically, Section 3.3 assumes that the leftmost prominent low-translation mode represents static displacement noise and that a subsequent valley provides a meaningful boundary. Slow true motion overlapping noise, poorly separated distributions, and changes with negligible center translation are not directly examined. Eq. (5) gates rotation using translation alone, while Eq. (6) freezes geometry for static-classified primitives; an anisotropic Gaussian can require an orientation change without center displacement.

W3. Tables 3–4 compare fixed thresholds, but do not isolate DAMM's benefit over another adaptive selector under the same MGO. Runtime attribution is also unclear: Table 7 shows increased runtime when the module is added under identical settings, so Table 1's speedups cannot be attributed to DAMM/MGO alone.

Questions for authors

Q1. Under matched streaming protocols, how does FS-GStream compare with CPOStream and, on a shared compatible backbone, Yun et al.'s observation-restoration method in reconstruction quality, temporal consistency and computational cost?

Q2. Does translation-based masking preserve slow motion and rotation with negligible center displacement? Please provide a targeted example and clarify threshold initialization and EMA fallback behavior when no valid peak/valley is detected.

Q3. With MGO fixed, how do alternative adaptive or flow-based masks affect final reconstruction and temporal consistency, including selector cost? Please separate module overhead from FastGS implementation effects and assess longer-term stability and dynamic-region fidelity.

5 comments

Thank you for your valuable review comments. We will address each of your concerns here. Below, we respond to each comment and will incorporate the corresponding clarifications and revisions into the final manuscript. If there is anything we haven't explained clearly, please don't hesitate to let us know.

Response to W1 and Q1 (Comparisons with CPOStream and Yun et al.)

Thank you for identifying these closely related methods. We provide direct comparisons with CPOStream and Yun et al.’s observation-restoration method. Since an official CPOStream implementation was unavailable when we conducted the experiments, we reimplemented it using the hyperparameters reported in its paper and the same initial Gaussian set as our method. As Yun et al. provide public code only for their 3DGStream-based implementation, we use their official code and restrict this comparison to 3DGStream as the shared backbone.

Comparison with CPOStream. We compare our CPOStream reproduction with ReCon-GS + FS-GStream. As shown in the table below, our method achieves better overall and dynamic-region PSNR, SSIM, and LPIPS, together with lower temporal warping error, than this reproduction on all four datasets.

DatasetMethodE_warp↓Dynamic PSNR↑Dynamic SSIM↑Dynamic LPIPS↓Overall PSNR↑Overall SSIM↑Overall LPIPS↓Payload (MB/f)↓Train (sec)↓
SelfCapReCon-GS + Ours0.01425.170.7980.20425.370.8370.2446.444.1
CPOStream0.02722.090.7060.34024.860.8150.2854.034.2
VRUReCon-GS + Ours0.02524.430.8480.11427.220.8890.1816.422.3
CPOStream0.03418.770.5910.40326.000.8710.2164.012.6
Meet RoomReCon-GS + Ours0.00632.240.9420.07732.510.9630.1611.512.1
CPOStream0.00929.220.9050.12532.090.9600.1693.991.9
N3DVReCon-GS + Ours0.00833.250.9440.11031.830.9470.1332.643.2
CPOStream0.01131.930.9310.14331.710.9460.1354.013.4

Comparison with Yun et al. on 3DGStream. Using 3DGStream as the shared backbone, our method outperforms Yun et al. in all three dynamic-region quality metrics on every dataset. It also achieves lower temporal warping error on SelfCap, VRU, and N3DV, and the same value on Meet Room. Overall image quality is also better except for PSNR and SSIM on Meet Room. Our evaluated pipeline has lower training time and payload across all four datasets.

DatasetMethodE_warp↓Dynamic PSNR↑Dynamic SSIM↑Dynamic LPIPS↓Overall PSNR↑Overall SSIM↑Overall LPIPS↓Payload (MB/f)↓Train (sec)↓
SelfCap3DGStream + Ours0.01425.040.8000.19925.360.8330.2526.004.4
Yun et al.0.01922.450.7090.33224.270.8180.2868.5110.3
VRU3DGStream + Ours0.02624.290.8390.12627.210.8860.1848.202.7
Yun et al.0.02921.190.6920.31326.790.8730.2218.806.2
Meet Room3DGStream + Ours0.00728.980.8840.13931.620.9550.1724.332.8
Yun et al.0.00728.680.8820.15332.010.9560.1898.044.4
N3DV3DGStream + Ours0.00731.890.9270.14231.500.9440.1374.416.4
Yun et al.0.00928.930.9010.16130.860.9430.1438.138.2

Response to W2 and Q2 (Slow Motion, Rotation, and EMA Fallback)

Slow motion and rotation. DAMM uses the displacement of each Gaussian’s center. An object rotating around a fixed center can still move its Gaussian centers, allowing DAMM to detect the motion when these displacements are distinguishable from static noise. However, very slow motion or rotation expressed mainly through orientation changes with negligible Gaussian-center displacement can be missed. We will explicitly discuss this failure mode as a limitation.

Threshold initialization and EMA fallback. Before EMA initialization, if a valid peak is found but no valid valley is detected, we use τ0=exp⁡(xpeak+2h)\tau_0=\exp(x_{\mathrm{peak}}+2h), where hh is the selected peak’s left HWHM (half width at half maximum) in log space. If no left half-maximum crossing is found, the search stops at the lower KDE evaluation boundary. If no peak passes the prominence criterion before EMA initialization, we instead set τ=0\tau=0 and treat all Gaussians with nonzero displacement as dynamic. The factor of 2 is based on median valley offsets of 2.21/1.89/2.26 HWHM for ReCon-GS/HiCoM/3DGStream, with leave-one-dataset-out estimates of 1.75–2.50. Neither of these initialization branches was used in the evaluated runs.

The first valid valley initializes the EMA in displacement units, using τ=exp⁡(xvalley)−ϵ\tau=\exp(x_{\mathrm{valley}})-\epsilon. Each new valid threshold receives weight 0.9 and the previous EMA receives weight 0.1. Once initialized, frames without a valid valley reuse the stored EMA without updating it. Fallback rates for ReCon-GS, HiCoM, and 3DGStream were 11.5%, 0.7%, and 25.5%, with maximum consecutive durations of 13, 3, and 26 frames, respectively.

We acknowledge that these implementation details were not fully specified in the original manuscript. We will clarify the threshold initialization, EMA update rule, and fallback conditions, and include the fallback statistics in the revised manuscript.

Targeted evaluation of slow motion. We compare our method with the “Only MGO” variant (all-update) from our submitted ablation study, both using ReCon-GS. This all-update variant disables DAMM and optimizes every Gaussian. We report PSNR and LPIPS within the same SAM3 dynamic masks.

We evaluate two separate groupings. First, we separate frames with and without a valid valley using our method’s DAMM logs. Second, we rank each scene’s frames by the mean UniMatch flow magnitude between consecutive GT images within the dynamic mask, divide them into five equally sized groups, and report the slowest and fastest groups. Both methods use identical frame groups. Results are averaged within each scene and then across all 13 scenes with equal scene weights.

As shown below, the slowest group exhibits larger dynamic-region PSNR and LPIPS gaps from all-update than the fastest group. The gaps are also larger on frames without a valid valley than on frames with a valley. These results are consistent with the risk of suppressing genuine slow motion when its displacement overlaps static noise. We will explicitly discuss this limitation.

FramesEstimated motion in GT videos (px/frame)Dynamic PSNR (ours / all-update)Dynamic LPIPS (ours / all-update)
With a valid valley1.4630.41 / 30.470.117 / 0.107
Without a valid valley0.7630.70 / 30.850.117 / 0.102
Slowest fifth0.6630.94 / 31.250.114 / 0.096
Fastest fifth2.3429.88 / 29.950.123 / 0.116

Response to W3 and Q3 (Alternative Masks, Runtime, and Longer-Sequence Evaluation)

We provide a controlled mask comparison with ReCon-GS and MGO fixed, together with separate runtime analysis and a 1,000-frame evaluation.

Alternative masks and selector cost. With ReCon-GS and MGO fixed, we compare DAMM with a UniMatch-based mask using a one-pixel flow threshold and a CPOStream-derived mask. DAMM outperforms the UniMatch-based mask in overall and dynamic-region PSNR, SSIM, and LPIPS on all four datasets. Temporal warping error is lower on SelfCap, the same on VRU and Meet Room, and slightly higher on N3DV.

The CPOStream-derived mask achieves better image quality in several settings but allows a much larger fraction of Gaussians to be optimized, leading to larger payloads. Its tendency toward better image quality but higher temporal warping error is similar to the “Only MGO” variant in our submitted ablation study, which allows all Gaussians to be optimized at each frame. DAMM achieves lower temporal warping error and smaller payloads than the CPOStream-derived mask on all four datasets. The DAMM-based pipeline also has the lowest selector cost and per-frame training time among the evaluated alternatives.

DatasetMethodE_warp ↓Dynamic PSNR ↑Dynamic SSIM ↑Dynamic LPIPS ↓Overall PSNR ↑Overall SSIM ↑Overall LPIPS ↓Dynamic ratio (%)Payload (MB/f) ↓Train (sec) ↓Selector cost (s/frame) ↓
SelfCapMGO + DAMM (Ours)0.01425.170.7980.20425.370.8370.24413.556.444.10.049
MGO + UniMatch (1-pixel threshold)0.01524.500.7770.21025.340.8340.24718.937.729.35.635
MGO + CPOStream-derived mask0.03224.450.8150.18424.620.8360.23998.1441.298.64.593
VRUMGO + DAMM (Ours)0.02524.430.8480.11427.220.8890.18127.176.422.30.047
MGO + UniMatch (1-pixel threshold)0.02523.930.8360.12826.030.8680.20028.686.393.61.502
MGO + CPOStream-derived mask0.03625.420.8740.09427.690.9020.16499.3424.264.22.052
Meet RoomMGO + DAMM (Ours)0.00632.240.9420.07732.510.9630.16115.281.512.10.045
MGO + UniMatch (1-pixel threshold)0.00626.670.8370.17930.890.9510.1767.291.243.11.011
MGO + CPOStream-derived mask0.00932.380.9440.07532.520.9630.16257.332.892.60.562
N3DVMGO + DAMM (Ours)0.00833.250.9440.11031.830.9470.1335.522.643.20.048
MGO + UniMatch (1-pixel threshold)0.00729.760.8830.16531.050.9390.1415.262.656.63.537
MGO + CPOStream-derived mask0.01333.750.9500.10232.060.9500.13075.718.324.51.410

Module overhead and implementation effects. We agree that the runtime gains in Table 1 should not be attributed to DAMM/MGO alone. With the implementation fixed, adding FS-GStream increases runtime in both Table 7 and our additional experiment with official ReCon-GS. The runtime gains in Table 1 therefore reflect the full FastGS-based system.

Longer-sequence evaluation and dynamic-region quality. We evaluate three configurations on the 1,000-frame Corgi sequence: (1) ReCon-GS trained using its official implementation, (2) the same official implementation with FS-GStream added, and (3) our FastGS-based ReCon-GS implementation with FS-GStream. Adding FS-GStream to official ReCon-GS improves overall and dynamic-region PSNR, SSIM, and LPIPS, while reducing temporal warping error and payload. Fig. 6 in Appendix A of the revised manuscript illustrates the corresponding per-frame PSNR trends. FS-GStream maintains higher reconstruction quality in later frames and avoids the pronounced PSNR drops observed in the baseline. These trends support improved reconstruction stability over the extended sequence.

MethodImplementationE_warp ↓Overall PSNR ↑Overall SSIM ↑Overall LPIPS ↓Train (s/frame) ↓Payload (MB/frame) ↓Dynamic PSNR ↑Dynamic SSIM ↑Dynamic LPIPS ↓
ReCon-GS (baseline)Official0.04823.550.7290.3945.814.6621.000.6420.434
ReCon-GS + FS-GStreamOfficial0.02525.000.7800.3006.810.6022.860.7250.301
ReCon-GS + FS-GStreamFastGS-based0.02225.730.7920.2794.79.1824.290.7810.228
Reviewer_yWrTOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

FS-GStream is a plug-and-play framework for reducing temporal flicker in two-stage, optimization-based Gaussian streaming pipelines. It introduces Distribution-Adaptive Motion Masking (DAMM), which estimates a frame-specific static–dynamic threshold from the KDE of log-transformed Gaussian translation magnitudes, and Mask-Guided Optimization (MGO), which controls parameter updates, densification, and pruning according to the resulting masks. Dynamic Gaussians undergo geometry and appearance refinement, while static geometry is preserved and selected static appearance parameters remain adaptable. Experiments on three streaming backbones and four datasets report improvements in reconstruction quality and temporal consistency.

Critical strengths
  1. The reported improvements in reconstruction quality and temporal consistency are substantial across the evaluated backbones.

Applying FS-GStream to 3DGStream, HiCoM, and ReCon-GS produces improvements across multiple datasets. The paper reports an average PSNR gain of approximately 1.52 dB and a 37.3% reduction in temporal warping error. The gains are particularly notable on challenging dynamic scenes, supporting the practical usefulness of the proposed module within the evaluated two-stage pipelines.

The controlled ReCon-GS experiment using identical FastGS initialization and low-level operations also supports improvements beyond initialization or implementation changes alone. This is useful evidence that the proposed masking and optimization policies contribute to reconstruction quality.

  1. The method reuses existing motion estimates to construct adaptive masks without introducing an additional motion-estimation network.

DAMM operates on the Gaussian displacements already estimated during motion alignment. It therefore avoids requiring an additional optical-flow or segmentation network for mask construction. Selecting the displacement threshold from each frame’s distribution also reduces the need to manually choose a fixed displacement threshold for different datasets.

The appendix provides comparisons against several fixed thresholds and shows higher mask IoU and F1 using the same adaptive procedure across datasets. This supports the motivation for distribution-dependent threshold selection, although it does not establish robustness in all motion regimes or superior mask accuracy to optical-flow-based alternatives.

Critical weaknesses
  1. The novelty and superiority relative to the most relevant existing methods are not sufficiently established.

Despite targeting flicker suppression, the paper does not directly compare against MGStream, DASS, or CPOStream. These methods already distinguish static and dynamic Gaussians, and MGStream and CPOStream specifically address flicker caused by unnecessary updates to static content. The current experiments establish improvements over the selected backbones, but do not establish an advantage over existing flicker-suppression approaches. CPOStream is particularly relevant because it also avoids external optical flow when deciding which Gaussians to freeze or optimize.

Motion gating and selective static–dynamic optimization are established principles. For example, QUEEN gates position residuals, while MoRGS weights Gaussian updates using motion confidence. MGO’s more specific distinction lies in preserving static geometry while selectively updating static appearance and restricting densification and pruning. However, the paper does not sufficiently isolate the technical advance and benefit of this particular combination. The selective static appearance update also does not provide consistent gains: on SelfCap, removing it leaves PSNR essentially unchanged while improving temporal error and payload.

Comparisons with relevant online approaches such as MoRGS and QUEEN are also missing. Their optimization structures differ from the two-stage backbones considered here, so direct plug-and-play integration may require adaptation. Nevertheless, standalone comparisons and a clearer discussion of applicability are needed. The demonstrated generality currently remains limited to the selected family of two-stage pipelines.

  1. The robustness of DAMM’s separation assumption and the benefits of the background-preservation policy require stronger validation.

DAMM assumes that low-displacement optimization noise can be separated from actual motion through the shape of the displacement distribution. Very slow motion may overlap substantially with static displacement noise, and log transformation and KDE do not inherently resolve this ambiguity. The paper specifies an EMA fallback when no suitable peak or valley is found, but does not separately evaluate slow-motion or poorly separated distributions. Reusing historical thresholds does not, by itself, establish reliable classification in the current frame.

Classification errors can propagate through several subsequent operations. If a genuinely moving Gaussian is classified as static, motion gating cancels its displacement, and geometry refinement and densification are restricted. Consequently, lower temporal error should be assessed together with motion fidelity to determine whether the method preserves true motion rather than suppressing it. Dedicated evaluations of this failure mode would strengthen the robustness claims.

The pruning policy also changes the representation-budget constraint. Unlike count-based pruning, the proposed policy protects static Gaussians and removes dynamic or newly added Gaussians only below an opacity threshold, potentially retaining more Gaussians. The pruning ablation improves reconstruction quality and temporal consistency while increasing payload. Comparisons under matched Gaussian-count or transmission budgets are therefore needed to distinguish the benefit of selective background protection from that of retaining a larger representation.

Questions for authors
  1. Could you provide direct comparisons with MGStream, DASS, and CPOStream under matched initialization, input views, optimization budgets, and temporal-consistency metrics? What evidence establishes an advantage over these existing flicker-suppression approaches?

  2. Could you compare with relevant online methods such as MoRGS and QUEEN, and clarify which structural requirements prevent direct integration? How broadly should the plug-and-play claim be interpreted beyond the three evaluated two-stage backbones?

  3. How does DAMM behave when genuine motion is very slow or its displacement distribution overlaps with static displacement noise? Could you evaluate unimodal or poorly separated distributions and report both motion fidelity and reconstruction quality?

  4. How is the EMA fallback initialized when no valid threshold has yet been detected, and how does it behave during extended periods without a suitable valley?

  5. What is the substantive novelty of MGO beyond existing static freezing, dynamic optimization, residual gating, and error-based selection strategies? In particular, when is selective static appearance adaptation beneficial, given the SelfCap ablation?

  6. Could you compare the proposed pruning policy with count-based pruning under matched Gaussian-count or payload budgets? How much of the improvement remains when both methods have comparable representation resources?

8 comments
AuthorsOpenReview ↗

Response to W1 and Q1 (Comparison with CPOStream)

We sincerely thank the reviewer for the thoughtful feedback and valuable observations. Below, we respond to each comment and will incorporate the corresponding clarifications and revisions into the final manuscript. If there is anything we haven't explained clearly, please don't hesitate to let us know.

Response to W1 and Q1 (Comparison with CPOStream)

Thank you for identifying these closely related methods. Official implementations of MGStream, DASS, and CPOStream were unavailable when we conducted the experiments. We prioritized reimplementing CPOStream (AAAI 2026), a closely related method that also suppresses flicker by preventing unnecessary updates to static Gaussians. Our reproduction follows the hyperparameters reported in its paper and uses the same initial Gaussian set as our method.

Comparison with CPOStream. We compare our CPOStream reproduction with ReCon-GS + FS-GStream. As shown in the table below, our method achieves better overall and dynamic-region PSNR, SSIM, and LPIPS, together with lower temporal warping error, than this reproduction on all four datasets.

DatasetMethodE_warp↓Dynamic PSNR↑Dynamic SSIM↑Dynamic LPIPS↓Overall PSNR↑Overall SSIM↑Overall LPIPS↓Payload (MB/f)↓Train (sec)↓
SelfCapReCon-GS + Ours0.01425.170.7980.20425.370.8370.2446.444.1
CPOStream0.02722.090.7060.34024.860.8150.2854.034.2
VRUReCon-GS + Ours0.02524.430.8480.11427.220.8890.1816.422.3
CPOStream0.03418.770.5910.40326.000.8710.2164.012.6
Meet RoomReCon-GS + Ours0.00632.240.9420.07732.510.9630.1611.512.1
CPOStream0.00929.220.9050.12532.090.9600.1693.991.9
N3DVReCon-GS + Ours0.00833.250.9440.11031.830.9470.1332.643.2
CPOStream0.01131.930.9310.14331.710.9460.1354.013.4
AuthorsOpenReview ↗

Response to W1 and Q2 (Comparison with MoRGS and QUEEN)

Response to W1 and Q2 (Comparison with MoRGS and QUEEN)

Comparison with MoRGS and QUEEN. We evaluate MoRGS and QUEEN using their official implementations. For this comparison, our method uses the same dense input point cloud as the baselines, obtained through 4DGaussians [1] preprocessing, instead of the sparse point cloud used in our main experiments. QUEEN retains its additional monocular-depth-based point augmentation.

Our method outperforms MoRGS in all three dynamic-region quality metrics and achieves lower temporal warping error on all four datasets, without external optical-flow or segmentation supervision during reconstruction. It also has smaller payloads and shorter per-frame training times on every dataset. While MoRGS achieves better overall image-quality metrics on VRU and lower overall LPIPS on Meet Room and N3DV, our method has smaller payloads and shorter per-frame training times on every dataset.

QUEEN focuses on efficient residual compression and does not explicitly target flicker suppression. Our method achieves lower temporal warping error on all four datasets and outperforms QUEEN in all three dynamic-region quality metrics on SelfCap, VRU, and Meet Room. While QUEEN achieves better dynamic-region SSIM and LPIPS and overall image quality on N3DV, as well as lower overall LPIPS on Meet Room, our method achieves better overall image quality on SelfCap and VRU. Our methodline also trains faster except on Meet Room, although its payload is larger except on SelfCap.

[1] 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

DatasetMethodE_warp ↓Dynamic PSNR ↑Dynamic SSIM ↑Dynamic LPIPS ↓Overall PSNR ↑Overall SSIM ↑Overall LPIPS ↓Payload (MB/frame) ↓Train (s/frame) ↓
SelfCapMoRGS0.02623.050.7200.32624.550.8200.2678.826.4
QUEEN0.03624.210.7570.28924.680.7940.3447.186.5
Ours (dense init)0.01325.130.7950.20825.540.8410.2415.363.7
VRUMoRGS0.02923.470.7980.19427.240.9000.1499.174.6
QUEEN0.04623.210.7860.20425.950.8730.1995.245.3
Ours (dense init)0.02424.420.8480.11327.200.8900.1706.632.1
Meet RoomMoRGS0.01129.800.9130.10931.940.9600.1481.933.7
QUEEN0.01031.070.9290.09432.290.9630.1510.311.6
Ours (dense init.)0.00732.130.9400.08232.410.9630.1551.192.1
N3DVMoRGS0.01332.050.9340.12030.960.9460.1274.785.1
QUEEN0.01433.550.9470.11132.580.9520.1290.654.5
Ours (dense init)0.00833.620.9450.11332.390.9490.1351.653.1

Scope of plug-and-play integration. FS-GStream is designed to integrate with streaming 3DGS pipelines that separate motion alignment from subsequent Gaussian refinement. It only requires per-Gaussian displacements from the motion-alignment stage and control over parameter updates, densification, and pruning during refinement. We demonstrate this flexibility across three backbones with different motion representations: 3DGStream, HiCoM, and ReCon-GS. Other approaches, such as QUEEN and MoRGS, employ different optimization procedures, and extending FS-GStream to these methods would require architecture-specific adaptations. Nevertheless, FS-GStream does not depend on a particular motion representation, allowing it to be integrated into different streaming backbones that provide the required optimization interface.

Response to W2 and Q3 (Slow-Motion Analysis)

Behavior under slow motion. When genuine motion overlaps with static displacement noise, DAMM may find no valid valley and reuse the stored EMA threshold. This threshold can classify some slowly moving Gaussians as static and suppress their displacement. For ReCon-GS, HiCoM, and 3DGStream, fallback rates are 11.5%, 0.7%, and 25.5%, with maximum consecutive durations of 13, 3, and 26 frames, respectively.

Motion fidelity and reconstruction quality. To examine whether DAMM suppresses genuine motion, we compare our method with an all-update variant, both using the ReCon-GS backbone. The all-update variant corresponds to “Only MGO” (all-update) in our submitted ablation study, which disables DAMM and optimizes every Gaussian. Using UniMatch, we estimate optical flow from both GT and rendered videos. Within the SAM3 dynamic mask, we measure the rendered flow component along the estimated GT flow direction, normalized by the GT flow magnitude. A motion ratio of 1 indicates matching motion along that direction. We also report PSNR and LPIPS within the same dynamic mask.

We evaluate two separate groupings: frames with versus without a valid valley, identified from our method’s DAMM logs, and frames grouped by motion estimated from the GT videos. For each frame, we use UniMatch to estimate optical flow between consecutive GT images and average its magnitude within the SAM3 dynamic mask. This measures image displacement in pixels per frame. We sort each scene’s frames by this value, divide them into five equally sized groups, and report the slowest and fastest groups. Both methods use the same masks and frame groups. Results are averaged within each scene and then across all 13 scenes with equal scene weights.

As shown in the table below, frames without a valid valley exhibit less estimated motion in the GT videos and a larger motion-ratio gap than frames with a valley. The gap is also larger in the slowest group than in the fastest group. These results indicate that DAMM can under-represent genuine slow motion, even when the average dynamic-region PSNR difference is small. We will explicitly discuss this failure mode as a limitation.

FramesEstimated motion in GT videos (px/frame)Motion ratio (ours / all-update)Dynamic PSNR (ours / all-update)Dynamic LPIPS (ours / all-update)
With a valid valley1.460.885 / 0.93030.41 / 30.470.117 / 0.107
Without a valid valley0.760.748 / 0.94730.70 / 30.850.117 / 0.102
Slowest fifth0.660.807 / 0.93230.94 / 31.250.114 / 0.096
Fastest fifth2.340.910 / 0.93829.88 / 29.950.123 / 0.116

Response to Q4 (EMA Initialization and Fallback)

Before EMA initialization, if a valid peak is found but no valid valley is detected, we use τ0=exp⁡(xpeak+2h)\tau_0=\exp(x_{\mathrm{peak}}+2h), where hh is the selected peak’s left HWHM (half width at half maximum) in log space. If no left half-maximum crossing is found, the search stops at the lower KDE evaluation boundary. If no peak passes the prominence criterion before EMA initialization, we instead set τ=0\tau=0 and treat all Gaussians with nonzero displacement as dynamic. The factor of 2 is based on median valley offsets of 2.21/1.89/2.26 HWHM for ReCon-GS/HiCoM/3DGStream, with leave-one-dataset-out estimates of 1.75–2.50. Neither of these initialization branches was used in the evaluated runs.

The first valid valley initializes the EMA in displacement units, using τ=exp⁡(xvalley)−ϵ\tau=\exp(x_{\mathrm{valley}})-\epsilon. Each new valid threshold receives weight 0.9 and the previous EMA receives weight 0.1. Once initialized, frames without a valid valley reuse the stored EMA without updating it. Fallback rates for ReCon-GS, HiCoM, and 3DGStream were 11.5%, 0.7%, and 25.5%, with maximum consecutive durations of 13, 3, and 26 frames, respectively.

For frames with valid valleys, we compared the static fractions obtained using the current EMA and a 40-frame-old EMA, each applied to the same current-frame displacements. The table reports the mean absolute difference from the static fraction obtained using the current valley threshold, in percentage points.

BackboneCurrent EMA40-frame-old EMA
ReCon-GS3.04.5
HiCoM11.612.9
3DGStream2.53.3

The 40-frame delay modestly increases the mean absolute difference in static fraction. This test measures agreement with the current valley threshold on frames with valid valleys and does not directly assess per-Gaussian classification accuracy during fallback.

We acknowledge that these implementation details were not fully specified in the original manuscript. We will clarify the threshold initialization, EMA update rule, and fallback conditions, and include the fallback statistics and this additional analysis in the revised manuscript.

Reviewer_rR7KOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies online 3D Gaussian Splatting (3DGS) reconstruction of dynamic scenes. When processing incoming video frame by frame, the model may mistake static background for moving objects. This can cause background positions to jitter and Gaussians to be repeatedly added and removed, ultimately leading to flickering and degraded image quality. The authors propose FS-GStream, a module that can be integrated into existing two-stage streaming reconstruction frameworks. It has two main components:

  • DAMM (Distribution-Adaptive Motion Masking): It measures Gaussian displacements in each frame, applies a logarithmic transform, and estimates their distribution using kernel density estimation (KDE). It then estimates a motion threshold from the low-displacement peak and the valley that follows, separating static from dynamic Gaussians. If a reliable threshold cannot be found for a frame, it falls back to an exponential moving average of previously valid thresholds.
  • MGO (Mask-Guided Optimization): Dynamic Gaussians can update both their geometry and appearance. Static Gaussians retain fixed geometry, but their appearance can be selectively updated based on image residuals. The method also restricts densification and pruning to dynamic and newly added Gaussians, helping preserve the static background.

The authors integrate the module into 3DGStream, HiCoM, and ReCon-GS and evaluate it on four datasets: SelfCap, VRU, Meet Room, and N3DV. They report improvements in both PSNR and temporal warping error across all 12 dataset–backbone comparisons in the table, with an average gain of about 1.52 dB in PSNR and a 37.3% reduction in Ewarp. Component ablations also suggest that combining motion masking with mask-guided optimization improves temporal stability.

Critical strengths
  1. The problem is clearly defined, and the proposed mechanisms directly address it. Unnecessary updates to the static background, excessive densification, and global pruning can all cause flickering. The method addresses these issues through motion-based filtering, protection of static geometry, and restricted addition and removal of Gaussians.

  2. It is easy to integrate into existing systems. The method uses per-Gaussian displacement information already available in two-stage frameworks and does not require an additional pretrained optical-flow network. This can be useful for streaming systems that aim to reduce reliance on external models and annotations.

Critical weaknesses
  1. DAMM assumes that static-scene noise and true motion can be separated in the displacement distribution, and uses the distribution’s peaks and valleys to set the threshold. If the two overlap, there is no clear valley, or the motion distribution changes abruptly, the mask may become unstable. The paper uses a fallback based on previous thresholds, but does not sufficiently report how often this fallback is needed or show cases where it fails.
  2. The reported average per-frame training time for FS-GStream is approximately 2.1–6.4 seconds. This is lower than the baseline methods, but still substantial.
  3. Flickering is not completely eliminated. Fairly severe flickering remains in sparsely observed regions, although this is understandable.
Questions for authors

I have no major concerns about this manuscript. I consider it an engineering-oriented contribution with practical value, though its novelty is incremental. It does not propose a new scene representation; instead, it combines adaptive motion thresholding with static-region preservation in a plug-and-play module. Its motivation is clear, the ablation results generally support the design choices, and it avoids reliance on external optical-flow models.

1 comment

We sincerely thank the reviewer for the positive assessment and for recognizing the practical value of FS-GStream. Below, we respond to each comment and will incorporate the corresponding clarifications and revisions into the final manuscript. If there is anything we haven't explained clearly, please don't hesitate to let us know.

Response to W1 (Fallback Frequency and Failure Cases)

Fallback behavior and frequency Once the EMA is initialized, frames without a valid valley reuse the stored EMA threshold without updating it. The EMA is maintained in displacement units and updated only from valid valley thresholds, assigning weight 0.9 to the new threshold and 0.1 to the previous EMA. Fallback rates were 11.5%, 0.7%, and 25.5% for ReCon-GS, HiCoM, and 3DGStream, with maximum consecutive durations of 13, 3, and 26 frames, respectively.

Analysis of fallback frames and slow motion. We conduct an additional evaluation using ReCon-GS to examine reconstruction quality in difficult motion regimes. We compare our method with the “Only MGO” variant (all-update) from our submitted ablation study. This variant disables DAMM and optimizes every Gaussian, providing a comparison without DAMM’s motion gating and static–dynamic masking. We report PSNR and LPIPS within the same SAM3 dynamic masks for both methods.

We evaluate two separate groupings. First, we separate frames with and without a valid valley using our method’s DAMM logs. Second, we estimate optical flow between consecutive GT images using UniMatch and compute its mean magnitude within the SAM3 dynamic mask for each frame. This measures image displacement in pixels per frame. We rank each scene’s frames by this value, divide them into five equally sized groups, and report the slowest and fastest groups. Both methods are evaluated using identical masks and frame groups. Results are averaged within each scene and then across all 13 scenes with equal scene weights.

Frames without a valid valley, where EMA fallback is used, exhibit less estimated motion in the GT videos and larger quality gaps from all-update than frames with a valid valley. The slowest group also exhibits larger gaps than the fastest group. These results are consistent with the risk that DAMM suppresses genuine slow motion when its displacement overlaps static noise.

FramesEstimated motion in GT videos (px/frame)Dynamic PSNR (ours / all-update)Dynamic LPIPS (ours / all-update)
With a valid valley1.4630.41 / 30.470.117 / 0.107
Without a valid valley0.7630.70 / 30.850.117 / 0.102
Slowest fifth0.6630.94 / 31.250.114 / 0.096
Fastest fifth2.3429.88 / 29.950.123 / 0.116

We acknowledge that the original manuscript did not sufficiently report fallback frequency or examine these failure cases. We will include the above statistics and analyses and explicitly discuss the limitations under overlapping motion distributions in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.