acceptodds
Under review as a conference paper at ICLR 2027

LifeStreamGS: Lifecycle-Aware Streaming 3D Gaussian Splatting for Unposed Videos

Abstract

Streaming feed-forward 3D Gaussian Splatting (3DGS) reconstructs renderable scenes from unposed video streams. In this setup, each frame's Gaussians are predicted without future observations and remain fixed as new frames arrive, causing prediction errors to accumulate. Dense per-pixel predictions further amplify this problem by adding redundant Gaussians, which increasingly degrade rendering quality over long sequences. To this end, we propose LifeStreamGS, a lifecycle-aware framework that treats dense Gaussian predictions as candidates and controls their placement, retention, and contribution to rendering. These decisions are implemented through merge-guided Gaussian proposal, existence-aware candidate validation, and view- and time-aware opacity aging, reducing redundancy and the impact of accumulated errors. Furthermore, for long videos, we extend this framework through chunk-wise streaming, reconstructing overlapping windows and aligning their reconstructions into a shared coordinate system. Experiments show competitive novel-view synthesis quality with lower runtime and GPU memory usage than the evaluated optimization-based baselines in most tested settings, while supporting 500-view reconstruction and online video stabilization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Streaming, Feed-forward Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ShuFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LifeStreamGS, to support streaming 3D recosntruction with compact gaussian splating from unposed inputs. LifeStreamGS is based on a causal feed-forward framework. It combines merge-guided candidate sampling, gating and opacity weighting to reduce the gaussian numbers and uses overlapping chunks to support long sequences reconstruction.

Critical strengths
  1. The pipeline looks reasonable in general.
  2. The experiment results, especillay for the 500 frames inputs looks impressive. 3.The supp include the code and video, making it more convicing.
Critical weaknesses
  1. The novelty of the proposed lifecycle components should be clarified. Frame-wise streaming is largely enabled by the inherited causal-attention backbone. Gaussian reduction relies on learned patch-local sampling, with up to 32 candidates per patch, followed by existence gating. The choice of 32 requires further justification and sensitivity analysis. More importantly, it remains unclear whether this mechanism removes cross-frame redundancy or primarily sparsifies individual frames. For example, does it suppress candidates on surfaces already represented by earlier observations? The dense-prediction-and-selection design should also be distinguished from EcoSplat, which already supports budget-controlled Gaussian selection.
  2. The contribution to long-sequence scalability requires stronger validation. Long-video reconstruction relies on overlapping chunks, which closely resemble existing chunk-based reconstruction strategies. A chunked AnySplat baseline using VGGT-Long-style alignment would also help assess whether its OOM failures can be mitigated through chunking, while acknowledging the distinction between chunk-level processing and frame-wise causal inference.
  3. The related work discussion is incomplete. Relevant methods, including YoNoSplat (ICLR 2026), Multi-view Pyramid Transformer (CVPR 2026), and C3G (CVPR 2026), should be discussed in relation to the proposed approach.
  4. High resolution scalability remains unverified. The experiments use inputs resized to a width of 518 pixels. Since image-token processing and dense attribute prediction remain necessary, reducing the final Gaussian count alone does not establish efficient high-resolution reconstruction. Please report reconstruction quality, processing speed, and peak GPU memory at higher input resolutions.
  5. Minor: qualitative comparisons should show errors more clearly. The renderings in Figure 1 are too small to assess detailed differences from ground truth. Enlarged crops and error maps with a shared color scale would improve the comparison. Blur and floaters remain visible in the supplementary videos; representative examples should be shown and discussed.
Questions for authors

Please see the weakness.

8 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/2)

Q1: Candidate-count sensitivity and difference from EcoSplat. Thank you for pointing this out. Table 1 below reports our sensitivity study with P=16/32/64P=16/32/64. P=32P=32 gives the best overall quality on DL3DV and RE10K. A smaller capacity limits candidate coverage, while increasing it to 64 gives no further gain. Therefore, 32 is a stable empirical choice under the current protocol, rather than a theoretical constant.

Table 1. Sensitivity to the maximum candidate count PP per patch. Bold values indicate the best results, and values marked with †\dagger indicate the second-best results.

PPDL3DV PSNR ↑\uparrowDL3DV SSIM ↑\uparrowDL3DV LPIPS ↓\downarrowRE10K PSNR ↑\uparrowRE10K SSIM ↑\uparrowRE10K LPIPS ↓\downarrow
1622.0780.673$\dagger$0.29825.6460.7820.256
3222.5150.7000.26626.1120.8360.202
6422.282$\dagger$0.6540.283$\dagger$25.712$\dagger$0.788$\dagger$0.249$\dagger$

The current learned patch-local sampling does not explicitly match the current candidates with the global Gaussian bank. Its direct role is mainly to sparsify the current frame and control the causal admission of new Gaussians. It does not guarantee the removal of all cross-frame duplication. A surface represented by a previous frame does not mean that a Gaussian from a later observation is always redundant. An existing Gaussian may only be accurate near its source view. At a distant view, it may be affected by occlusion, pose error, or geometry inconsistency. A later observation may therefore provide a complementary representation. Future observations are not available at admission time, so it is also difficult to fully judge the long-term value of a Gaussian. LifeStreamGS therefore uses aging to adjust the contribution of accepted Gaussians for different queries. This reduces rendering interference caused by cross-frame accumulation.

EcoSplat uses a user-specified global Gaussian count. In offline multi-view prediction, it ranks candidates by importance and adjusts their parameters to provide a controllable quality-budget trade-off. In contrast, LifeStreamGS performs streaming reconstruction and cannot make a global budget allocation. In LifeStreamGS, PP is only the maximum candidate capacity of each patch. It is not the final scene budget. The existence gate decides whether a current candidate enters the causal stream, but it does not guarantee an exact global count. Aging continues to manage the later contribution of accepted Gaussians. EcoSplat asks which candidates to keep under a given global budget. We instead ask how streaming Gaussians should be admitted and how they should contribute to later rendering.

Q2: Matched-chunk comparison with an offline baseline. Thank you for the suggestion. Existing results show that WorldMirror consistently outperforms AnySplat. We therefore adapt WorldMirror as the stronger representative offline baseline for the matched-chunk comparison. Table 2 below compares WorldMirror with LifeStreamGS using the same chunk size, overlap, and VGGT-Long-style alignment. WorldMirror gives higher quality on the evaluated metrics. It can be viewed as a non-causal reference upper bound when full information can be jointly accessed. LifeStreamGS is also faster across the four evaluated settings. However, WorldMirror must wait for and jointly process a complete chunk. Its current predictions can use future frames within that chunk. It therefore does not meet the requirement of frame-wise causal streaming inference. In contrast, LifeStreamGS keeps causal information dependence within each chunk.

Table 2. Quality and efficiency comparison with WorldMirror under matched chunking. Bold values indicate the best results, and values marked with †\dagger indicate the second-best results.

DatasetViewsMethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowTime (s) ↓\downarrowPeak Mem. (GB) ↓\downarrow
DL3DV64LifeStreamGS22.151$\dagger$0.701$\dagger$0.264$\dagger$22.476.11
DL3DV64WorldMirror + same chunk23.1300.7270.21845.34$\dagger$42.62$\dagger$
RE10K64LifeStreamGS22.651$\dagger$0.756$\dagger$0.243$\dagger$22.356.01
RE10K64WorldMirror + same chunk24.1280.7700.21641.63$\dagger$40.89$\dagger$
7-Scenes500LifeStreamGS23.997$\dagger$0.768$\dagger$0.272$\dagger$198.236.01
7-Scenes500WorldMirror + same chunk24.5770.8050.246231.73$\dagger$40.28$\dagger$
Tanks & Temples500LifeStreamGS21.509$\dagger$0.678$\dagger$0.269$\dagger$197.146.02
Tanks & Temples500WorldMirror + same chunk22.1020.7010.210223.45$\dagger$41.76$\dagger$
AuthorsOpenReview ↗

Official Comment by Authors (2/2)

Q3: Additional related work. Thank you for the suggestion. YoNoSplat predicts per-view local Gaussians and camera poses from an arbitrary number of jointly available images and aggregates them into a global representation, whereas LifeStreamGS restricts information to past frames and manages Gaussian admission and rendering contributions online. Multi-view Pyramid Transformer uses local-to-global inter-view and fine-to-coarse intra-view hierarchies to jointly reconstruct tens to hundreds of views in a single forward pass; it targets efficient global multi-view interaction rather than causal streaming accumulation. C3G uses learnable tokens to aggregate sparse multi-view features and decode an explicitly compact representation with about 2K Gaussians, while LifeStreamGS uses per-frame causal proposal and validation together with query-dependent opacity aging for a growing video stream. We will add these distinctions to the revised paper.

Q4: Scope of high-resolution reconstruction. Thank you for the observation. Our goal is not to solve high-resolution streaming feed-forward reconstruction. To our knowledge, this setting remains largely underexplored; existing feed-forward studies, both offline and online, primarily focus on reconstruction quality at moderate input resolutions. LifeStreamGS addresses a different and complementary bottleneck: as a video stream grows, dense and fixed Gaussian predictions from successive frames accumulate redundancy and errors, increasing representation size and interfering with rendering. Increasing the input resolution alone does not resolve this temporal accumulation. Without controlling the Gaussian lifecycle, a higher resolution would instead produce more candidates per frame and further amplify computation, memory use, and representation redundancy. High-resolution streaming reconstruction is an important future direction, and we will explore combining lifecycle management with tiled or hierarchical token processing and multi-scale attribute prediction.

Q5: Clearer qualitative comparisons and temporal artifacts. Thank you for the suggestion. In the revision, we will redesign Figure 1, enlarge the local crops in the qualitative comparison, and add pixel-level error maps with the same color scale.

Thank you for the observation about the supplementary video. We acknowledge local temporal jumps, including blur and floaters, in the video. These issues are not caused by camera pose estimation. Aging changes the rendering weights of different Gaussians based on the current query view and temporal relation. When neighboring frames or chunks have remaining geometry and pose differences, the dominant Gaussians can switch as the query changes. This can cause local discontinuities in edges or appearance, which appear as visual jumps. Similar temporal jumps in ARTDECO can be observed at 0:24-0:27, 0:29-0:32, and 1:05-1:08 in the supplementary video. This may occur because ARTDECO optimizes all Gaussians at each frame, which can change the dominant Gaussians used for rendering. Without aging, inconsistent Gaussians may be blended and temporal jumps may appear weaker. However, this usually produces more blur and ghosting. This shows the current trade-off between per-frame sharpness and temporal consistency. The source of this issue is cross-frame inconsistency caused by remaining geometry errors in causal reconstruction. It is a common issue in current streaming reconstruction methods. We will study explicit temporal consistency constraints in future work.

AuthorsOpenReview ↗

Thank You for Your Review and Consideration of Our Revision

Thank you again for your thoughtful review and constructive comments. We have uploaded a revised version of the manuscript that incorporates the additional clarifications and experiments discussed in our rebuttal and follow-up responses. For ease of review, all revisions are highlighted in blue. We would be very grateful if you could take a moment to review these updates and, if appropriate, reconsider your evaluation. Thank you very much for your time and effort.

Reviewer_ma53OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses streaming feed-forward 3DGS from unposed video, where Gaussians are predicted causally and never revised; dense per-pixel prediction therefore piles up redundant Gaussians and errors that degrade quality on long sequences. Building on OF3GS, the paper proposes Gaussian lifecycle modeling: merge-guided proposal picks a few candidate locations per patch using learned weights, existence-aware validation uses a learned gate to decide which candidates are kept, and view- and time-aware opacity aging lowers opacity at render time according to viewing angle and temporal distance. Long videos are processed as overlapping chunks aligned by Sim(3) alignment. On DL3DV, RE10K, zero-shot NYUv2, and 500-view 7-Scenes / Tanks & Temples, compared against OF3GS, ARTDECO, OnTheFly-NVS and others, the method is competitive in quality with optimization-based online methods while being faster and lighter on GPU memory.

Critical strengths
  1. Strong results on a real failure mode. OF3GS, with the same backbone, falls from 21.836 to 17.190 dB on DL3DV from 5 to 64 views, whereas this method drops only 0.4 dB; it also completes 500 views, stays near 6 GB through 128 frames, and runs 2.8 to 8.6$\times$ faster than ARTDECO. Quality is close to the optimization-based online methods on all five datasets, at much lower cost where reported. This is the main reason I consider the paper a substantive contribution to online NVS.

  2. The evaluation is reasonably careful, e.g. the intrinsics control in Supp. Table 2.

Critical weaknesses
  1. The experiments do not show which component produces the gain, and the ablation skips the long-sequence setting the paper targets. In Table 3 (5 views only), proposal and validation without aging add only 0.23 dB over the variant with everything off, while adding aging, a hand-set rule, brings the gain to 1.07 dB; all three datasets show this pattern. Two variants are missing, aging alone and OF3GS plus aging, and there is no component ablation at 64, 128 or 500 views. This weakens two of the three stated contributions.

  2. Long-sequence and efficiency gains cannot be separated from chunking. Both methods share a backbone, chunking comes from VGGT-Long and was not given to the baselines (Supp. Sec. 2.1), and LifeStreamGS itself fails without it (Table 1), so the OF3GS failure there and the memory/speed margins cannot be credited to lifecycle modeling alone.

Questions for authors
  1. In Table 3, proposal + validation without aging gives only +0.23 dB over the variant with everything off, while aging accounts for most of the improvement. Could you report two additional variants: aging applied directly to the dense predictions, and OF3GS with your aging and chunk alignment added? I would like to understand how much of the gain is actually due to the learned components.

  2. The ablation is only run at 5 views, but the paper is motivated by long sequences. Can you provide the same component ablation at 64 and 128 views, with the number of Gaussians, runtime, and memory?

  3. Was chunking already enabled in the 64- and 128-view experiments? If so, what do the numbers look like without chunking, and how much of the efficiency advantage over OF3GS remains once OF3GS is given the same chunking?

7 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/3)

Q1: Contributions of learned admission and aging. Thank you for the suggestion. Table 1 below reports the first control experiment, where we apply aging directly to dense predictions. Compared with Everything off, dense aging-only improves PSNR, which shows that aging itself is effective. However, its PSNR is lower than Full on all three datasets. More importantly, aging-only still keeps about 1.34M Gaussians. This is 5.43, 5.50, and 5.44 times the number used by Full on the three datasets. This shows that the main role of learned proposal and validation is causal admission and strong representation compression. Under the same aging setting, they also further improve rendering quality. The gain of the full model therefore cannot be explained only by hand-designed aging.

Table 1. Lifecycle attribution with 5 input views. Bold values indicate the best results.

VariantDL3DV PSNR ↑\uparrowDL3DV SSIM ↑\uparrowDL3DV LPIPS ↓\downarrowDL3DV Num. GS ↓\downarrowRE10K PSNR ↑\uparrowRE10K SSIM ↑\uparrowRE10K LPIPS ↓\downarrowRE10K Num. GS ↓\downarrowNYUv2 PSNR ↑\uparrowNYUv2 SSIM ↑\uparrowNYUv2 LPIPS ↓\downarrowNYUv2 Num. GS ↓\downarrow
Everything off21.4460.6430.3471,341,62024.6740.7750.2411,341,62024.6340.7200.3331,341,620
Aging-only dense22.0360.6410.2901,341,62025.6270.8110.2251,341,62024.8260.7040.3271,341,620
Ours Full22.5150.7000.266247,26226.1120.8360.202243,90525.2670.7470.299246,657

In the second control experiment, reported in Table 2 below, we also apply the same aging and chunk alignment to OF3GS. OF3GS + aging + chunk still performs worse than the full LifeStreamGS. Simply adding the aging rule to the baseline is therefore not enough to obtain our improvement. This comparison further shows that the final result comes from the combination of learned admission and later aging, rather than aging alone. This supports the role of our full Gaussian lifecycle management.

Table 2. Comparison with OF3GS using the same aging and chunking at 500 input views.

Variant7-Scenes PSNR ↑\uparrow7-Scenes SSIM ↑\uparrow7-Scenes LPIPS ↓\downarrowT&T PSNR ↑\uparrowT&T SSIM ↑\uparrowT&T LPIPS ↓\downarrow
OF3GS + aging + chunk21.6440.7430.29420.8080.6550.269
LifeStreamGS + aging + chunk23.9970.7680.27221.5090.6780.269
AuthorsOpenReview ↗

Official Comment by Authors (2/3)

Q2: Long-sequence lifecycle component ablation. Thank you for the suggestion. To evaluate the lifecycle components directly in the long-sequence regime, we conduct a more challenging 500-view ablation rather than limiting the study to 64 or 128 views, as shown in Table 3 below. The full model gives the best quality and the lowest runtime on both 7-Scenes and Tanks & Temples. Removing proposal or validation greatly increases the number of Gaussians and lowers quality. Removing aging clearly hurts rendering quality while keeping almost the same number of Gaussians. This experiment shows that the three modules have separate roles in representation compression and contribution adjustment over long sequences. Their gains are not limited to the 5-view setting.

Table 3. Lifecycle component ablation with 500 input views.

Variant7-Scenes PSNR ↑\uparrow7-Scenes SSIM ↑\uparrow7-Scenes LPIPS ↓\downarrow7-Scenes Num. GS ↓\downarrow7-Scenes Time (s) ↓\downarrow7-Scenes Memory (GB) ↓\downarrowT&T PSNR ↑\uparrowT&T SSIM ↑\uparrowT&T LPIPS ↓\downarrowT&T Num. GS ↓\downarrowT&T Time (s) ↓\downarrowT&T Memory (GB) ↓\downarrow
Full23.9970.7680.27211,637,278198.236.0121.5090.6780.26910,672,798197.146.02
W/o all22.8370.6420.37372,490,992211.566.3820.3830.6020.32372,490,992209.436.29
W/o merge23.3740.7120.32359,272,282201.306.0220.8330.6150.30360,822,687205.296.11
W/o validation23.2340.7200.31447,341,056200.246.1720.9430.6310.29447,341,056200.356.21
W/o aging22.9320.6550.34711,637,278209.876.0320.5440.6290.31610,672,798203.426.03
Aging only23.1780.7090.34272,490,992201.656.2820.8430.6300.29872,490,992206.646.25
AuthorsOpenReview ↗

Official Comment by Authors (3/3)

Q3: Effect of chunking under a matched OF3GS setting. Thank you for the question. Our original 64/128-view experiments in the main paper use chunking by default. Table 4 below reports the corresponding ablation without chunking and the comparison with OF3GS under the same chunking and alignment pipeline. Without chunking, LifeStreamGS avoids the repeated computation introduced by overlapping chunks and therefore runs faster, but its peak memory increases substantially and its reconstruction quality clearly drops on both DL3DV and RE10K. This shows that chunking trades additional computation for bounded memory usage and better reconstruction quality over long sequences. It is not the only source of our efficiency advantage. When OF3GS uses exactly the same chunking and alignment pipeline, LifeStreamGS still gives 2.04-2.52 dB higher PSNR. It also achieves a 1.30–1.75× speedup and uses about 57%-70% less peak memory. Our quality and efficiency advantages over OF3GS therefore remain after controlling for the same chunking setting.

Table 4. Chunking ablation and matched-OF3GS comparison at 64 and 128 input views. Bold values indicate the best results, and values marked with †\dagger indicate the second-best results.

ViewsMethodDL3DV PSNR ↑\uparrowDL3DV SSIM ↑\uparrowDL3DV LPIPS ↓\downarrowDL3DV Time (s) ↓\downarrowDL3DV Mem. (GB) ↓\downarrowRE10K PSNR ↑\uparrowRE10K SSIM ↑\uparrowRE10K LPIPS ↓\downarrowRE10K Time (s) ↓\downarrowRE10K Mem. (GB) ↓\downarrow
64LifeStreamGS + chunk22.1510.7010.26422.47$\dagger$6.1122.6510.7560.24322.35$\dagger$6.01
64LifeStreamGS w/o chunk21.062$\dagger$0.658$\dagger$0.295$\dagger$16.5612.03$\dagger$21.287$\dagger$0.728$\dagger$0.271$\dagger$15.8611.97$\dagger$
64OF3GS + same chunk20.0200.6180.38539.2714.1320.1350.7250.27438.5315.82
128LifeStreamGS + chunk21.7870.6850.27238.29$\dagger$6.3822.6860.7570.24236.72$\dagger$6.33
128LifeStreamGS w/o chunk20.372$\dagger$0.659$\dagger$0.302$\dagger$25.2718.48$\dagger$21.568$\dagger$0.726$\dagger$0.271$\dagger$26.2717.39$\dagger$
128OF3GS + same chunk19.6720.6040.38949.6220.4520.6440.7070.29947.5919.21
Reviewer_JduhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

LifeStreamGS presents a feed-forward framework for reconstructing renderable 3D Gaussian scenes from unposed video streams. Building on a frozen StreamVGGT-based causal feature extractor and camera prediction components, it manages per-frame Gaussian predictions through patch-based candidate placement, existence-aware candidate selection, and view- and time-dependent opacity weighting. For long videos, it processes overlapping chunks and aligns their reconstructions using confidence-weighted similarity transformations. Experiments evaluate novel-view synthesis quality, runtime, and GPU memory usage, including reconstruction from 500-view sequences and an application to online video stabilization.

Critical strengths
  1. The method improves both representation efficiency and reconstruction quality through Gaussian placement and selection.

Rather than accumulating all dense per-frame predictions, the method selects candidate locations and suppresses unnecessary candidates to construct a more compact representation. Reducing the Gaussian count while also improving rendering quality is a meaningful practical benefit. In the five-view DL3DV ablation, the complete model reduces the Gaussian count from approximately 1.34 million to 247,000 compared with the variant without lifecycle modeling, while achieving a higher PSNR.

These benefits also appear in runtime and GPU memory usage. For example, on DL3DV with 64 input views, LifeStreamGS achieves a higher PSNR than ARTDECO while reducing cumulative processing time from 62.24 to 22.47 seconds and GPU memory usage from 7.51 to 6.11 GB. Although it does not achieve the highest reconstruction quality in every setting, it offers a useful balance between quality and computational cost for online reconstruction.

  1. The method produces perceptually strong renderings, and the ablation supports the usefulness of adjusting Gaussian contributions.

Modulating Gaussian opacity according to viewing direction and temporal distance is a reasonable strategy for reducing interference between inconsistent predictions. It favors observations that are more relevant to the requested rendering viewpoint. Although the weighting itself is a predefined rule rather than an explicitly learned reliability estimator, it can still be a useful rendering strategy.

The ablation shows that removing opacity aging reduces reconstruction quality without changing the Gaussian count. On five-view DL3DV, the difference is approximately 0.84 dB PSNR, supporting a contribution beyond representation compression alone. In the supplementary videos, LifeStreamGS also appears to preserve sharper details and better perceptual quality than ARTDECO. This visual quality is a positive aspect, considered separately from the concerns about camera trajectory stability.

Critical weaknesses
  1. The technical novelty relative to closely related work is not sufficiently established.

The basic causal processing capability comes from the pretrained StreamVGGT feature extractor and camera prediction components, which remain frozen. Moreover, StreamVGGT itself already discusses windowed streaming and alignment between reconstructed segments to address memory growth. LifeStreamGS also follows VGGT-Long for chunk alignment. Consequently, the novelty of the streaming mechanism itself appears limited, and the paper needs to more clearly distinguish its new contributions from the capabilities inherited from these components.

The Gaussian management modules also share important ideas with prior work. In particular, patch-wise heatmap-based primitive placement and confidence-based suppression are closely related to Off The Grid. There are differences in implementation and causal input processing, but the paper does not sufficiently explain why these differences constitute a substantive technical advance. A more explicit comparison is needed to clarify what merge-guided proposal and existence-aware validation add beyond adapting existing ideas to streaming inputs.

The online video stabilization application provides limited additional evidence of a distinct contribution. OF3GS already demonstrates the same type of application by smoothing the estimated camera trajectory and rendering novel views along it. A direct comparison is therefore needed to establish whether LifeStreamGS improves stabilization quality or temporal consistency.

  1. The experiments do not sufficiently isolate the necessity and additional benefits of the proposed lifecycle design.

The long-sequence evaluation does not compare against OF3GS, AnySplat, or WorldMirror equipped with the same chunk size, overlap, and alignment procedure. It is therefore difficult to determine how much of the long-sequence capability comes from lifecycle modeling rather than from applying chunking to control memory usage. StreamSplat is also a relevant online feed-forward method with static-scene NVS evaluations. Although its dynamic reconstruction and interpolation setting differs, results under a comparable setting, or a more specific explanation of why comparison is infeasible, would strengthen the evaluation.

The choice of Gaussian representation also requires further justification. C3G retains a substantially smaller reported Gaussian count even after test-time optimization. This motivates evaluating alternatives that combine a compact representation with short optimization and chunk alignment. Such comparisons should control for input conditions, processing time, and update latency to establish the advantages of the proposed placement and management strategy.

The effects of the individual design choices are insufficiently separated. The existence gate uses a threshold of 0.5 during training and inference, but the paper does not report how this threshold affects rendering quality and Gaussian count. Similarly, the opacity-aging ablation does not distinguish the contributions of view weighting, temporal decay, and the inference-time restriction to Gaussians from at most three neighboring input frames. This makes it difficult to assess whether the improvement comes from the proposed weighting strategy or primarily from restricting the contributing observations to a narrow temporal neighborhood.

Finally, although the supplementary videos appear perceptually sharper than ARTDECO, the camera motion seems to exhibit substantially larger jumps. Evaluating online reconstruction and video stabilization requires assessing camera pose accuracy and temporal stability alongside image quality. The current evaluation does not sufficiently explain this apparent trade-off.

Questions for authors
  1. What is the main technical advance in the streaming mechanism beyond using a frozen StreamVGGT-based causal model and existing windowed reconstruction and alignment strategies? How does the long-sequence design differ from the windowed streaming already discussed in StreamVGGT?

  2. How do merge-guided proposal and existence-aware validation differ substantively from Off The Grid’s heatmap-based primitive placement and confidence-based suppression? Which differences are essential for the reported improvements?

  3. Could you compare against OF3GS, AnySplat, and WorldMirror using the same chunk size, overlap, and alignment procedure? Would a compact alternative such as C3G, optionally followed by short optimization, offer a competitive quality–memory–runtime trade-off? Could StreamSplat also be evaluated under a comparable static-scene setting?

  4. What motivates the existence threshold of 0.5, and how sensitive are rendering quality and Gaussian count to this choice? Is there evidence that useful geometry is consistently retained across different scenes and sequence lengths?

  5. Could you separately ablate view-dependent weighting, temporal decay, and the inference-time temporal window? In particular, how much improvement remains when using the same three-frame restriction without the additional opacity weighting? Is there evidence that the weighting preferentially suppresses geometrically or photometrically unreliable Gaussians?

  6. When is each chunk aligned to the global reconstruction, and when does its output become available? Does global rendering require waiting for a complete chunk or overlap region? Please clarify the resulting update latency and availability of frame-wise outputs.

  7. The supplementary videos appear sharper than ARTDECO, but the camera motion seems less stable. Could you explain this observation and provide quantitative evaluations of camera pose accuracy and temporal stability? For video stabilization, how does LifeStreamGS compare directly with the corresponding OF3GS application?

8 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/4)

Q1: Technical advance beyond causal prediction and window alignment. Thank you for raising this question. StreamVGGT mainly handles streaming feature states and causal geometry prediction. Existing window-based methods extend reconstruction to long videos through overlapping windows and alignment. However, simply combining StreamVGGT with window alignment only allows continuous input processing and alignment of local results. It does not manage the Gaussians that remain in the scene and continue to contribute to rendering after decoding. Directly accumulating Gaussians from frames or chunks therefore causes redundancy and geometry errors to build up and affect later rendering. As shown in Fig. 4 of the main paper, OF3GS accumulates per-frame Gaussians without lifecycle management and performs substantially worse than LifeStreamGS as the input sequence grows, demonstrating the importance of managing accumulated Gaussian contributions. The technical contribution of LifeStreamGS is at this representation level. Proposal and validation control the admission of new Gaussians. Aging adjusts the later contribution of accepted Gaussians based on the query view and temporal relation. For long sequences, the same lifecycle mechanism continues to work on the aligned chunks.

Q2: Difference from Off The Grid. Thank you for pointing out this connection. We agree that proposal and validation share the basic ideas of patch-local placement and confidence-based suppression with Off The Grid. However, Off The Grid is designed for offline reconstruction, where multiple views can be accessed together to build a compact representation through global primitive allocation. If directly applied to an online setting, it can only sparsify newly predicted Gaussians in the current frame. This is because future frames are unavailable and past predictions cannot be revised. It cannot fully identify future cross-frame redundancy at admission time or adjust Gaussians that have already entered the scene. As a result, redundant or inconsistent contributions can still accumulate over time. LifeStreamGS redesigns proposal and validation as a causal admission mechanism for streaming reconstruction. It further applies aging to control the query-dependent contributions of accepted Gaussians. Together, these modules form a complete streaming lifecycle. The key distinction is the combination of redesigned causal admission and query-dependent contribution management after admission. The component ablation in Table 3 and the quality-efficiency comparison in Fig. 4 of the main paper support the effectiveness of this design.

AuthorsOpenReview ↗

Official Comment by Authors (2/4)

Q3: Matched chunking and additional baseline comparisons. Thank you for the suggestions. First, we compared OF3GS and WorldMirror using the same chunk size, overlap, and Sim(3)\mathrm{Sim}(3) alignment. Since WorldMirror consistently outperforms AnySplat in our original results, we use it as the stronger representative offline baseline for the matched-chunk comparison. As shown in Table 1 below, LifeStreamGS outperforms OF3GS on all datasets and sequence lengths. WorldMirror is an offline method that can use all frames within each chunk jointly, and it achieves higher reconstruction quality. However, LifeStreamGS is consistently faster and uses less peak GPU memory than both matched-chunk baselines.

Table 1. NVS quality and efficiency comparison under matched chunking at 64 and 500 input views. Bold values indicate the best results, and values marked with †\dagger indicate the second-best results.

DatasetViewsMethodPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowTime (s) ↓\downarrowPeak Mem. (GB) ↓\downarrow
DL3DV64LifeStreamGS22.151$\dagger$0.701$\dagger$0.264$\dagger$22.476.11
DL3DV64OF3GS + same chunk20.0200.6180.38539.27$\dagger$14.13$\dagger$
DL3DV64WorldMirror + same chunk23.1300.7270.21845.3442.62
RE10K64LifeStreamGS22.651$\dagger$0.756$\dagger$0.243$\dagger$22.356.01
RE10K64OF3GS + same chunk20.1350.7250.27438.53$\dagger$15.82$\dagger$
RE10K64WorldMirror + same chunk24.1280.7700.21641.6340.89
7-Scenes500LifeStreamGS23.997$\dagger$0.768$\dagger$0.272$\dagger$198.236.01
7-Scenes500OF3GS + same chunk20.8930.7050.324221.62$\dagger$14.76$\dagger$
7-Scenes500WorldMirror + same chunk24.5770.8050.246231.7340.28
Tanks & Temples500LifeStreamGS21.509$\dagger$0.678$\dagger$0.269$\dagger$197.146.02
Tanks & Temples500OF3GS + same chunk20.2920.6420.271219.27$\dagger$13.71$\dagger$
Tanks & Temples500WorldMirror + same chunk22.1020.7010.210223.4541.76

We also added StreamSplat experiments under a comparable static-scene setting. Table 2 below reports the results with 5 and 64 input views. Compared with the LifeStreamGS results in the main paper under the same datasets and view settings, StreamSplat gives lower overall reconstruction quality.

Table 2. StreamSplat results for static-scene NVS with 5 and 64 input views.

DatasetViewsPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrowTime (s) ↓\downarrowMemory (GB) ↓\downarrow
DL3DV519.3320.5270.4193.986.85
DL3DV6421.4390.6140.37035.2310.67
RE10K525.8350.8080.2934.256.10
RE10K6421.3590.7230.27832.349.45
NYUv2524.0640.6590.3673.659.28
NYUv26422.2420.6830.33534.289.74

Table 3 below reports the results with 500 input views. StreamSplat also gives lower reconstruction quality in this setting. This further supports the effectiveness of LifeStreamGS for causal static-scene reconstruction.

Table 3. StreamSplat results for static-scene NVS with 500 input views.

DatasetViewsPSNR ↑\uparrowSSIM ↑\uparrowLPIPS ↓\downarrow
7-Scenes50023.0310.7070.301
Tanks & Temples50020.4460.6140.290

Finally, a C3G-style compact representation is worth exploring, but without a dedicated causal compression, initialization, and optimization pipeline, we cannot rigorously guarantee either a fair comparison or that its offline compactness transfers to streaming. We expect a direct adaptation to perform poorly because C3G aggregates jointly available multi-view information offline, whereas causal decoding cannot access future observations and may therefore produce less accurate 3DGS parameters; its aggressive compression may also compromise fine-detail preservation even in offline reconstruction.

AuthorsOpenReview ↗

Official Comment by Authors (3/4)

Q4: Existence-threshold choice and sensitivity. Thank you for the suggestion. We use 0.5 because the existence confidence is produced by a sigmoid. A value of 0.5 corresponds to a zero logit, which is a standard decision threshold. Table 4 below reports our sensitivity study with thresholds from 0.2 to 0.9. When the threshold increases from 0.2 to 0.9, the average Gaussian counts on DL3DV and RE10K decrease by 6.4% and 3.5%, respectively. Therefore, 0.5 is not chosen to minimize the number of Gaussians. It is a stable trade-off between rendering quality and representation compactness.

Table 4. Sensitivity to the existence threshold.

ThresholdDL3DV PSNR ↑\uparrowDL3DV SSIM ↑\uparrowDL3DV LPIPS ↓\downarrowDL3DV Num. GS ↓\downarrowRE10K PSNR ↑\uparrowRE10K SSIM ↑\uparrowRE10K LPIPS ↓\downarrowRE10K Num. GS ↓\downarrow
0.222.3980.7000.267257,38226.0700.8370.202249,287
0.322.4010.7000.267249,02726.0490.8370.202245,832
0.522.5150.7000.266247,26226.1120.8360.202243,905
0.722.4460.7000.268243,82925.9310.8350.204241,229
0.922.3880.6980.272240,83225.3660.8290.212240,632

To address whether useful geometry is consistently retained across different scenes and sequence lengths, we first clarify that, in causal streaming reconstruction, whether a Gaussian represents "useful geometry" is not a fixed global property. It depends on the query view and time. A Gaussian can have accurate geometry and appearance near its source view. It may become less suitable for a distant query view because of occlusion, pose error, or geometry inconsistency. Future observations are not available when the Gaussian is generated. The method therefore cannot make a global decision about the long-term value of each Gaussian. We admit Gaussians based on the current evidence. Aging then adjusts the rendering contribution of accepted Gaussians for different queries, instead of permanently marking them as reliable or unreliable.

Q5: Separate ablation of opacity-aging components. Thank you for the suggestion. Table 5 below separately ablates view-dependent weighting, temporal decay, and the inference-time temporal window. The full model gives the best results on all metrics across the three datasets. This shows that the three parts are complementary. We also kept the same three-frame window while disabling the two additional weights. This Window-only variant performs worse than the full model. The gain therefore cannot be explained only by limiting the frames used for rendering. View-dependent weighting and temporal decay still provide consistent improvements.

Table 5. Ablation of the three components in opacity aging.

VariantDL3DV PSNR ↑\uparrowDL3DV SSIM ↑\uparrowDL3DV LPIPS ↓\downarrowRE10K PSNR ↑\uparrowRE10K SSIM ↑\uparrowRE10K LPIPS ↓\downarrowNYUv2 PSNR ↑\uparrowNYUv2 SSIM ↑\uparrowNYUv2 LPIPS ↓\downarrow
Ours Full22.5150.7000.26626.1120.8360.20225.2670.7470.299
No view decay22.3830.6960.27426.0280.8120.21725.0110.7210.315
No temporal decay22.1920.6880.27725.8750.8070.23124.9970.7100.332
No temporal window22.4330.6890.27225.9780.8100.22125.0620.7270.319
Window only21.9030.6780.28324.9330.7250.30924.6150.7100.325

To address whether aging preferentially suppresses geometrically or photometrically unreliable Gaussians, we clarify its design principle. Aging is designed to reduce rendering interference from redundant cross-frame Gaussians by adjusting their contributions according to source-query view and temporal relations. We did not design it to explicitly detect geometric or photometric unreliability and then age individual Gaussians accordingly. Since geometry predicted by current streaming reconstruction methods often contains residual errors, and the reliability of each Gaussian cannot be rigorously determined without future observations, we instead address the problem at rendering time: Gaussians that are less compatible with the current query view or temporally distant from its reference receive lower opacity. The ablation shows that this rendering-level adjustment reduces accumulated interference overall. Learning reliability-aware aging weights is an important future direction.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Revised Manuscript

Dear All Reviewers,

Thank you once again for your time and insightful feedback on our submission. As the discussion period is drawing to a close, we wanted to kindly follow up on our previous responses and the updated manuscript (with all changes highlighted in blue).

We have carefully addressed every concern raised with new experiments and detailed clarifications. We would truly appreciate it if you could take a brief look at our replies and let us know if any further questions remain. Thank you very much for your continued support and consideration!

AuthorsOpenReview ↗

Overview of Reviews and Subsequent Discussion

We thank the reviewers and the Area Chair for their time and constructive feedback. We have incorporated additional experiments and clarifications into the revised manuscript, with all revisions highlighted in blue.

Reviewer Jduh (raised the score to 3, Weak Acceptance) stated that the rebuttal had “resolved most of my main concerns.” They confirmed that the additional comparisons and ablations better isolate the lifecycle contribution and that the timing clarification resolves their latency question. Recognizing a “consistent quality–efficiency advantage over the closest online baseline,” they raised their score to 3.

Reviewer ma53 (maintained the score of 3, Weak Acceptance) explicitly confirmed that “the new ablation experiments answer my questions.” They acknowledged that LifeStreamGS retains its quality advantage over OF3GS under matched chunking and maintained their positive rating.

Reviewer ShuF (score 2, Weak Rejection) identified only two remaining concerns after our initial responses and additional experiments: the quality–efficiency trade-off relative to WorldMirror and the high-resolution limitation. We subsequently addressed both points, explicitly acknowledging WorldMirror’s higher fidelity and clarifying our quality–causality–efficiency trade-off, while adding high-resolution reconstruction to the limitations. We have not received further feedback on these responses.

Two reviewers now support acceptance, and we have responded directly to the remaining reviewer’s latest concerns. We hope that the additional evidence and subsequent discussion demonstrate the practical value of lifecycle-aware streaming reconstruction, and that the paper can be considered favorably in the final decision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.