- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes S2Tok for online Gaussian reconstruction from uncalibrated image streams. It maintains persistent scene tokens and learns when to add new tokens using geometric novelty supervision and self-critical sequence training (SCST). The current tokens are jointly decoded into a Gaussian scene. Experiments on four datasets report competitive rendering quality with substantially fewer Gaussians.
Critical strengths
-
The persistent token representation supports online reconstruction with compact Gaussian outputs. On 50-view DL3DV, the method uses 66K Gaussians at 81 ms per frame.
-
The experiments cover four datasets, including cross-dataset evaluation on Mip-NeRF 360 and ScanNet++. The method achieves higher PSNR than the evaluated feed-forward streaming baselines on DL3DV, RealEstate10K, and Mip-NeRF 360.
Critical weaknesses
-
The admission objective does not directly optimize the quality–memory tradeoff. The SCST reward measures reconstruction error, while the KL term keeps the policy close to a geometric novelty oracle (Eqs. 29–32). Neither term explicitly accounts for token count or update cost. Compactness is therefore controlled indirectly by the oracle and admission threshold. The benefit of this policy optimization over oracle distillation remains limited: at the same 66K Gaussian budget, SCST improves Final/Intermediate PSNR by only 0.10/0.08 dB (Table 3).
-
The state update remains expensive as the scene grows. Eq. 4 only appends tokens; admitted tokens are never merged or removed. Meanwhile, global attention, scene attention, and the hierarchical decoder operate on the accumulated state (Eqs. 1, 2, and 6), with quadratic attention computation in the number of scene tokens. Selective admission slows growth but cannot reduce an already expanded state, so per-frame computation increases with the accumulated token count.
-
The frozen appearance memory is insufficiently justified. Appendix D.2 fixes each token's color feature at admission, while its scene feature and decoded Gaussians continue to change. Later observations can refine appearance through the main feature path, but cannot update the high-frequency color bank. The paper does not explain how this asymmetry affects reconstruction across updates. In the 50-view setting, Final PSNR drops from 19.11 to 18.00 relative to Intermediate on DL3DV, and from 22.63 to 20.54 on ScanNet++ (Tables 1 and 7); the contribution of the color-memory design to this degradation is unresolved.
-
The comparison with streaming Gaussian reconstruction is incomplete. StreamGS (Li et al., 2025) is discussed but not evaluated. It directly addresses online Gaussian reconstruction and redundancy reduction, making it an important baseline for the claimed quality–efficiency tradeoff. Comparisons with pointmap predictors through point splatting do not establish this advantage.
Questions for authors
- Could the authors clarify how the reconstruction reward and oracle regularization balance quality against state size?
- Could the authors characterize how per-frame runtime and memory scale with the number of retained tokens?
- Could the authors explain the rationale for freezing the color bank while updating scene tokens?
- Could the authors provide a comparison with StreamGS (Li et al., 2025) under the same protoco, reporting rendering quality, Gaussian count, and reconstruction time?
Thanks for the response. I will keep my original rating.