- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents S3-SLAM, a training-free monocular 3DGS SLAM system that combines UniDepthV2 for single-frame metric depth with VGGT for multi-view-consistent but scale-free geometry. The main idea is to fuse the two depth sources using a spatial-frequency decomposition with online band-wise weighting. The system further uses persistent semantic object landmarks to estimate short-term scale corrections and to support loop closure and submap correction. The method is evaluated on TUM RGB-D, ScanNet, and 7-Scenes, with additional ablations, failure analysis, rendering results, and runtime measurements.
Critical strengths
-
The paper provides a relatively thorough empirical analysis of the proposed system.
-
The paper is generally clearly written and the system components are easy to identify.
Critical weaknesses
-
The overall methodological novelty is limited. Combining multiple pretrained foundation models with complementary outputs is not, by itself, a novel contribution. The system primarily integrates UniDepthV2, VGGT, Grounding DINO/SAM2 with 3DGS. Most of these foundation models and their roles in this work are well established in NeRF and 3DGS-based SLAMs. The paper does not convincingly demonstrate that their particular combination constitutes a substantial methodological advance.
-
The proposed frequency-based fusion is technically quite incremental. The method decomposes depth maps into multiple spatial-frequency bands and performs a weighted interpolation between the two depth predictions in each band. The main additional component is an online heuristic for selecting these weights from temporal reprojection consistency. This is a relatively modest extension of standard multiscale fusion ideas. More importantly, the paper does not compare against sufficiently strong simple alternatives, such as fixed band weights, a single global mixing weight, confidence-based fusion, or other multiscale fusion schemes. Therefore, it is unclear whether the proposed optimization itself provides meaningful novelty.
-
The semantic component does not actually introduce a new metric cue. The paper explicitly states that semantics provide correspondence rather than an independent source of metric depth. The scale correction is computed from the ratio between previously accumulated map depth and the current monocular depth prediction. In principle, the same measurement could be constructed using generic persistent tracks, instance tracks, or other stable correspondences without semantic labels. The paper does not include such baselines, so the necessity of semantic information itself is not established.
-
The semantic ablation is confounded. The “No semantics” setting simultaneously removes multiple functions of the semantic subsystem, including scale correction and loop closure support. As a result, the reported degradation cannot be attributed specifically to semantic scale correction. A more informative evaluation would separately ablate semantic scale correction, semantic loop-closure proposal, instance identity, and generic non-semantic persistent correspondences.
-
The complete SLAM system is not sufficiently robust. On ScanNet, the method exhibits very large trajectory errors on several sequences and completely fails on one sequence. These are not minor degradations. The appendix further identifies a structural failure mode in which VGGT motion estimates become highly unreliable under weak parallax, while the pose-estimation branch does not contain an equivalent robustness mechanism. Once tracking diverges, loop closure may also fail to recover the trajectory. This significantly weakens the claim that the proposed combination yields geometrically consistent monocular SLAM.
-
The experimental evidence for the central frequency-fusion contribution is weak. The main ablation of frequency-dependent fusion is reported only on a single TUM sequence. Removing the fusion component results in only a relatively small change in trajectory and depth metrics compared with removing semantics or loop closure.
The method also does not consistently outperform substantially simpler or existing systems. On 7-Scenes, the proposed method is not consistently better than VGGT-SLAM, HI-SLAM2, or SING3R-SLAM. On ScanNet, it is substantially worse than strong monocular baselines on several sequences. Under the depth-evaluation protocol used by Splat-SLAM, its average depth error is also slightly worse. Given the much greater system complexity, the empirical benefit is not convincing.
-
The semantic loop-closure mechanism itself appears insufficiently reliable. The reported loop-closure evaluation is limited to two TUM sequences, and the semantic version achieves only moderate precision. Since incorrect loop closures can be particularly damaging in SLAM, the paper should provide a more extensive evaluation across datasets and sequences, including the effect of false closures on final trajectory quality.
-
The computational cost is very high relative to the demonstrated gains. Tracking requires several seconds per frame on an A6000, with additional mapping and semantic-processing cost. The full pipeline depends on several large pretrained models and iterative optimization stages. This would be less concerning if the method provided a clear accuracy or robustness advantage, but the current results do not demonstrate such an advantage.
-
The paper does not sufficiently separate contribution from backbone capability. Because the system relies heavily on strong pretrained models, it is difficult to determine how much of the performance comes from the proposed method versus simply using UniDepthV2 and VGGT together. In particular, stronger baselines that use the same foundation models with substantially simpler fusion and tracking strategies are missing. Without these controls, the incremental value of the proposed algorithm is difficult to assess.
Besides, the core assumption is the depth quality of VGGT is not so good and therefore requires another foundation model for compensation. However, as widely known, there are a lot of geometry foundation models today that have much better depth quality than vggt or even UniDepthV2. The practical value of this work is not fully evaluated in today's background.
Questions for authors
See the weaknesses.
4 comments
Outperforming simpler and existing systems
We haven't reproduced VGGT-SLAM, HI-SLAM2, or SING3R-SLAM ourselves beyond the 7-Scenes comparison already in the paper, and we're not claiming to beat them uniformly there or on ScanNet. We'd rather say that plainly than argue around it.
Semantic loop closure reliability and false closures
We built the false-closure stress test you're asking for. We fabricate a loop-closure candidate between two frames with no genuine revisit, verified against ground-truth poses, and give it a deliberately wrong relative pose. The gate doesn't reject it, it fires at the same confidence and drift values a real high-drift revisit would, and Sim(3) ATE roughly doubles for the rest of the sequence with no later closure recovering it. The submap-rigid correction does bound the damage to the one affected submap at the moment it happens, but it doesn't heal over time, later real closures just build on top of the now-wrong baseline instead of correcting it. We've added this to the appendix next to the existing precision-recall table.
Computational cost and backbone separation
We ran the matched-backbone control directly, plain SplaTAM with raw UniDepthV2, no VGGT, semantics, or loop closure. It beats our full system on every metric on the more static fr1/desk, and our full system only pulls ahead on the more dynamic fr1/desk2. So our added machinery earns its cost specifically as motion gets harder to track, not uniformly, and we're not claiming an efficiency advantage on top of that. We don't have a new cost number to offer here.
Newer geometry foundation models
We did some small-scale testing with the newer foundation model VGGT-Omega, but found it not make a substantial difference for our use case. However, we have not benchmarked against other new backbones than UniDepthV2 and VGGT in this round. We're noting that as a limitation of our current approach. It should be noted though that methods such as MapAnything are somewhat similar to VGGT.
Dear reviewer Z8VJ,
Thank you for your careful review and helpful feedback. As the discussion period draws to a close, we would appreciate your thoughts on whether our additional experiments and clarifications address your concerns. We would be happy to answer any remaining questions.
I appreciate the authors' efforts in the rebuttal. However, after reviewing the rebuttal and the revised manuscript, I find that the new results acknowledged several of my original concerns, while the fundamental issues remain unresolved.
First, my concern regarding methodological novelty remains. The additional experiments cannot address the fundamental lack of methodological contribution, which is just performance comparison but not methodological declaration. Moreover, the newly added backbone-only comparison uses the relatively old SplaTAM baseline. Even so, unmodified SplaTAM with raw UniDepthV2 outperforms the full system on all reported metrics on fr1/desk and achieves substantially lower depth error on fr1/desk2.
The new ablations further raise concerns about the actual contributions of the proposed components. Semantic scale correction alone provides only marginal improvements over disabling semantics, while loop-closure proposals alone achieve better SE(3) ATE than the full system. The online frequency-based fusion optimizer also fails to consistently outperform simpler fixed or global-scalar alternatives. Furthermore, substantial tracking degradation on three ScanNet scenes and a complete tracking failure on another remain unresolved. The verification gate also fails to reject the injected false loop closure, resulting in persistent trajectory degradation.
Finally, although VGGT is incorporated into the system, the evaluation still lacks systematic comparisons with newer and stronger geometry foundation models. The claimed but excluded simple test with just VGGT-Omega does not address this concern.
Overall, the fundamental concerns regarding methodological novelty, empirical advantages, and robustness remain. Therefore, I maintain my original rating, which is in the consensus of other reviewers. It's encouraged for the authors to carefully revise the work in the next submission.
We thank the reviewer for their thorough analysis and comments. We address them in the following text, as well as in the paper.
Methodological novelty
Fair on its own, combining pretrained foundation models isn't novel by itself. We've added a paragraph to related work distinguishing our design from the two closest systems, ConfidentSplat and MyGO-Splat. ConfidentSplat fuses depth per pixel through confidence weighting, MyGO-Splat through closed-loop geometric feedback. We fuse per spatial-frequency band instead, and use persistent semantic object identity rather than pixel confidence for scale correction. We also built the confidence-weighted fusion baseline ConfidentSplat's design implies and measured it directly, it's dataset-dependent, best on TUM, worst on ScanNet.
Frequency fusion baselines and evidence scope
We ran the simple alternatives you're asking for, fixed band weights, a single global mixing weight, confidence-based fusion, and a frozen ground-truth-fitted curve, against our online optimizer. On TUM, the frozen curve matches or beats the online optimizer on every rendering and depth metric, and the online optimizer's only edge is Sim(3) ATE. We extended this to ScanNet too, since the original ablation was TUM-only. The ranking flips there, confidence-based fusion goes from best on TUM to worst on ScanNet, and our shipped optimizer goes from mid-table on TUM to best-or-tied on ScanNet. So the online mechanism's value over simpler alternatives is real, but it's dataset-dependent, not a clean uniform win, and we say so directly in the appendix now.
Semantic necessity and generic correspondence
We built exactly the baseline you're describing. Instead of semantic object identity, we track generic persistent correspondences using the same grid-based point tracking VGGT already computes, grouped by fixed image-space cell rather than object, feeding the identical downstream scale-correction and loop-closure pipeline. On ScanNet it underperforms the real semantic subsystem on every metric. We measured the same thing on TUM too, semantic tracking wins there as well. The mechanism is a fixed grid cell tracks image region, not object, so its content silently changes identity as the camera moves. Semantic tracking keeps true object correspondence across viewpoint change, which is what the scale-correction measurement actually needs.
Confounded semantic ablation
We decomposed "No semantics" into scale correction alone and loop-closure proposals alone, with SE(3) ATE and fitted scale added alongside Sim(3) ATE and Depth L1. Scale correction alone tracks the fully-disabled row closely. Loop-closure proposals alone recover almost all the way to the full system. So the semantic benefit comes almost entirely from the loop-closure proposals, not the scale correction. On instance identity specifically, that's what the generic-correspondence baseline above already isolates, region identity versus object identity, so we didn't add a fourth redundant arm for it.
SLAM robustness on ScanNet
We traced scenes 0000, 0207, and 0181 to one shared mechanism, VGGT's motion estimate degenerating under weak parallax. The motion-derived scale ratio spikes 12 to 24x on 0207, 142x on 0000, and up to 710x on 0181. On 0181, tracking holds through frame 320, then ATE goes from 5.69 m to 25.6 m across frames 360 to 392 and reaches 17.23 m by frame 744, with no loop closure firing to recover it. We also found 0181's failure is seed-dependent, not deterministic, a second seed on the same config reaches frame 812 cleanly at 0.48 m. We don't have a fix for this yet. What's new here is a characterized mechanism instead of an unexplained failure, plus evidence it isn't fixed to the scene.