acceptodds
Under review as a conference paper at ICLR 2027

: Semantic Scale Synthesis for Training-Free Metric Depth from Complementary Foundation Models in Monocular 3DGS SLAM

Abstract

Monocular depth foundation models can predict metric depth from individual images, but these predictions vary in scale and do not provide the multi-view geometric consistency required for estimation of motion while building a map. We introduce -SLAM, a training-free monocular 3D Gaussian Splatting SLAM system that synthesizes two complementary foundation models: UniDepthV2, which infers single-frame metric depth, and VGGT, which infers multi-view-consistent but scale-free geometry. We show that their depth errors are complementary across spatial depth frequencies, introducing an online method of frequency-based fusion that exploits this structure while decoupling the mapping and tracking steps of SLAM. We use persistent semantic object landmarks to further correct frame-to-frame drift on short timescales, and to provide long-term geometric consistency when revisiting previously observed regions. By evaluating this approach across a range of indoor SLAM benchmarks, we demonstrate recovery of consistent metric-scale depth and trajectory estimates.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Depth Foundation Models, Semantic SLAM, 3D Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Z8VJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents S3-SLAM, a training-free monocular 3DGS SLAM system that combines UniDepthV2 for single-frame metric depth with VGGT for multi-view-consistent but scale-free geometry. The main idea is to fuse the two depth sources using a spatial-frequency decomposition with online band-wise weighting. The system further uses persistent semantic object landmarks to estimate short-term scale corrections and to support loop closure and submap correction. The method is evaluated on TUM RGB-D, ScanNet, and 7-Scenes, with additional ablations, failure analysis, rendering results, and runtime measurements.

Critical strengths
  1. The paper provides a relatively thorough empirical analysis of the proposed system.

  2. The paper is generally clearly written and the system components are easy to identify.

Critical weaknesses
  1. The overall methodological novelty is limited. Combining multiple pretrained foundation models with complementary outputs is not, by itself, a novel contribution. The system primarily integrates UniDepthV2, VGGT, Grounding DINO/SAM2 with 3DGS. Most of these foundation models and their roles in this work are well established in NeRF and 3DGS-based SLAMs. The paper does not convincingly demonstrate that their particular combination constitutes a substantial methodological advance.

  2. The proposed frequency-based fusion is technically quite incremental. The method decomposes depth maps into multiple spatial-frequency bands and performs a weighted interpolation between the two depth predictions in each band. The main additional component is an online heuristic for selecting these weights from temporal reprojection consistency. This is a relatively modest extension of standard multiscale fusion ideas. More importantly, the paper does not compare against sufficiently strong simple alternatives, such as fixed band weights, a single global mixing weight, confidence-based fusion, or other multiscale fusion schemes. Therefore, it is unclear whether the proposed optimization itself provides meaningful novelty.

  3. The semantic component does not actually introduce a new metric cue. The paper explicitly states that semantics provide correspondence rather than an independent source of metric depth. The scale correction is computed from the ratio between previously accumulated map depth and the current monocular depth prediction. In principle, the same measurement could be constructed using generic persistent tracks, instance tracks, or other stable correspondences without semantic labels. The paper does not include such baselines, so the necessity of semantic information itself is not established.

  4. The semantic ablation is confounded. The “No semantics” setting simultaneously removes multiple functions of the semantic subsystem, including scale correction and loop closure support. As a result, the reported degradation cannot be attributed specifically to semantic scale correction. A more informative evaluation would separately ablate semantic scale correction, semantic loop-closure proposal, instance identity, and generic non-semantic persistent correspondences.

  5. The complete SLAM system is not sufficiently robust. On ScanNet, the method exhibits very large trajectory errors on several sequences and completely fails on one sequence. These are not minor degradations. The appendix further identifies a structural failure mode in which VGGT motion estimates become highly unreliable under weak parallax, while the pose-estimation branch does not contain an equivalent robustness mechanism. Once tracking diverges, loop closure may also fail to recover the trajectory. This significantly weakens the claim that the proposed combination yields geometrically consistent monocular SLAM.

  6. The experimental evidence for the central frequency-fusion contribution is weak. The main ablation of frequency-dependent fusion is reported only on a single TUM sequence. Removing the fusion component results in only a relatively small change in trajectory and depth metrics compared with removing semantics or loop closure.

    The method also does not consistently outperform substantially simpler or existing systems. On 7-Scenes, the proposed method is not consistently better than VGGT-SLAM, HI-SLAM2, or SING3R-SLAM. On ScanNet, it is substantially worse than strong monocular baselines on several sequences. Under the depth-evaluation protocol used by Splat-SLAM, its average depth error is also slightly worse. Given the much greater system complexity, the empirical benefit is not convincing.

  7. The semantic loop-closure mechanism itself appears insufficiently reliable. The reported loop-closure evaluation is limited to two TUM sequences, and the semantic version achieves only moderate precision. Since incorrect loop closures can be particularly damaging in SLAM, the paper should provide a more extensive evaluation across datasets and sequences, including the effect of false closures on final trajectory quality.

  8. The computational cost is very high relative to the demonstrated gains. Tracking requires several seconds per frame on an A6000, with additional mapping and semantic-processing cost. The full pipeline depends on several large pretrained models and iterative optimization stages. This would be less concerning if the method provided a clear accuracy or robustness advantage, but the current results do not demonstrate such an advantage.

  9. The paper does not sufficiently separate contribution from backbone capability. Because the system relies heavily on strong pretrained models, it is difficult to determine how much of the performance comes from the proposed method versus simply using UniDepthV2 and VGGT together. In particular, stronger baselines that use the same foundation models with substantially simpler fusion and tracking strategies are missing. Without these controls, the incremental value of the proposed algorithm is difficult to assess.

    Besides, the core assumption is the depth quality of VGGT is not so good and therefore requires another foundation model for compensation. However, as widely known, there are a lot of geometry foundation models today that have much better depth quality than vggt or even UniDepthV2. The practical value of this work is not fully evaluated in today's background.

Questions for authors

See the weaknesses.

4 comments

We thank the reviewer for their thorough analysis and comments. We address them in the following text, as well as in the paper.

Methodological novelty

Fair on its own, combining pretrained foundation models isn't novel by itself. We've added a paragraph to related work distinguishing our design from the two closest systems, ConfidentSplat and MyGO-Splat. ConfidentSplat fuses depth per pixel through confidence weighting, MyGO-Splat through closed-loop geometric feedback. We fuse per spatial-frequency band instead, and use persistent semantic object identity rather than pixel confidence for scale correction. We also built the confidence-weighted fusion baseline ConfidentSplat's design implies and measured it directly, it's dataset-dependent, best on TUM, worst on ScanNet.

Frequency fusion baselines and evidence scope

We ran the simple alternatives you're asking for, fixed band weights, a single global mixing weight, confidence-based fusion, and a frozen ground-truth-fitted curve, against our online optimizer. On TUM, the frozen curve matches or beats the online optimizer on every rendering and depth metric, and the online optimizer's only edge is Sim(3) ATE. We extended this to ScanNet too, since the original ablation was TUM-only. The ranking flips there, confidence-based fusion goes from best on TUM to worst on ScanNet, and our shipped optimizer goes from mid-table on TUM to best-or-tied on ScanNet. So the online mechanism's value over simpler alternatives is real, but it's dataset-dependent, not a clean uniform win, and we say so directly in the appendix now.

Semantic necessity and generic correspondence

We built exactly the baseline you're describing. Instead of semantic object identity, we track generic persistent correspondences using the same grid-based point tracking VGGT already computes, grouped by fixed image-space cell rather than object, feeding the identical downstream scale-correction and loop-closure pipeline. On ScanNet it underperforms the real semantic subsystem on every metric. We measured the same thing on TUM too, semantic tracking wins there as well. The mechanism is a fixed grid cell tracks image region, not object, so its content silently changes identity as the camera moves. Semantic tracking keeps true object correspondence across viewpoint change, which is what the scale-correction measurement actually needs.

Confounded semantic ablation

We decomposed "No semantics" into scale correction alone and loop-closure proposals alone, with SE(3) ATE and fitted scale added alongside Sim(3) ATE and Depth L1. Scale correction alone tracks the fully-disabled row closely. Loop-closure proposals alone recover almost all the way to the full system. So the semantic benefit comes almost entirely from the loop-closure proposals, not the scale correction. On instance identity specifically, that's what the generic-correspondence baseline above already isolates, region identity versus object identity, so we didn't add a fourth redundant arm for it.

SLAM robustness on ScanNet

We traced scenes 0000, 0207, and 0181 to one shared mechanism, VGGT's motion estimate degenerating under weak parallax. The motion-derived scale ratio spikes 12 to 24x on 0207, 142x on 0000, and up to 710x on 0181. On 0181, tracking holds through frame 320, then ATE goes from 5.69 m to 25.6 m across frames 360 to 392 and reaches 17.23 m by frame 744, with no loop closure firing to recover it. We also found 0181's failure is seed-dependent, not deterministic, a second seed on the same config reaches frame 812 cleanly at 0.48 m. We don't have a fix for this yet. What's new here is a characterized mechanism instead of an unexplained failure, plus evidence it isn't fixed to the scene.

Outperforming simpler and existing systems

We haven't reproduced VGGT-SLAM, HI-SLAM2, or SING3R-SLAM ourselves beyond the 7-Scenes comparison already in the paper, and we're not claiming to beat them uniformly there or on ScanNet. We'd rather say that plainly than argue around it.

Semantic loop closure reliability and false closures

We built the false-closure stress test you're asking for. We fabricate a loop-closure candidate between two frames with no genuine revisit, verified against ground-truth poses, and give it a deliberately wrong relative pose. The gate doesn't reject it, it fires at the same confidence and drift values a real high-drift revisit would, and Sim(3) ATE roughly doubles for the rest of the sequence with no later closure recovering it. The submap-rigid correction does bound the damage to the one affected submap at the moment it happens, but it doesn't heal over time, later real closures just build on top of the now-wrong baseline instead of correcting it. We've added this to the appendix next to the existing precision-recall table.

Computational cost and backbone separation

We ran the matched-backbone control directly, plain SplaTAM with raw UniDepthV2, no VGGT, semantics, or loop closure. It beats our full system on every metric on the more static fr1/desk, and our full system only pulls ahead on the more dynamic fr1/desk2. So our added machinery earns its cost specifically as motion gets harder to track, not uniformly, and we're not claiming an efficiency advantage on top of that. We don't have a new cost number to offer here.

Newer geometry foundation models

We did some small-scale testing with the newer foundation model VGGT-Omega, but found it not make a substantial difference for our use case. However, we have not benchmarked against other new backbones than UniDepthV2 and VGGT in this round. We're noting that as a limitation of our current approach. It should be noted though that methods such as MapAnything are somewhat similar to VGGT.

Dear reviewer Z8VJ,

Thank you for your careful review and helpful feedback. As the discussion period draws to a close, we would appreciate your thoughts on whether our additional experiments and clarifications address your concerns. We would be happy to answer any remaining questions.

I appreciate the authors' efforts in the rebuttal. However, after reviewing the rebuttal and the revised manuscript, I find that the new results acknowledged several of my original concerns, while the fundamental issues remain unresolved.

First, my concern regarding methodological novelty remains. The additional experiments cannot address the fundamental lack of methodological contribution, which is just performance comparison but not methodological declaration. Moreover, the newly added backbone-only comparison uses the relatively old SplaTAM baseline. Even so, unmodified SplaTAM with raw UniDepthV2 outperforms the full system on all reported metrics on fr1/desk and achieves substantially lower depth error on fr1/desk2.

The new ablations further raise concerns about the actual contributions of the proposed components. Semantic scale correction alone provides only marginal improvements over disabling semantics, while loop-closure proposals alone achieve better SE(3) ATE than the full system. The online frequency-based fusion optimizer also fails to consistently outperform simpler fixed or global-scalar alternatives. Furthermore, substantial tracking degradation on three ScanNet scenes and a complete tracking failure on another remain unresolved. The verification gate also fails to reject the injected false loop closure, resulting in persistent trajectory degradation.

Finally, although VGGT is incorporated into the system, the evaluation still lacks systematic comparisons with newer and stronger geometry foundation models. The claimed but excluded simple test with just VGGT-Omega does not address this concern.

Overall, the fundamental concerns regarding methodological novelty, empirical advantages, and robustness remain. Therefore, I maintain my original rating, which is in the consensus of other reviewers. It's encouraged for the authors to carefully revise the work in the next submission.

Reviewer_rctmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents S³-SLAM, a training-free monocular Gaussian Splatting SLAM system that integrates UniDepthV2 and VGGT to estimate metrically consistent depth and camera trajectories. The approach combines spatial-frequency-dependent depth fusion, semantic landmark-based scale correction, VGGT-supported pose estimation, and semantic loop closure.

Experiments are conducted on TUM RGB-D, ScanNet, and 7-Scenes. Although the method demonstrates certain improvements over its internal ablation variants, its reconstruction quality and camera localization accuracy remain substantially inferior to competitive existing approaches, with severe trajectory failures on several ScanNet sequences.

Critical strengths
  1. Relevant problem formulation. Combining single-image metric depth priors with multi-view geometric consistency addresses a meaningful challenge in monocular SLAM. The observation that different foundation models exhibit complementary depth errors across spatial-frequency bands is potentially useful.

  2. Evaluation across multiple datasets. The authors evaluate the system on three established SLAM benchmarks and provide ablation studies, computational profiling, and failure-case analysis. These experiments help expose the limitations of the current approach.

Critical weaknesses
  1. Substantially inferior reconstruction quality and severe localization failures.

The primary weakness of this paper is that the proposed method fails to demonstrate competitive reconstruction quality or robust camera localization, despite incorporating multiple powerful pretrained foundation models.

In Appendix Table 6, S³-SLAM achieves substantially worse rendering results than Splat-SLAM across all five TUM RGB-D sequences. The average PSNR decreases from 25.85 dB to 19.19 dB, while average SSIM decreases from 0.838 to 0.695. LPIPS also deteriorates significantly, from 0.186 to 0.408.

The ScanNet results are even more concerning. On the five sequences where the proposed method successfully completes tracking, its average PSNR is approximately 19.37 dB, compared with 29.14 dB for Splat-SLAM on the same sequences. Furthermore, the method completely fails on scene 0181.

The camera localization results reinforce these concerns. On ScanNet scenes 0000 and 0207, the proposed method reports Sim(3) ATE errors of 160.07 cm and 222.22 cm, respectively, compared with 5.5 cm and 7.5 cm for Splat-SLAM. Such substantial trajectory errors raise serious questions about the reliability and practical applicability of the proposed system.

The depth estimation results also fail to demonstrate a clear advantage. Under the Sim(3)-scaled evaluation protocol in Appendix Table 9, the proposed method achieves an average Depth L1 error of 17.07 cm, compared with 15.78 cm for Splat-SLAM.

These limitations directly undermine the paper's central claims concerning metrically consistent mapping and reliable monocular localization. In my assessment, the empirical results are insufficient to establish a competitive contribution for ICLR.

  1. Limited independent methodological contribution, inadequate validation, and excessive computational complexity.

The proposed architecture largely integrates existing components, including UniDepthV2, VGGT, SplaTAM, SAM2, Grounding DINO, and iSAM2, supplemented by heuristic frequency-dependent fusion and semantic scale correction.

The contribution of the central depth-fusion mechanism is insufficiently established. According to Table 5, removing depth fusion decreases PSNR by only 0.18 dB and increases Sim(3) ATE from 4.06 cm to 4.59 cm on TUM fr1/desk2. More importantly, the quantitative ablation is restricted to a single sequence, despite the substantial failures observed elsewhere.

The system also incurs considerable computational overhead. On an NVIDIA RTX A6000, tracking requires approximately 4–6 seconds per frame, while mapping requires an additional 1.5–2 seconds per frame. This is difficult to reconcile with the practical requirements of an online SLAM system, particularly when the resulting reconstruction and localization performance is considerably worse than that of existing methods.

Finally, Figures 2–4 are highly stylized and visually dense, with numerous small labels and schematic elements. Their presentation does not sufficiently clarify the algorithmic dependencies, optimization steps, and interactions between the foundation models and the SLAM backend. The complexity of the architecture is not matched by an equally clear demonstration of its independent technical contributions.

Overall, the method introduces significant architectural and computational complexity without establishing corresponding improvements in reconstruction accuracy, robustness, or efficiency.

Questions for authors
  1. Reconstruction competitiveness: Given the substantial PSNR, SSIM, and LPIPS deficits relative to Splat-SLAM on both TUM RGB-D and ScanNet, what experimental evidence supports the practical advantage of the proposed system? Can the authors provide additional protocol-matched comparisons that demonstrate meaningful improvements over existing monocular Gaussian SLAM methods?

  2. Localization robustness: What are the fundamental causes of the severe trajectory errors on ScanNet scenes 0000 and 0207 and the complete failure on scene 0181? How can the authors establish reliable metric-scale localization when the estimated trajectories exhibit errors exceeding two meters in the evaluated scenes?

  3. Independent contributions and computational efficiency: Can the authors extend the ablation study beyond a single TUM sequence to isolate the benefits of frequency-dependent fusion, semantic scale correction, and loop closure across all benchmarks? How is the substantial computational overhead justified, given that the complete system remains substantially less accurate than existing alternatives?

2 comments

We thank the reviewer for their thorough analysis and comments. We address them in the following text, as well as in the paper.

Reconstruction competitiveness

We have modified our paper to avoid claiming a uniform rendering win. A backbone-only baseline, plain SplaTAM with raw UniDepthV2 and no VGGT, semantics, or loop closure, beats our full system on every rendering metric on fr1/desk, which is a fairly static sequence (Appendix B.1). On fr1/desk2, which is more dynamic, our full system wins both Sim(3) ATE and PSNR instead. So the added components show viability specifically as camera motion gets harder to track, not as a blanket improvement over a simple metric-depth baseline. We've updated the abstract, conclusion, and limitations to say this directly instead of implying a broader reconstruction win.

Localization robustness

Scenes 0000, 0207, and 0181 all come from the same mechanism, weak-parallax VGGT motion degeneracy. The motion-derived scale ratio should sit near 1, but spikes to 12 to 24x on scene 0207, 142x on scene 0000, and up to 710x on scene 0181. On 0181, tracking holds steady through frame 320, then ATE jumps from 5.69 m to 25.6 m across frames 360 to 392 and reaches 17.23 m by frame 744, with no loop closure firing to pull it back. This failure also turns out to be seed-dependent, not deterministic. A second seed on the same config reaches frame 812 cleanly at 0.48 m ATE RMSE. We don't have a fix for this yet. What we have is a characterized mechanism instead of an unexplained failure, plus evidence it isn't baked into the scene, which points at a pose-side degeneracy detector as the next concrete step, the same way our depth fusion already has one.

Independent contributions and computational efficiency

We extended the ablation to ScanNet scene0059, covering frequency-dependent fusion, the scale-correction and loop-closure-proposal decomposition, and the no-depth-fusion and no-loop-closure arms. It doesn't replicate TUM cleanly. Loop closure matters a lot more on ScanNet, Sim(3) ATE goes from 12.03 to 35.65 cm without it, roughly 3x the TUM effect. The semantic decomposition is weaker there too. Scale correction alone actually gets the best ATE of the three decomposed arms, the opposite of what TUM showed. We haven't extended this to 7-Scenes yet. On computational overhead, we don't have a new efficiency number to offer here. What the evidence above does tell us is where the cost is earning its keep rather than how to lower it overall. Loop closure matters more as trajectories get longer, while the frequency-fusion optimizer's edge over simpler alternatives is real but narrower and dataset-dependent (see the Fusion Mechanism Across Datasets appendix).

In general, we agree with the reviewer on overstating the contributions of the paper in the initial version, and have hence softened those claims.

Dear reviewer rctm,

Thank you for your careful review and helpful feedback. As the discussion period draws to a close, we would appreciate your thoughts on whether our additional experiments and clarifications address your concerns. We would be happy to answer any remaining questions.

Reviewer_kvZzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a monocular RGB 3D Gaussian Splatting SLAM system built on SplaTAM’s RGB-D mapping and tracking pipeline. Instead of sensor depth, it constructs depth supervision from two frozen foundation models: UniDepthV2 provides per-frame metric depth, while VGGT provides multi-view-consistent but scale-free depth and relative pose estimates over a sliding keyframe window. VGGT depth is first affinely aligned to UniDepthV2. The two predictions are then decomposed into spatial-frequency bands and fused using online per-band weights selected by minimizing a cross-view geometric residual. A second mechanism uses persistent object landmarks from Grounding DINO, SAM2, and VGGT point tracks to estimate temporal scale variation in UniDepthV2. Per-category landmark ratios are filtered with a state-space GP and combined by a scalar Kalman filter. The same landmarks propose loop closures, which are geometrically verified with VGGT and applied as rigid submap corrections through an iSAM2 pose graph. VGGT relative poses also initialize and constrain keyframe tracking. Experiments on TUM RGB-D, ScanNet, and 7-Scenes show that the estimated global scale is close to metric on most TUM sequences, but trajectory, rendering, and depth accuracy remain below the strongest monocular baselines, with severe failures on three of six ScanNet scenes and a runtime of roughly 6–8 seconds per frame.

Critical strengths
  1. Figure 5 analyzes the error spectra of the two depth sources and connects that analysis to the proposed frequency-dependent weights rather than treating fusion as an unexplained architectural choice.

  2. Semantics are used as correspondence rather than as a hidden metric-depth source. The landmark ratio compares rendered map depth with current UniDepthV2 depth, and the category-dependent uncertainty model is sensible in principle.

  3. The appendix identifies weak-parallax VGGT motion degeneracy, quantifies extreme motion-scale spikes, and distinguishes tracking failure from local depth quality.

Critical weaknesses
  • Splat-SLAM has lower Sim(3) ATE on every reported TUM and ScanNet sequence. On ScanNet, S³ reaches 1.60 m on scene 0000, 2.22 m on scene 0207, and fails on scene 0181, while Splat-SLAM remains between 5.5 and 9.1 cm on all six scenes. Rendering is also roughly 5–12 dB lower in PSNR depending on the sequence, and the protocol-matched TUM Depth L1 average is 17.07 cm versus 15.78 cm. The method does recover near-metric global scale on several sequences, but this is not enough to support a broad claim of consistent metric trajectory estimation across benchmarks.

  • First, the paper should run unmodified SplaTAM with raw UniDepthV2 depth and no VGGT, semantics, or loop closure. This would show whether the full system improves over the direct use of a metric depth foundation model. Second, it should take a strong up-to-scale system such as Splat-SLAM and estimate one global metric scale from UniDepthV2 without ground truth. This would test whether the proposed frame-dependent correction is necessary. Third, a fixed frequency-weight baseline should show whether online per-band optimization adds value beyond a hand-set low/high-frequency split.

  • Splat-SLAM already constructs proxy depth from multi-view geometry and learned monocular depth and optimizes pose/depth jointly through DSPO. Therefore, combining multi-view geometry with a monocular prior is not itself new. ConfidentSplat is even closer in using confidence-weighted fusion of multi-view and scale-aligned monocular depth for RGB-only 3DGS SLAM. The paper should distinguish its metric model, frequency decomposition, and semantic temporal correction from these designs. MyGO-Splat is also relevant to scale-aware foundation-depth alignment and closed-loop geometric feedback.

  • Removing semantics removes both the scale-correction measurements and semantic loop-closure proposals, while removing loop closure disables only the latter. The contribution of the scale correction alone is therefore not measured. Because this mechanism is intended to preserve metric scale, it should be evaluated with SE(3) ATE, fitted scale, and Depth L1, not only Sim(3) ATE and rendering metrics. The paper should also test whether comparison against a map partly built from earlier corrected UniDepthV2 predictions creates a self-reinforcing bias.

Questions for authors

Refer to the Weaknesses section.

2 comments

We thank the reviewer for their thorough analysis and comments. We address them in the following text, as well as in the paper.

W1

We agree about your comparison to Splat-SLAM is real and have revised the claim accordingly. The abstract and conclusion claims on consistency have been softened appropriately. The abstract now states we recover near-metric-scale depth and trajectory estimates on most sequences and identify the specific VGGT motion-degeneracy failure mode behind the sequences causing tracking divergence(scenes 0000, 0207, 0181). Section 5 (Limitations) now states plainly that our rendering quality trails Splat-SLAM on both TUM and ScanNet. However, we would like to make the key caveat that Splat-SLAM does post-hoc bundle adjustment, and also undistorts their images in preprocessing, neither of which we do.

W2

We ran all three requested experiments.

  1. Unmodified SplaTAM + raw UniDepthV2 (no VGGT, semantics, or loop closure), full TUM sequences. Mixed result (Appendix B.1): this baseline beat our full system on every metric on fr1/desk (a comparatively static sequence), but our full system wins Sim(3) ATE and PSNR on the more dynamic fr1/desk2. Our added components however, perform better when camera movement becomes more challenging to track (as in desk2)
  2. About the single-scale fit with UniDepthV2 (Appendix, "Is a Single Global Scale Sufficient?"). The fit recovers 90.5% of the ground-truth-optimal scale and cuts ATE from 64.93 cm to 9.76 cm, but this is still 2–2.3x worse than our frame-dependent correction (4.06–4.87 cm) on the same sequence. A single global scale is not sufficient here, supporting the frame-dependent scale design.
  3. Fixed frequency-weight baseline vs. the online optimizer. Main text (Table 6) compares our shipped optimizer against a frozen, ground-truth-fitted band-weight curve on TUM: the frozen curve matches or beats the online optimizer on every rendering/depth metric, and the online optimizer's only advantage is Sim(3) ATE. We extended this to five mechanisms (fixed bands, frozen curve, global scalar, confidence-weighted, online) on both TUM and ScanNet (Appendix, "Fusion Mechanism Across Datasets"): the ranking does not transfer across datasets. A confidence-weighted baseline is the best mechanism on TUM and the worst on ScanNet; our shipped online optimizer is only mid-table on TUM but best-or-tied on ScanNet. The value of online optimization over simpler alternatives is real but dataset-dependent, not a uniform margin.

W3

Related work now distinguishes our design from both systems directly: "ConfidentSplat and MyGO-Splat fuse depth via per-pixel confidence and closed-loop geometric feedback, respectively. They have also been added to related work. We fuse per spatial-frequency band instead, using persistent semantic object identity rather than pixel confidence for scale correction, and our own confidence-weighted baseline depends on the dataset, being best on TUM and worst on ScanNet."

W4

We broke down the "No semantics" ablation into scale-correction-only and loop-closure-proposals-only, and added SE(3) ATE and fitted scale alongside Sim(3) ATE and Depth L1 as requested (Table 5). Scale-correction-only tracks the fully-disabled setting closely (ATE 9.39 cm vs. 10.60 cm); loop-closure-proposals-only recovers to near the full system (4.39 cm vs. 4.06 cm). The semantic subsystem's benefit comes almost entirely from its loop-closure proposals, not scale correction. Fitted scale stays within 3% of 1 across every row regardless of ATE, including the worst-performing row, showing global metric scale and trajectory shape are governed by different parts of the system.

On self-reinforcing bias: we injected a fixed, incorrect +15% bias into UniDepthV2's raw depth for 80 frames, then removed it, and compared the correction applied vs. withheld (Appendix, "Self-Reinforcing Bias Check"). The correction trace recovers within two to three keyframe cycles after the bias is removed, and the applied/withheld configurations produce statistically indistinguishable correction traces throughout, meaning the correction does not compound. Applying it during the bias costs a small, non-compounding amount of accuracy (6.63 cm vs. 6.15 cm ATE) relative to withholding it, a modest overreaction to a transient error as opposed to a self-reinforcing loop.

Dear reviewer kvZz,

Thank you for your careful review and helpful feedback. As the discussion period draws to a close, we would appreciate your thoughts on whether our additional experiments and clarifications address your concerns. We would be happy to answer any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.