acceptodds
Under review as a conference paper at ICLR 2027

SMART-SLAM: Feed-Forward SLAM with self-adaptive submap memory

Abstract

Long-stream visual geometry needs access to past observations without keeping all scene tokens in active attention. We introduce SMART-SLAM, a feed-forward SLAM system with self-adaptive submap memory. Finalized token banks remain in a host-side archive; each online forward activates only the current submap, its predecessor, and up to k retrieved historical submaps. A detached Q⊤V feature- affinity signal adapts a lightweight retriever while the geometry backbone stays frozen. At end of sequence, Boundary-Pair Backend Re-query (BPBR) composes sparse adjacent relations into an open SE(3) chain for poses and point maps, without loop closure or iterative geometric optimization. On matched RTX A6000 prefixes, bounded-context online inference is 3.83× faster and uses 60.8% less resident GPU memory than native full-history attention. BPBR reduces the online pose error by 53.2% on TUM and 39.7% on Oxford. Indoor reconstruction reaches F-scores of 74.70% on NRGBD and 83.80% on 7-Scenes. Host-side storage grows with the stream while active attention remains bounded.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual SLAM, Visual Geometry Foundation Models, Retrieval-Augmented Inference, Test-Time Adaptation, Long-Term Scene Memory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eFt1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes SMART-SLAM, which preserves past observations while limiting the active context for online attention. The full submap token bank is stored in a host-side archive, and relevant submaps are retrieved using a lightweight retriever adapted at test time. At the end of a sequence, it re-estimates the relative poses between adjacent submaps, and the resulting relative poses are connected through an open SE(3) chain to update the poses and point map. Experiments are conducted on multiple indoor and outdoor datasets, and the results are presented as evidence supporting the claims regarding efficiency, accuracy, and other aspects.

Critical strengths
  • Clear motivation and system design: Separating the preservation of historical information from the active context is well motivated by the growing memory demands of long streams. Also, BPBR provides a clear way to address potential inconsistencies between submaps arising from the limited active context.

  • Experiments across diverse environments: Evaluation across datasets covering diverse indoor and outdoor environments shows both the strengths and limitations of the proposed method.

Critical weaknesses
  • Writing quality and clarity of presentation: The paper needs substantial improvement in clarity, readability, and presentation. The abstract is overly dense with technical details, obscuring the motivation and key contributions. The method section introduces excessive notation and reads as a list of procedural details, without a clear narrative connecting the design choices. The experimental section similarly reports numerical results with insufficient interpretation or analysis. These issues make it difficult to understand the method and assess its contributions.

  • Insufficient design justification and ablation: There is insufficient evidence that Q⊤VQ^\top V affinity serves as a reliable teacher for identifying geometrically useful historical submaps. In particular, comparisons with alternative affinity measures, such as Q⊤KQ^\top K, and validation of the relationship between teacher scores and actual improvements in pose estimation and reconstruction are needed. In Table 1, it is difficult to disentangle the benefits of introducing descriptor-based retrieval from those of updating the retriever parameters at test time using the QV teacher. A controlled experiment that varies only whether the retriever is updated is necessary.

  • Insufficient comparisons and experimental validation: Comparison methods and evaluation scopes vary across tables, making it difficult to assess consistent comparisons against major competing streaming/SLAM methods. The main paper should provide more detailed explanations and analysis of the experimental results, along with clearer definitions of some evaluation metrics. For example, Figure 4 presents both improvements and regressions, but further analysis is needed to explain why these occur and what they reveal about strengths and limitations of the proposed method. The GPU memory results in Tables 8 and 9 show opposite trends, which require further explanation. Regarding the reported FPS improvements, comparisons with other methods are needed to assess the significance of these gains. Additionally, quantitative results on the runtime of terminal BPBR should be provided. As SMART-SLAM employs a submap-level retrieval strategy, further discussion or comparison with closely related methods, such as STAC (Wang et al., 2026), which store historical information at the token level outside the active context and retrieve it when needed, would be valuable.

Questions for authors
  • Is there a specific reason for selecting evaluation subsets (e.g., 00/05/06/07/09/10 sequences for KITTI in Table 5 and 00/06/09/10 in Tables 8 and 9)?
  • Have you evaluated your method against more recent token eviction methods, such as OVGGT (Lu et al., 2026)? How does it compare in terms of accuracy, GPU memory usage, and runtime?
1 comment

Thank you for the response. However, the following concerns remain unresolved:

  • The abstract remains dense with technical details and experimental results.
  • The method section remains largely procedural and still lacks a clear narrative connecting the design choices.
  • The comparisons with competing methods remain inconsistent across experiments.
  • The connection between the Q⊤VQ^\top V teacher and the retrieval of useful historical submaps remains insufficiently supported, as the Spearman correlations are weak.
  • In addition, the revised manuscript and the newly provided results raise further concerns
    • Several results are described only in the text without being presented in the main-paper tables.
    • The claimed efficiency gains are not consistently observed.

Thus, I will maintain my original rating.

Reviewer_65njOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduce SMART-SLAM, a novel feed-forward SLAM system to address the memory issue of SLAM Transformer. The contribution lays on the backend re-using of previous submaps' tokens, efficient retrieval and the post-processing.

Critical strengths
  1. The paper introduce a novel mechanism to cooperate with the frozen SLAM Transformers.

  2. The paper addresses the speed&memory limitation of SLAM Transformers with long-term history.

Critical weaknesses

Major concerns:

  1. The online backend can retrieve loop and refine with the looped submaps, but seems it only stores the current submap and keeps the looped submaps unchanged. So the online backend maynot geometrically fit with the stored looped maps.

  2. The post-processing seems only cares about the adjacent submaps, which seems conflict with the SLAM's loopclosing requirements for global consistency.

  3. The SMARS-SLAM** utilize a conventional PGO to close the loop, which raises questions about the contribution of the proposed feed-forward framework to global consistency.

  4. It doesnot shows the reconstruction comparison with SLAM-Former.

Minor concerns:

  1. What is "Native" should be clarified in all tables
  2. post-processing BPBR time should also be reported.
Questions for authors

Address my major concerns, including the global consistency issue over all components, and the reconstruction comparison.

Reviewer_8cpbOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SMART-SLAM separates persistent submap history from bounded active attention and adapts a retriever online. At sequence end, BPBR re-queries adjacent submap boundaries to correct poses and point maps. Recent-context, adaptive-retrieval, and full-history comparisons show efficiency gains from bounded context and accuracy improvements on some datasets.

Critical strengths
  1. Table 8 controls device, inputs, and model when comparing bounded context with full history, supporting memory and throughput benefits. The paper acknowledges growing host storage.

  2. Recent, QV-TTA, and terminal-correction controls include regressions, helping explain practical trade-offs.

Critical weaknesses
  1. The local-to-global handoff is incomplete. Section 3.2 points to H.4, but H.4 mainly discusses rigid-frame invariance and assumes compatible translation scales. It does not fully specify how each prediction is committed to the existing world frame. This step is essential for reproducing Online and BPBR outputs.

  2. Online adaptation is not isolated. Comparing QV-TTA with Recent changes both retrieval policy and parameter updating. Their similar speeds in Table 8 indicate that bounded context explains most efficiency gains. Adaptation should primarily be supported by accuracy differences against the same content-based retriever held fixed.

P.S. I do not favor using generated or idealized illustrations in the main pipeline’s output panels, even when labeled schematic, as they obscure actual reconstruction quality and failure modes. Please replace them with real intermediate and final outputs from the same sequence.

Questions for authors
  1. Please give equations or pseudocode for the pass-local-to-committed-world handoff, including the use of overlapping frames and the treatment of scale. What evidence supports compatible translation scales between Online and boundary re-query outputs?

  2. How does QV-TTA compare with the identical initialized content-based retriever held fixed, using the same context budget, accepted frames, and terminal policy? How is the retriever initialized before any online update?

  3. Can you report the default output's terminal BPBR cost and summarize which sequences improve or degrade? Do the existing diagnostics connect failures to scale disagreement, boundary geometry, or submap configuration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.