- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper adds an external, revisable memory (SMORE) to frozen 3D geometry models. It splits long videos into 32-frame windows, stores frames, poses and point maps, retrieves old frames with DINOv2, checks them geometrically, and uses accepted revisits to correct window placement. It tests 8 models on 7 datasets over 5 tasks and reports better pose, depth, reconstruction, SLAM and NVS results.
Critical strengths
1.The core result is broad and consistent. SMORE improves 8 frozen backbones on pose, depth and reconstruction, and gains grow with sequence length. This is a practical, training-free interface: the geometry model stays frozen, and memory handles long-horizon correction. This is the main reason I lean toward acceptance. 2.The verification step is well designed. Requiring two independent historical references to agree, and spreading the correction progressively in Sim(3), is a sensible guard against a single false loop closure. This is the step that makes retrieval usable for correction, and the results depend on it.
Critical weaknesses
1.The NVS comparison does not test the memory. Appendix J.2 says the baseline uses one prediction, while +SMORE uses four input orderings plus consensus. With only four input frames there is one 32-frame window; retrieval removes candidates within 64 frames and needs two pairs from earlier windows, so the memory never triggers. Table 2 is therefore test-time ensembling, not SMORE. Some gains are within noise (+0.05 dB, +0.08 dB). This does not invalidate the main method, but the “five tasks” claim should be corrected: either give the baseline the same four-ordering consensus, or report NVS as an ablation of repeated prediction. 2.The closest prior work and verification statistics are missing. VGGT-Long is not cited, and VGGT-SLAM 2.0 already does training-free attention-based retrieval verification; VGGT-SLAM++ uses VPR + Sim(3) pose graph. SMORE compares to neither, nor to a standard VPR baseline (SALAD/AnyLoc) + geometric verification + pose graph. The lookup-table ablation is a strawman because it needs frame identity. Also, SLAM uses published numbers on 7-Scenes only, and Appendix N admits joint processing can hurt even when revisits are rejected, but the main text reports no distortion gate or false-accept rate. These limit the strength of the novelty/SOTA claims, but they are addressable with additional comparisons and reporting.
Questions for authors
1.NVS: Does the baseline get the same four-ordering consensus? If yes, what does SMORE add with only four frames? If no, please rerun with identical test-time augmentation, or relabel NVS as an ablation. 2.Verification: What are the precision/recall and false-accept rate of revisit verification? What is the default distortion gate, how often does it trigger, and is there any rollback after a bad revision? 3.Scaffold: How does the full pipeline compare to standard VPR + geometric verification + Sim(3) pose graph with the same windowing and backbones? Does using the drifted global pose as the memory address propagate drift? 4.SLAM: Can you rerun the baselines under the same protocol and report large-scale results (KITTI/Waymo/TUM) against VGGT-Long and VGGT-SLAM 2.0? Also, why is Sim(3) enough given VGGT-SLAM's SL(4) argument for uncalibrated submaps? 5.Cost: What is the per-frame latency and total runtime at N=200, including DINOv2, retrieval and the 36-frame joint passes? Is it online/real-time?
4 comments
Response to Reviewer GLRg (Part 2)
GLRg.4 - Question 2. Verification: What are the precision/recall and false-accept rate of revisit verification? What is the default distortion gate, how often does it trigger, and is there any rollback after a bad revision?
Response: We thank the reviewer for these important questions regarding verification reliability and revision robustness.
1. Verification reliability and distortion gating. SMoRe uses a 5° distortion gate by default to reject geometrically inconsistent corrections. As reported in Table R8c, this configuration achieves 97.30% precision, 90.00% recall, and a 2.27% false-positive rate on 7-Scenes. The table additionally reports accepted and rejected revisit counts across different gating thresholds. To directly address the reviewer's question about gating frequency, we additionally measure the average distortion-gate trigger rate per query window, which is only 2.38% under our default 5° configuration. This indicates that the distortion gate is triggered relatively infrequently, allowing most candidate corrections to proceed without being rejected by this additional safeguard. Please also refer to R46d.2 – Weakness 2 for a detailed sensitivity analysis of the distortion gate and Tables R8d–R8e for comparisons of verification reliability against conventional VPR methods.
2. Revision safeguards and rollback. We clarify that SMoRe does not currently implement an explicit rollback mechanism after an accepted revision. Instead, we mitigate erroneous corrections through multiple consistency checks before applying them, including scale-consistency thresholds, orientation verification, and pairwise camera-distance consistency. Only geometrically verified corrections are propagated to historical window placements.
3. Progressive and retrospective geometric revision. To evaluate the importance of propagating verified corrections, we additionally compare SMoRe against two alternative optimization strategies: (i) deferred pose-graph optimization, where corrections are applied only after processing the entire sequence, and (ii) progressive Sim(3) optimization without backward propagation, where corrections update the current window but leave historical placements unchanged. As shown in Table R11, SMoRe's default progressive revision strategy outperforms both alternatives on 7-Scenes SLAM and CO3Dv2 camera pose estimation. These results demonstrate the benefit of propagating verified corrections to previously estimated window placements rather than leaving historical geometric inconsistencies unresolved. Please refer to R46d.5 – Question 1 for the detailed comparison.
Response to Reviewer GLRg (Part 3)
GLRg.5 - Question 3. Scaffold: How does the full pipeline compare to standard VPR + geometric verification + Sim(3) pose graph with the same windowing and backbones? Does using the drifted global pose as the memory address propagate drift?
Response:
1. Comparison with conventional VPR and geometric optimization. We draw the reviewer's attention to Table S6 (Group C) in the manuscript, where we replace SMoRe's associative scaffold with established VPR methods, including SALAD, NetVLAD, MixVPR, EigenPlaces, CosPlace, and BoQ, while keeping the geometry backbone, windowing strategy, geometric verification, and downstream revision procedures fixed. These controlled comparisons demonstrate the additional benefit of associative scaffold memory over conventional visual retrieval.
2. Why associative scaffold memory improves retrieval. In SMoRe, each video frame is associated with a shared memory representation (), which links its visual descriptor () to its spatial scaffold state (). Historical observations are retrieved by comparing these associative representations rather than directly matching visual descriptors. This design allows spatial information to implicitly influence retrieval through learned visual–spatial associations, providing a spatially informed retrieval mechanism rather than a conventional visual-feature lookup table.
3. Does a drifted spatial address propagate geometric errors? Importantly, the stored spatial state () is used only to propose retrieval candidates, not to directly correct geometric placements. Each retrieved pair is jointly re-estimated with the current window, and a correction is propagated to historical windows only when two independently retrieved pairs from different source windows produce consistent placements. Specifically, Eq. (4) requires agreement within 3° in rotation, () in position, and the scale ratio below 1.5. Therefore, although drift in stored spatial states may affect candidate retrieval, it does not directly propagate into geometric corrections. Placement updates depend on newly estimated and geometrically verified correspondences rather than the stored spatial addresses themselves. This separation also allows SMoRe to retain historical scaffold states without recomputing them after every revision.
4. Additional experiment: Retrospective updates of stored associative memory. To directly investigate whether updating historical memory after geometric revision improves performance, we conduct an additional controlled experiment. Whenever historical camera poses are corrected, we refresh their associated spatial states and memory representations (), together with the corresponding associative mappings (), (), and (). All subsequent retrieval, geometric verification, and placement revision procedures remain unchanged.
We evaluate this retrospective-update variant on 7-Scenes SLAM and CO3Dv2 camera pose estimation, with results reported in Table R1. Retrospective memory updates achieve performance comparable to our default implementation but incur additional computational overhead from refreshing historical representations and associations. These findings suggest that explicitly refreshing historical associative memory after every geometric correction provides limited additional benefit under the evaluated conditions. Our default implementation therefore maintains a more efficient separation between associative retrieval and verified geometric revision, while online adaptation of the associative mappings continues to incorporate new observations.
Response to Reviewer GLRg (Part 4)
GLRg.6 - Question 4. SLAM: Can you rerun the baselines under the same protocol and report large-scale results (KITTI/Waymo/TUM) against VGGT-Long and VGGT-SLAM 2.0? Also, why is Sim(3) enough given VGGT-SLAM's SL(4) argument for uncalibrated submaps?
Response:
1. Large-scale SLAM comparisons under a unified evaluation protocol. Following the reviewer's suggestion, we rerun VGGT-Long and VGGT-SLAM 2.0 on TUM RGB-D, KITTI Odometry, and Waymo using a consistent evaluation protocol, reporting absolute trajectory error (ATE) in metres. Tables R13a–R13c present results across nine TUM sequences, four KITTI sequences, and the Waymo dataset. On TUM RGB-D, SMoRe achieves a mean ATE of 0.0523 m online and 0.0489 m after final optimization, compared with 0.0544 m for VGGT-SLAM 2.0 under the same protocol. On KITTI, SMoRe achieves the lowest mean ATE of 3.1 m, outperforming both VGGT-Long and VGGT-SLAM 2.0. On Waymo, SMoRe achieves a modest improvement in mean ATE over VGGT-Long while requiring less processing time per frame than both VGGT-Long and VGGT-SLAM 2.0. These additional experiments extend our evaluation to larger-scale indoor and outdoor trajectories and provide direct comparisons with recent geometry-based SLAM systems.
2. Sim(3) versus SL(4) alignment. We acknowledge the reviewer's point that SL(4) provides greater flexibility for aligning uncalibrated submaps, as discussed in VGGT-SLAM. SMoRe adopts Sim(3) by default because it models relative rotation, translation, and isotropic scale while preserving Euclidean geometric structure. However, Sim(3) cannot fully represent the more general projective distortions that SL(4) can accommodate. To quantify this tradeoff, we additionally compare Sim(3) and constrained SL(4) alignment under identical local predictions and geometric constraints, as reported in Table R13d. The results show that SL(4) provides modest geometric accuracy improvements but incurs greater computational overhead. Thus, our default Sim(3) backend represents a practical tradeoff between geometric accuracy and processing efficiency, rather than an assumption that similarity transformations are universally sufficient. We will incorporate these large-scale benchmark comparisons and the Sim(3)/SL(4) ablation into the revised manuscript, explicitly discussing the benefits and limitations of each alignment model.
GLRg.7 - Question 5. Cost: What is the per-frame latency and total runtime at , including DINOv2, retrieval and the 36-frame joint passes? Is it online/real-time?
Response:
We thank the reviewer for requesting a detailed computational cost analysis. We now provide additional experiments quantifying the latency and memory requirements of SMoRe.
1. Per-frame latency and total runtime. We profile the computational cost of each major component, including DINOv2 feature extraction, associative memory retrieval, 36-frame joint inference for geometric revision, and windowed geometry estimation using VGGT-Ω. Table R14a reports the individual latency contributions. The dominant computational cost comes from the VGGT-Ω backbone for windowed geometry inference. Overall, SMoRe requires approximately 0.16 seconds per frame, corresponding to 31.3 seconds for a sequence of 200 keyframes. SMoRe operates online, processing observations incrementally without requiring access to future frames. However, online processing does not necessarily imply real-time operation; real-time performance depends on the input frame rate and hardware configuration. When optional final pose-graph optimization is enabled, it adds approximately 0.62 seconds, increasing the total processing time to 31.92 seconds for 200 keyframes. We emphasize that this final optimization is optional and is not included in our default online results in the main text.
2. Memory scalability over long sequences. We additionally evaluate memory consumption as the sequence length increases to 5,000 keyframes. As shown in Table R14c, SMoRe's active GPU memory allocation remains approximately bounded, while host memory usage increases with the number of retained historical observations. This behavior contrasts with native causal streaming models whose growing inference state can lead to substantially higher peak GPU memory consumption and eventually out-of-memory failures. Table R3 provides controlled comparisons among native causal streaming, bounded-window chaining, and SMoRe. Importantly, SMoRe maintains a GPU memory profile comparable to bounded-window chaining while achieving higher geometric accuracy, demonstrating that its retrieval and revision mechanisms do not require unbounded GPU-resident inference states. Please refer to Pe6y.4 – Weakness 2(a) for further analysis of streaming inference, computational cost, and memory scalability. These results clarify that SMoRe supports incremental online processing with bounded active GPU memory and moderate computational overhead.
Response to Reviewer GLRg (Part 1)
Reviewer GLRg:
We sincerely thank the reviewer for the constructive feedback and insightful suggestions, which have helped us strengthen the manuscript. For ease of reference, we will reproduce the reviewers’ questions and concerns verbatim in bold throughout our responses, allowing reviewers to locate the corresponding clarifications and additional evidence quickly. We will incorporate all clarifications, additional experimental results, and analyses presented in our rebuttal into the camera-ready version.
GLRg.1 - Weakness 1.The NVS comparison does not test the memory. Appendix J.2 says the baseline uses one prediction, while +SMORE uses four input orderings plus consensus. With only four input frames there is one 32-frame window; retrieval removes candidates within 64 frames and needs two pairs from earlier windows, so the memory never triggers. Table 2 is therefore test-time ensembling, not SMORE. Some gains are within noise (+0.05 dB, +0.08 dB). This does not invalidate the main method, but the “five tasks” claim should be corrected: either give the baseline the same four-ordering consensus, or report NVS as an ablation of repeated prediction.
Response: We thank the reviewer for identifying this important experimental confound. To address this concern, we have redesigned the NVS experiments, replacing the original object-centric Objaverse-LVIS and GSO datasets with two long-sequence-based benchmarks: 7-Scenes and CO3Dv2. Please refer to Tables R5a-b and response Pe6y.4 - Weakness 2(d) for more details.
GLRg.2 - Weakness 2. The closest prior work and verification statistics are missing. VGGT-Long is not cited, and VGGT-SLAM 2.0 already does training-free attention-based retrieval verification; VGGT-SLAM++ uses VPR + Sim(3) pose graph. SMORE compares to neither, nor to a standard VPR baseline (SALAD/AnyLoc) + geometric verification + pose graph. The lookup-table ablation is a strawman because it needs frame identity. Also, SLAM uses published numbers on 7-Scenes only, and Appendix N admits joint processing can hurt even when revisits are rejected, but the main text reports no distortion gate or false-accept rate. These limit the strength of the novelty/SOTA claims, but they are addressable with additional comparisons and reporting.
Response: Thank you! We now evaluate VGGT-Long, VGGT-SLAM 2.0 and VGGT-SLAM++ in our SLAM task table. All three revise earlier estimates through loop closure and a pose graph. We ran the three systems under our protocol, with the same window sizes and backbones (Table R12). On 7-Scenes SLAM task they reach 0.0420, 0.0514 and 0.0648 m against 0.0301 m for SMoRe, and on CO3Dv2 camera estimation 90.6 to 91.7 (VGGT) and 90.4 to 92.5 (VGGT-Ω) against 93.9 and 94.7 on AUC@30. SMoRe performs better than these established SLAM systems using the same protocol on the same dataset sequences.
We would like to draw the reviewer’s attention to Table S6 Group C in the manuscript, where we only replace the scaffold with all the established VPRs like SALAD, NetVLAD, MixVPR, EigenPlaces, CosPlace and BoQ while keeping the geometric verification and other thresholds fixed to the default modes.
Please refer to Tables R8c-e and response R46d.2 - Weakness 2 for more details on the distortion gate analysis and false accept rates associated with each gate.
GLRg.3 - Question 1. NVS: Does the baseline get the same four-ordering consensus? If yes, what does SMORE add with only four frames? If no, please rerun with identical test-time augmentation, or relabel NVS as an ablation.
Response: We thank the reviewer for identifying this important experimental confound. To address this concern, we have redesigned the NVS experiments, replacing the original object-centric Objaverse-LVIS and GSO datasets with two long-sequence-based benchmarks: 7-Scenes and CO3Dv2. For fair comparisons, we removed the four-ordering consensus for all baselines and our SMORE in these experiments. Please refer to Tables R5a-b and response Pe6y.4 - Weakness 2(d) for more details.