acceptodds
Under review as a conference paper at ICLR 2027

Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction

Abstract

Streaming 3D reconstruction from extremely long videos requires online estimation of camera motion and scene geometry with bounded model memory and per-frame computation. Early streaming models use finite context buffers or compact recurrent states, yet their estimates often deteriorate as sequences grow. Recent methods improve long-horizon stability through persistent or multi-level long-range memory. We pursue a different route: local geometric prediction with reliable sequential composition. We present LC3R, a simple streaming model that caches KV features from only the preceding 11 frames. It predicts a point map in the current camera coordinate system and an adjacent-frame relative pose, keeping prediction targets local as the stream grows. Global poses and geometry are recovered by composing these local measurements. To ensure reliable long-horizon composition, a lightweight temporal rotation refiner improves relative rotations using recent visual and motion context, while a composition-aware pose loss directly supervises multi-step pose composition. On Oxford Spires, LC3R achieves an ATE of 4.35 m and an RPE-R of , reducing both errors by approximately 40% relative to the respective best prior streaming results. It runs at 24.45 FPS with 6.71 GB peak GPU memory, excluding input storage, on an NVIDIA H100 GPU.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Streaming 3d Reconstruction, Feedforward Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_N1xzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LC3R, a streaming 3D reconstruction model that uses only the most recent 12 frames. Built upon π³, it predicts adjacent-frame relative poses and local point maps, then recovers global reconstruction through sequential pose composition. A Rotation Refiner and Composition-aware Loss are introduced to reduce accumulated drift. The authors aim to show that accurate and efficient long-horizon reconstruction can be achieved without persistent long-term memory.

Critical strengths
  1. The approach is simple and avoids complex long-term memory management, offering a lightweight alternative for streaming reconstruction.
  2. The experimental results are competitive. On Oxford Spires, ATE decreases from 8.81 m (HorizonStream) to 4.35 m, and on KITTI, from 22.51 m to 18.25 m.
  3. The effectiveness of Composition-aware Loss is worth noting. In Table 4, adding this loss alone reduces KITTI ATE from 56.60 m to 27.66 m.
  4. The method achieves relatively high inference efficiency, reaching 24.45 FPS with 6.71 GB of GPU memory.
Critical weaknesses
  1. Limited methodological originality. Local-to-global pose recovery has already been explored in methods such as SLAM3R and Anchor3R, while HorizonStream combines local and long-term context for long-horizon reconstruction. In comparison, LC3R mainly differs by using a fixed local window, directly predicting adjacent-frame relative poses, and composing them sequentially, but this idea itself is not new. The Rotation Refiner relies on standard temporal feature fusion and residual rotation correction, while the Composition-aware Loss follows the established approach of multi-frame pose supervision. Overall, the paper appears to be more of a simplification and engineering combination of existing streaming reconstruction frameworks, rather than introducing a distinct core technical contribution.

  2. The experiments do not establish that local context is responsible for the performance gains. In Table 4, the baseline achieves 56.60 m ATE on KITTI, which drops to 27.66 m simply by adding Composition-aware Loss. On Oxford Spires, the largest improvement comes from longer-sequence training. It is somewhat counterintuitive that retaining less historical information would lead to better reconstruction accuracy, and this claim requires more direct evidence and explanation. The authors lack a direct comparison between local and long-term context under the same backbone, training conditions, and loss functions. The current results appear to demonstrate the effectiveness of additional supervision and training strategies rather than the superiority of local context over long-term memory.

  3. Concerns about comparison fairness and long-horizon stability. LC3R uses 30 training datasets, including 7.71% internal data, along with a different pretrained model and longer training sequences. The efficiency comparisons also use different input resolutions. Moreover, without loop closure, LC3R performs slightly worse than HorizonStream on VBR. These factors make it difficult to determine whether the reported advantages consistently hold in longer and more challenging scenarios.

Questions for authors
  1. Why would retaining less historical information lead to better results? This seems counterintuitive, as long-term information generally helps maintain geometric consistency. Can the authors explain why long-term memory might instead be detrimental and provide direct experimental evidence?

  2. How is long-term scale drift handled? When adjacent poses are continuously composed, scale errors may also accumulate. Without explicit long-term scale constraints, how does the model maintain scale consistency across different parts of a sequence? Could Sim(3)-aligned ATE conceal some of these scale-related errors?

  3. Are the improvements mainly due to the local architecture or Composition-aware Loss? Table 4 shows that the latter accounts for a large portion of the improvement. If the same loss and rotation refinement were applied to other existing methods, would they achieve similar gains?

  4. What fundamentally distinguishes LC3R from Anchor3R, SLAM3R, and HorizonStream? Beyond reducing the context window, changing the pose prediction formulation, and adding two components, what problem does this method solve that existing approaches cannot?

  5. How can the effects of training data and computational settings be ruled out? Can the authors provide comparisons under matched training data, pretrained backbones, and input resolutions? How much do the internal datasets contribute to the final results?

5 comments
AuthorsOpenReview ↗

Responses to Reviewer N1xz (1/3)

W1 & Q4. Difference between our method and SLAM3R, anchor3R, and HorizonStream & Key contribution.

Our central contribution is to show that accurate long-horizon dense reconstruction can be achieved by composing a single chain of learned adjacent-frame poses. At each step, LC3R predicts the current frame’s pose relative to the preceding frame and a dense point map in the current camera coordinates. Sequential pose composition places each point map in the global frame. This keeps the network’s prediction task the same throughout the sequence, regardless of how far the camera has traveled. The rotation refiner corrects each relative rotation before composition, and the composition-aware pose loss supervises the motion obtained by composing several consecutive predictions.

The distinction from prior work lies in how the global reconstruction is recovered. SLAM3R learns to register keyframe-centered point maps to the existing scene. Anchor3R and HorizonStream fuse pose information from multiple frames within a window. LC3R directly composes one predicted transform per frame. With this formulation, LC3R achieves the lowest ATE among the compared streaming methods without loop closure on KITTI (18.25 m) and Oxford Spires (4.35 m), and runs at 24.45 FPS with 6.71 GB of GPU memory on KITTI-02.


W2 & Q1 & Q3. Effectiveness of Local Context and Architectural Design.

We summarize the reviewer's concerns regarding the Local Architecture, including its effectiveness, the source of performance improvements, the relationship between Local Context and reconstruction accuracy, and the transferability of additional components. To better clarify the rationale and contributions of each architectural component, we conduct a series of ablation studies and address these concerns individually.

Our Local Architecture consists of two key designs: Adjacent-relative Pose Prediction and Local Context. To disentangle their contributions, we construct several model variants based on the original ablation baseline:

  1. First, following global pose prediction approaches such as StreamVGGT and LingBot-Map, we construct a global pose prediction variant that additionally retains the first-frame features as a persistent anchor and predicts the current pose relative to the first frame.
  2. Next, we keep both the first-frame context and the sliding-window context, but change the prediction formulation to adjacent-frame relative pose prediction, isolating the contribution of Adjacent-relative Pose Prediction.
  3. We further remove the first-frame context and retain only the recent 12-frame sliding window, recovering our original baseline and isolating the contribution of Local Context.
  4. Additionally, we extend the sliding window to 24 frames and shorten it to 6 frames to further investigate the effect of context length on model performance.

All architectural ablations use the same backbone, training configurations, and training budget. Except for experiment (6), which specifically evaluates Composition-aware Loss, all other experiments use the same baseline supervision without Composition-aware Loss, ensuring fair architectural comparisons.

IDPose PredictionTemporal ContextComposition-aware LossKITTI ATE ↓Oxford ATE ↓
1First-frame-globalFirst-frame + 12-frame Window−230.0235.06
2Adjacent-relativeFirst-frame + 12-frame Window−196.0827.54
3Adjacent-relative12-frame Window (Baseline)−56.6010.16
4Adjacent-relative24-frame Window−82.9517.66
5Adjacent-relative6-frame Window−72.1416.76
6Adjacent-relative12-frame Window+27.669.40
AuthorsOpenReview ↗

Responses to Reviewer N1xz (2/3)

Q: Does the Local Architecture independently contribute to performance gains?

Our Local Architecture consists of two components: Adjacent-relative Pose Prediction and Local Context.

First, comparing experiments (1) and (2), while keeping the first-frame anchor and sliding-window context unchanged, replacing global pose prediction with adjacent-frame relative pose prediction reduces KITTI ATE from 230.02 m to 196.08 m and Oxford ATE from 35.06 m to 27.54 m. This demonstrates that local pose prediction can improve geometric estimation.

Second, comparing experiments (2) and (3), while keeping adjacent-frame pose prediction unchanged, removing the persistent first-frame context further reduces KITTI ATE substantially from 196.08 m to 56.60 m and Oxford ATE from 27.54 m to 10.16 m. This demonstrates that strictly local context is also an important source of performance gains.

Both architectural modifications bring substantial improvements, demonstrating the importance of the complete Local Architecture for achieving high-quality long-horizon streaming reconstruction under a limited training budget.

Q: Why would retaining less historical information lead to better results?

First, comparing experiments (2) and (3), we find that retaining additional first-frame context significantly degrades performance even with adjacent-frame relative pose prediction. We hypothesize that, since the first frame is persistently available as a fixed reference, the model may develop an implicit dependency on this distant anchor, which reduce its Long-horizon extrapolation.

Second, comparing experiments (3) and (4), directly extending the window from 12 to 24 frames also degrades performance. KITTI ATE increases from 56.60 m to 82.95 m, while Oxford ATE increases from 10.16 m to 17.66 m. We hypothesize that although a longer window provides more historical information, it also introduces additional features that may be less relevant to the current adjacent-frame pose prediction task, increasing the complexity of feature fusion and optimization.

Meanwhile, comparing experiments (3) and (5), shortening the window to 6 frames also degrades performance, indicating that an overly short context cannot provide sufficient local temporal information.

Overall, our choice of a 12-frame window is not simply driven by the goal of retaining less historical information, but rather by matching the context length to the local geometric prediction task. Insufficient context limits the information available to the model, while blindly incorporating more historical information does not necessarily bring benefits and may instead increase optimization difficulty. These results support our choice of a fixed local context.

Q: Are the improvements mainly due to the Local Architecture or Composition-aware Loss?

Adding Composition-aware Loss to the original baseline yields experiment (6). KITTI ATE decreases from 56.60 m to 27.66 m, while Oxford ATE decreases from 10.16 m to 9.40 m, demonstrating that composition-aware supervision provides additional performance gains.

Combining this result with the preceding architectural ablations, we observe that, even without Composition-aware Loss, local pose prediction and local context alone reduce KITTI ATE from 230.02 m to 56.60 m and Oxford ATE from 35.06 m to 10.16 m. This demonstrates that the Local Architecture itself already provides substantial performance improvements, while Composition-aware Loss further enhances the reliability of multi-step pose composition.

Therefore, we consider the Local Architecture to be the more significant source of performance improvements, while Composition-aware Loss provides complementary gains.

AuthorsOpenReview ↗

Responses to Reviewer N1xz (3/3)

Q: How transferable are the additional components?

Regarding the transferability of our additional components, the Rotation Refiner is specifically designed for adjacent-frame pose prediction. Its input feature fusion and residual prediction mechanisms are tailored to this formulation, making it difficult to directly apply to other pose prediction architectures.

Although Composition-aware Loss has some degree of generality, its effectiveness also depends on the underlying pose prediction formulation. To further investigate this, we conduct an additional experiment based on the original +Comp. Loss variant. We further experimented with an Anchor3R-like multi-reference pose fusion formulation under the same composition-aware supervision, but observed degraded performance. This may because jointly optimizing transforma- tions from the current frame to multiple reference frames introduces competing optimization paths, which may be further amplified by composition-aware supervision. Therefore, these components are not necessarily plug-and-play improvements and should be considered together with the underlying architecture.

MethodKITTI ATE ↓KITTI RPE-R ↓KITTI RPE-T ↓Oxford Spires ATE ↓Oxford Spires RPE-R ↓Oxford Spires RPE-T ↓
Anchor3R-style99.230.430.3326.380.550.07
Ours27.660.140.139.400.150.03

W3 & Q5. Comparison Fairness and the Role of Internal Data.

Strictly matching training data, pretrained backbones, and computational settings across methods is challenging, particularly when complete training code and configurations are unavailable. Besides, both LingBot-Map and HorizonStream also use internal data in their training mixtures, which makes it difficult to reproduce their exact training sets.

Our training mixture is comparable in the number of datasets to those of LingBot-Map and HorizonStream, and our sampling strategy draws on their reported data compositions. We build LC3R on the π³ architecture and initialize it with the released π³ weights because its reference-view-free design suits adjacent-frame relative pose prediction, while its performance is comparable to that of VGGT. Starting from these weights, LC3R achieves competitive performance with 74K optimization steps (32K + 38K + 4K). For context, LingBot-Map reports 320K steps across base-model and streaming training, HorizonStream reports 100K, and Anchor3R reports 80K.

We include internal data primarily to expose LC3R to a wider range of scenes and encourage generalization to unseen environments. These data account for 7.71% of the sampling distribution in our full training setup. To assess their contribution to benchmark performance, we trained two LC3R baselines in a reduced-data setting, keeping both the number of public datasets and the amount of public data the same across runs. Adding internal data reduced KITTI ATE by 1.02 m, from 23.61 to 22.59 m, while Oxford Spires ATE increased by 0.02 m, from 7.38 to 7.40 m. The measured effect was modest on these two benchmarks.

Training dataKITTI ATE (m) ↓KITTI RPE-R (°) ↓KITTI RPE-T (m) ↓Oxford Spires ATE (m) ↓Oxford Spires RPE-R (°) ↓Oxford Spires RPE-T (m) ↓
Public datasets23.610.170.117.380.380.02
Public datasets + internal data22.590.170.107.400.380.02

Q2. Scale Drift and Sim(3) Alignment.

We address scale consistency through both training and prediction formulation. During training, we apply a unified scale normalization to each entire 32/128-frame sequence rather than normalizing individual frames independently, encouraging consistent scale estimation across the sequence. Furthermore, multi-dataset training enables the model to internalize stable scale priors across diverse scenes, keeping its scale estimates within a relatively consistent range.

More importantly, unlike predicting poses relative to an increasingly distant initial frame, LC3R predicts adjacent-frame relative poses, keeping the prediction target local and independent of the inference horizon. As shown in Fig.~6(a), LC3R maintains stable adjacent-frame translational errors throughout KITTI-02.

Regarding Sim(3)-aligned ATE, we acknowledge that it can indeed conceal certain scale-related errors. Feed-forward 3D reconstruction models typically do not define an absolute scale. Therefore, unified Sim(3) alignment to remove global scale differences is a standard evaluation protocol in this field.

Reviewer_w473OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents LC3R, a streaming 3D reconstruction model that uses a fixed 12-frame causal attention window to predict camera-local point maps and adjacent-frame relative poses. Global trajectories and geometry are recovered through sequential pose composition. To reduce accumulated drift, the authors introduce a lightweight motion–visual temporal rotation refiner and a composition-aware pose loss that supervises relative transformations over multiple temporal spans. Experiments demonstrate strong long-sequence pose estimation and competitive dense reconstruction, with favorable accuracy–efficiency trade-offs. Ablations show consistent benefits from composition-aware supervision, longer training sequences, and rotation refinement.

Critical strengths
  1. The proposed temporal rotation refiner and composition-aware pose loss effectively mitigate long-horizon drift, with ablations demonstrating consistent improvements across multiple benchmarks.

  2. Although loop closure provides substantial benefits in some cases, the results without loop closure are already promising, demonstrating the effectiveness of the local-context formulation.

  3. The method achieves a favorable trade-off between reconstruction accuracy, inference speed, and GPU memory usage, highlighting its potential for practical applications.

Critical weaknesses
  1. Remaining dependence on loop closure. The method still relies on an external loop-closure backend for global consistency in challenging cases. For example, in the Third Ring Road sequence, the trajectory fails to close the loop without this backend.

  2. Limited reconstruction performance. Despite strong pose estimation results, the method underperforms some existing baselines on 7Scenes and TUM-Dynamic, suggesting that the pose improvements do not consistently translate into better geometry reconstruction.

Questions for authors
  1. Could the authors explain why the local window is set to 12 frames, or provide an ablation showing how different window sizes affect accuracy and efficiency?

  2. The gains from loop closure suggest that long-range constraints remain important at larger scales. How do the authors view this in relation to learning global consistency within the network?

  3. The method underperforms some prior approaches on 7Scenes and TUM-Dynamic. Its strong pose results may primarily stem from the rotation refiner and composition-aware loss, while geometry quality remains less compelling. I would appreciate an explanation of the reconstruction performance gap and a discussion of potential solutions.

2 comments

W1. Dependence on Loop Closure.

We agree that the 48-km Third Ring Road trajectory does not close accurately without loop closure. This limitation arises because LC3R estimates and composes local relative poses without explicitly enforcing consistency between distant revisited locations. Our method focuses on drift mitigation, where composition-aware pose supervision and rotation refinement reduce local pose estimation errors and their sequential accumulation, rather than guaranteeing global loop consistency. In contrast, the optional loop-closure backend performs global drift correction by retrieving revisited frames, using LC3R itself to estimate their relative poses, and applying pose-graph optimization to enforce long-range consistency. We acknowledge that such global constraints remain necessary for accurate loop closure in challenging long trajectories. Nevertheless, our contribution lies in enabling accurate and scalable streaming reconstruction using only fixed local context, without persistent long-range memory, while the optional loop-closure backend provides complementary correction of residual global drift.


W2 & Q3. Reconstruction Performance.

LC3R achieves lower dense reconstruction accuracy than some baselines on 7Scenes and TUM-Dynamic. A plausible contributor is how LC3R selects its local context. We designed the model for ultra-long video streams and retain only the most recent 12 frames to keep memory use bounded. In the two compact indoor benchmarks, we identify two factors that may influence LC3R’s reconstruction performance: (1) dense frame sampling makes a fixed 12-frame window span a shorter temporal interval, which may favor methods that retain historical information over longer horizons, such as HorizonStream and TTT3R; (2) the compact spatial extent of these scenes often keeps subsequent camera positions close to the initial position, so the first frame may remain a useful geometric reference throughout the sequence, potentially benefiting methods with persistent first-frame anchors, such as LingBot-Map and StreamVGGT. Therefore, mild performance degradation is expected.

We identify three potential directions to improve performance in these scenarios. First, from a training perspective, the sampling strategy could be adapted to simulate dense observations in spatially confined environments. Second, from an architectural perspective, selectively retaining anchor frames or introducing an external scene-level state could provide persistent geometric references. Third, the model could learn an adaptive context update policy that slows down context updates when visual evidence is limited, thereby preserving useful historical information for longer.


Q1. Choice of the 12-Frame Window.

We empirically selected a 12-frame window to balance trajectory accuracy and inference efficiency. We conduct additional ablations with 6-, 12-, and 24-frame windows using the same baseline architecture, training settings, and budget. As shown in the table, the 12-frame window achieves the lowest ATE on both KITTI (56.60 m) and Oxford Spires (10.16 m) and performs best on five of the six pose metrics. This may be because shorter windows provide insufficient temporal context, whereas longer windows introduce additional, potentially less relevant information, making optimization more challenging under the same training budget. LingBot-Map reports a similar observation, with a bounded pose-reference window yielding lower ATE than full causal attention. Regarding inference efficiency, the 6-frame window provides only 1.06× the throughput of the 12-frame window, whereas the 24-frame window reduces throughput to 0.69×. These results support our choice of 12 frames, which achieves the lowest trajectory error among the evaluated settings while maintaining high inference throughput.

WindowKITTI ATE ↓KITTI RPE-r ↓KITTI RPE-t ↓Oxford ATE ↓Oxford RPE-r ↓Oxford RPE-t ↓
672.140.160.2116.760.290.04
12 (Ours)56.600.190.1710.160.190.03
2482.950.370.3517.660.430.05

Q2. Learned Consistency vs. Explicit Global Optimization.

We agree that long-range constraints are important for global consistency. A key advantage of explicit optimization, such as loop closure, is that it can be readily integrated into streaming reconstruction systems as a plug-and-play module to correct accumulated drift.

In contrast, learning global consistency within a streaming model is a more challenging problem. It first requires the model to maintain long-term information, involving dedicated mechanisms for memory compression and management. More importantly, as new evidence arrives, the model needs to retrieve and update relevant historical information to refine its global understanding. This requires architectural modifications that support iterative state updates or test-time optimization, rather than conventional immutable KV caching. Exploring emerging architectures such as Loop Transformers and Test-Time Training (TTT) may offer promising directions toward this goal.

LC3R currently focuses on local-context modeling. We believe extending it with learned long-range constraints can be a promising direction for future work.

Reviewer_wBW9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LC3R, a streaming 3D reconstruction model which handle large-scale scenes. Unlike previous methods which maintain the long-term memory, this paper uses a fixed 12-frame attention window and predicts geometry in the current camera coordinate system and motion between adjacent frames. Each chunk results are stitched into a long-term predictions with a light-weight rotation refiner. Experiments show strong long-sequence performance, particularly on KITTI and Oxford Spires, with optional loop closure providing further improvements.

Critical strengths
  1. The proposed method is simple but effective. Keeping prediction targets local avoids asking the network to directly estimate poses relative to an increasingly distant starting frame. And the rotation correction helps reduce accumulated errors.
  2. The results on the city-scale scenes are impressive. And the experiments demonstrate strong performance on long outdoor sequences. It also achieves good balance between accuracy, speed, and GPU memory.
Critical weaknesses
  1. I think the global drift remains a limitation because stitching the local predictions still accumulates errors. For example, on VBR, the proposed does not achieve the best ATE without loop closure, and the large improvement after adding the loop closure shows the continued importance of global constraints.
  2. The effect of context length is insufficiently explored. The main ablations use a fixed 12-frame window. Comparing shorter and longer windows would clarify how much temporal context is needed and whether the chosen window provides a good accuracy–efficiency trade-off.
Questions for authors
  1. How robust is LC3R to dropped frames or temporary occlusions? Can it recover accurate tracking after temporal continuity is disrupted?
  2. Will the authors release the internal datasets?
3 comments

W1. Global Drift and VBR Performance.

We agree that global drift remains a challenge for our method. This is a general challenge for streaming reconstruction, rather than a limitation unique to LC3R. Loop-closure is commonly used as a backend in streaming reconstruction and serves as a complementary component to LC3R. Specifically, LC3R and opitonal loop-closure reduce accumulated drift through two aspects: LC3R mitigates local dirft by introducing composition-aware pose loss and rotation refiner. In contrast, loop-closure corrects global accumulated drift by pose-graph optimization to enforce long-range consistency. We emphasize that LC3R achieves the best ATE among the compared streaming methods on two of the three long-horizon benchmarks without loop closure: KITTI (18.25 m vs. 22.51 m) and Oxford Spires (4.35 m vs. 7.32 m). On VBR, our ATE (30.14 m) is only slightly higher than HorizonStream (29.02 m, a 3.9% difference), while achieving identical RPE-R (0.60°) and RPE-t (0.04 m).


W2. The Effect of Context Length.

The 12-frame context length was empirically selected to balance reconstruction accuracy and inference efficiency. We evaluate different context lengths (6, 12, and 24 frames) using the same baseline architecture with identical training settings and budgets.

The experiments show that the 12-frame window achieves the best convergence and reconstruction accuracy among the evaluated configurations. A shorter window provides insufficient temporal information, while a longer window may introduce redundant or less relevant features, making optimization more challenging under the same training budget.

In terms of efficiency, the 6-frame window achieves only a 1.06$\times$ speedup over the 12-frame setting, whereas the 24-frame window reduces throughput to 0.69$\times$. These results demonstrate that 12 frames provide the best accuracy–efficiency trade-off among the evaluated settings.

WindowKITTI ATE ↓KITTI RPE-r ↓KITTI RPE-t ↓Oxford ATE ↓Oxford RPE-r ↓Oxford RPE-t ↓
672.140.160.2116.760.290.04
12 (Ours)56.600.190.1710.160.190.03
2482.950.370.3517.660.430.05

Q1. Robustness to Dropped Frames and Temporary Occlusions.

We conduct robustness experiments on two types of temporal disruptions: dropped frames and temporary occlusions. For the former, we randomly drop each input frame with a probability of 20% during inference. For the latter, we replace four consecutive frames with black images every 120 frames. The results are reported in Table below.

SettingOxford Spires (ATE ↓ / RPE-R ↓ / RPE-T ↓)KITTI (ATE ↓ / RPE-R ↓ / RPE-T ↓)
Clean input4.354 / 0.1189 / 0.0202918.297 / 0.0960 / 0.10400
20% random frame dropping3.911 / 0.1334 / 0.0251018.169 / 0.1182 / 0.19244
Periodic temporary occlusions4.787 / 0.8401 / 0.0362924.817 / 0.5659 / 0.20262

The results show that LC3R is robust to random frame dropping, with ATE remaining stable and even slightly improving over dense input. This is because our training already uses random interval sampling, making the model robust to varying frame intervals. Random frame dropping further aligns inference intervals with the training distribution, which can even improve performance. Temporary occlusions cause some degradation, but do not lead to tracking failure. A qualitative example illustrating this behavior is provided in the Kingsman sequence of our supplementary video.

Furthermore, we evaluate local pose estimation over short segments following each occlusion to examine the recovery capability. The results are reported in Table below.

SettingRPE-R (°) ↓RPE-T (m) ↓
Clean input0.123600.02884
Temporary occlusion0.123640.02885

The nearly identical local pose errors demonstrate that LC3R quickly recovers accurate tracking once visual observations become available again, without resetting its streaming state.


Q2. Releasing Internal Datasets.

We will consider releasing the internal datasets after further evaluation of their release feasibility and associated considerations.

Thank you for the additional ablations and experiments. These results help clarify the design choices and the model’s behavior under temporal disruptions. I will maintain my original rating

Thank you for your time, constructive feedback, and positive recognition of our work. We will incorporate all the additional ablations and robustness analyses into the final manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

As the discussion phase concludes, we would like to briefly summarize the current status of our submission:

  1. Reviewers' Consensus: All reviewers recognized the simplicity and effectiveness of our local formulation (wBW9, N1xz), our strong performance on both quality and efficiency (wBW9, w473, N1xz), and the validity of our ablation studies (w473, N1xz).

  2. Rebuttal Progress & Status:

  • Reviewer wBW9 confirmed maintaining their positive rating, satisfied with our responses regarding window size, robustness, and indoor benchmarks.
  • Reviewer w473 had all questions addressed in detail during the initial response.
  • Reviewer N1xz acknowledged the value of our ablations but raised further questions regarding novelty, memory designs, and scale drift. We promptly provided a follow-up response—including sequence-level scale stability statistics to address the drift concern, as well as clarifications on our holistic paradigm. We also welcomed any specific literature references to further distinguish our design.

All newly provided experiments, statistics, and discussions will be carefully incorporated into the final revision. We sincerely thank the AC and all reviewers for their dedicated time and constructive feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.