- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LC3R, a streaming 3D reconstruction model that uses only the most recent 12 frames. Built upon π³, it predicts adjacent-frame relative poses and local point maps, then recovers global reconstruction through sequential pose composition. A Rotation Refiner and Composition-aware Loss are introduced to reduce accumulated drift. The authors aim to show that accurate and efficient long-horizon reconstruction can be achieved without persistent long-term memory.
Critical strengths
- The approach is simple and avoids complex long-term memory management, offering a lightweight alternative for streaming reconstruction.
- The experimental results are competitive. On Oxford Spires, ATE decreases from 8.81 m (HorizonStream) to 4.35 m, and on KITTI, from 22.51 m to 18.25 m.
- The effectiveness of Composition-aware Loss is worth noting. In Table 4, adding this loss alone reduces KITTI ATE from 56.60 m to 27.66 m.
- The method achieves relatively high inference efficiency, reaching 24.45 FPS with 6.71 GB of GPU memory.
Critical weaknesses
-
Limited methodological originality. Local-to-global pose recovery has already been explored in methods such as SLAM3R and Anchor3R, while HorizonStream combines local and long-term context for long-horizon reconstruction. In comparison, LC3R mainly differs by using a fixed local window, directly predicting adjacent-frame relative poses, and composing them sequentially, but this idea itself is not new. The Rotation Refiner relies on standard temporal feature fusion and residual rotation correction, while the Composition-aware Loss follows the established approach of multi-frame pose supervision. Overall, the paper appears to be more of a simplification and engineering combination of existing streaming reconstruction frameworks, rather than introducing a distinct core technical contribution.
-
The experiments do not establish that local context is responsible for the performance gains. In Table 4, the baseline achieves 56.60 m ATE on KITTI, which drops to 27.66 m simply by adding Composition-aware Loss. On Oxford Spires, the largest improvement comes from longer-sequence training. It is somewhat counterintuitive that retaining less historical information would lead to better reconstruction accuracy, and this claim requires more direct evidence and explanation. The authors lack a direct comparison between local and long-term context under the same backbone, training conditions, and loss functions. The current results appear to demonstrate the effectiveness of additional supervision and training strategies rather than the superiority of local context over long-term memory.
-
Concerns about comparison fairness and long-horizon stability. LC3R uses 30 training datasets, including 7.71% internal data, along with a different pretrained model and longer training sequences. The efficiency comparisons also use different input resolutions. Moreover, without loop closure, LC3R performs slightly worse than HorizonStream on VBR. These factors make it difficult to determine whether the reported advantages consistently hold in longer and more challenging scenarios.
Questions for authors
-
Why would retaining less historical information lead to better results? This seems counterintuitive, as long-term information generally helps maintain geometric consistency. Can the authors explain why long-term memory might instead be detrimental and provide direct experimental evidence?
-
How is long-term scale drift handled? When adjacent poses are continuously composed, scale errors may also accumulate. Without explicit long-term scale constraints, how does the model maintain scale consistency across different parts of a sequence? Could Sim(3)-aligned ATE conceal some of these scale-related errors?
-
Are the improvements mainly due to the local architecture or Composition-aware Loss? Table 4 shows that the latter accounts for a large portion of the improvement. If the same loss and rotation refinement were applied to other existing methods, would they achieve similar gains?
-
What fundamentally distinguishes LC3R from Anchor3R, SLAM3R, and HorizonStream? Beyond reducing the context window, changing the pose prediction formulation, and adding two components, what problem does this method solve that existing approaches cannot?
-
How can the effects of training data and computational settings be ruled out? Can the authors provide comparisons under matched training data, pretrained backbones, and input resolutions? How much do the internal datasets contribute to the final results?
5 comments
Responses to Reviewer N1xz (2/3)
Q: Does the Local Architecture independently contribute to performance gains?
Our Local Architecture consists of two components: Adjacent-relative Pose Prediction and Local Context.
First, comparing experiments (1) and (2), while keeping the first-frame anchor and sliding-window context unchanged, replacing global pose prediction with adjacent-frame relative pose prediction reduces KITTI ATE from 230.02 m to 196.08 m and Oxford ATE from 35.06 m to 27.54 m. This demonstrates that local pose prediction can improve geometric estimation.
Second, comparing experiments (2) and (3), while keeping adjacent-frame pose prediction unchanged, removing the persistent first-frame context further reduces KITTI ATE substantially from 196.08 m to 56.60 m and Oxford ATE from 27.54 m to 10.16 m. This demonstrates that strictly local context is also an important source of performance gains.
Both architectural modifications bring substantial improvements, demonstrating the importance of the complete Local Architecture for achieving high-quality long-horizon streaming reconstruction under a limited training budget.
Q: Why would retaining less historical information lead to better results?
First, comparing experiments (2) and (3), we find that retaining additional first-frame context significantly degrades performance even with adjacent-frame relative pose prediction. We hypothesize that, since the first frame is persistently available as a fixed reference, the model may develop an implicit dependency on this distant anchor, which reduce its Long-horizon extrapolation.
Second, comparing experiments (3) and (4), directly extending the window from 12 to 24 frames also degrades performance. KITTI ATE increases from 56.60 m to 82.95 m, while Oxford ATE increases from 10.16 m to 17.66 m. We hypothesize that although a longer window provides more historical information, it also introduces additional features that may be less relevant to the current adjacent-frame pose prediction task, increasing the complexity of feature fusion and optimization.
Meanwhile, comparing experiments (3) and (5), shortening the window to 6 frames also degrades performance, indicating that an overly short context cannot provide sufficient local temporal information.
Overall, our choice of a 12-frame window is not simply driven by the goal of retaining less historical information, but rather by matching the context length to the local geometric prediction task. Insufficient context limits the information available to the model, while blindly incorporating more historical information does not necessarily bring benefits and may instead increase optimization difficulty. These results support our choice of a fixed local context.
Q: Are the improvements mainly due to the Local Architecture or Composition-aware Loss?
Adding Composition-aware Loss to the original baseline yields experiment (6). KITTI ATE decreases from 56.60 m to 27.66 m, while Oxford ATE decreases from 10.16 m to 9.40 m, demonstrating that composition-aware supervision provides additional performance gains.
Combining this result with the preceding architectural ablations, we observe that, even without Composition-aware Loss, local pose prediction and local context alone reduce KITTI ATE from 230.02 m to 56.60 m and Oxford ATE from 35.06 m to 10.16 m. This demonstrates that the Local Architecture itself already provides substantial performance improvements, while Composition-aware Loss further enhances the reliability of multi-step pose composition.
Therefore, we consider the Local Architecture to be the more significant source of performance improvements, while Composition-aware Loss provides complementary gains.
Responses to Reviewer N1xz (3/3)
Q: How transferable are the additional components?
Regarding the transferability of our additional components, the Rotation Refiner is specifically designed for adjacent-frame pose prediction. Its input feature fusion and residual prediction mechanisms are tailored to this formulation, making it difficult to directly apply to other pose prediction architectures.
Although Composition-aware Loss has some degree of generality, its effectiveness also depends on the underlying pose prediction formulation. To further investigate this, we conduct an additional experiment based on the original +Comp. Loss variant. We further experimented
with an Anchor3R-like multi-reference pose fusion formulation under the same composition-aware
supervision, but observed degraded performance. This may because jointly optimizing transforma-
tions from the current frame to multiple reference frames introduces competing optimization paths,
which may be further amplified by composition-aware supervision. Therefore, these components are
not necessarily plug-and-play improvements and should be considered together with the underlying architecture.
| Method | KITTI ATE ↓ | KITTI RPE-R ↓ | KITTI RPE-T ↓ | Oxford Spires ATE ↓ | Oxford Spires RPE-R ↓ | Oxford Spires RPE-T ↓ |
|---|---|---|---|---|---|---|
| Anchor3R-style | 99.23 | 0.43 | 0.33 | 26.38 | 0.55 | 0.07 |
| Ours | 27.66 | 0.14 | 0.13 | 9.40 | 0.15 | 0.03 |
W3 & Q5. Comparison Fairness and the Role of Internal Data.
Strictly matching training data, pretrained backbones, and computational settings across methods is challenging, particularly when complete training code and configurations are unavailable. Besides, both LingBot-Map and HorizonStream also use internal data in their training mixtures, which makes it difficult to reproduce their exact training sets.
Our training mixture is comparable in the number of datasets to those of LingBot-Map and HorizonStream, and our sampling strategy draws on their reported data compositions. We build LC3R on the π³ architecture and initialize it with the released π³ weights because its reference-view-free design suits adjacent-frame relative pose prediction, while its performance is comparable to that of VGGT. Starting from these weights, LC3R achieves competitive performance with 74K optimization steps (32K + 38K + 4K). For context, LingBot-Map reports 320K steps across base-model and streaming training, HorizonStream reports 100K, and Anchor3R reports 80K.
We include internal data primarily to expose LC3R to a wider range of scenes and encourage generalization to unseen environments. These data account for 7.71% of the sampling distribution in our full training setup. To assess their contribution to benchmark performance, we trained two LC3R baselines in a reduced-data setting, keeping both the number of public datasets and the amount of public data the same across runs. Adding internal data reduced KITTI ATE by 1.02 m, from 23.61 to 22.59 m, while Oxford Spires ATE increased by 0.02 m, from 7.38 to 7.40 m. The measured effect was modest on these two benchmarks.
| Training data | KITTI ATE (m) ↓ | KITTI RPE-R (°) ↓ | KITTI RPE-T (m) ↓ | Oxford Spires ATE (m) ↓ | Oxford Spires RPE-R (°) ↓ | Oxford Spires RPE-T (m) ↓ |
|---|---|---|---|---|---|---|
| Public datasets | 23.61 | 0.17 | 0.11 | 7.38 | 0.38 | 0.02 |
| Public datasets + internal data | 22.59 | 0.17 | 0.10 | 7.40 | 0.38 | 0.02 |
Q2. Scale Drift and Sim(3) Alignment.
We address scale consistency through both training and prediction formulation. During training, we apply a unified scale normalization to each entire 32/128-frame sequence rather than normalizing individual frames independently, encouraging consistent scale estimation across the sequence. Furthermore, multi-dataset training enables the model to internalize stable scale priors across diverse scenes, keeping its scale estimates within a relatively consistent range.
More importantly, unlike predicting poses relative to an increasingly distant initial frame, LC3R predicts adjacent-frame relative poses, keeping the prediction target local and independent of the inference horizon. As shown in Fig.~6(a), LC3R maintains stable adjacent-frame translational errors throughout KITTI-02.
Regarding Sim(3)-aligned ATE, we acknowledge that it can indeed conceal certain scale-related errors. Feed-forward 3D reconstruction models typically do not define an absolute scale. Therefore, unified Sim(3) alignment to remove global scale differences is a standard evaluation protocol in this field.
Responses to Reviewer N1xz (1/3)
Our central contribution is to show that accurate long-horizon dense reconstruction can be achieved by composing a single chain of learned adjacent-frame poses. At each step, LC3R predicts the current frame’s pose relative to the preceding frame and a dense point map in the current camera coordinates. Sequential pose composition places each point map in the global frame. This keeps the network’s prediction task the same throughout the sequence, regardless of how far the camera has traveled. The rotation refiner corrects each relative rotation before composition, and the composition-aware pose loss supervises the motion obtained by composing several consecutive predictions.
The distinction from prior work lies in how the global reconstruction is recovered. SLAM3R learns to register keyframe-centered point maps to the existing scene. Anchor3R and HorizonStream fuse pose information from multiple frames within a window. LC3R directly composes one predicted transform per frame. With this formulation, LC3R achieves the lowest ATE among the compared streaming methods without loop closure on KITTI (18.25 m) and Oxford Spires (4.35 m), and runs at 24.45 FPS with 6.71 GB of GPU memory on KITTI-02.
We summarize the reviewer's concerns regarding the Local Architecture, including its effectiveness, the source of performance improvements, the relationship between Local Context and reconstruction accuracy, and the transferability of additional components. To better clarify the rationale and contributions of each architectural component, we conduct a series of ablation studies and address these concerns individually.
Our Local Architecture consists of two key designs: Adjacent-relative Pose Prediction and Local Context. To disentangle their contributions, we construct several model variants based on the original ablation baseline:
All architectural ablations use the same backbone, training configurations, and training budget. Except for experiment (6), which specifically evaluates Composition-aware Loss, all other experiments use the same baseline supervision without Composition-aware Loss, ensuring fair architectural comparisons.