acceptodds
Under review as a conference paper at ICLR 2027

Along and Across the Ray: Post-Optimization for Feed-Forward Reconstruction

Abstract

Feed-forward 3D reconstruction models typically process long videos in windows and align their predictions across windows, yet reconstructions of the same surface can still form offset layers and produce surface ghosting. We show that these errors decompose into transverse and radial components in viewing-ray coordinates, where the transverse component is determined by the cameras and ray directions, while with fixed cameras, projection-preserving corrections can only adjust depth along the rays. Based on this structure, we propose RayCon, a training-free global-to-local post-optimization framework. Globally, RayCon exploits the dependence of reconstruction errors on window boundaries through staggered dual-window inference and fuses two trajectories with complementary errors. Locally, with the cameras fixed, Temporal–Range Consistency Filtering removes inconsistent predictions, while Surface-Consensus Refinement infers surface membership from multi-view consistency and refines depths along the viewing rays. Without retraining the backbone, RayCon reduces mean ATE on KITTI from 31.91m to 9.59m, achieves the lowest mean ATE among the compared methods on KITTI and Oxford Spires, and reduces local Chamfer distance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Reconstruction, Feed-Forward 3D

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3CPXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes RayCon, a training-free post-optimization framework for long-sequence feed-forward 3D reconstruction. The method decomposes reconstruction errors into transverse and radial components and addresses them through global-to-local optimization. Globally, staggered dual-window inference and trajectory fusion improve camera poses. Locally, Temporal–Range Consistency Filtering (TRCF) removes inconsistent predictions, while Surface-Consensus Refinement (SCR) refines depths using multi-view geometric constraints. Experiments using LingBot-Map as the backbone demonstrate substantial improvements in trajectory accuracy and surface quality on KITTI, Oxford Spires, and 7-Scenes.

While I do not see major issues with the paper's core contributions, the experimental validation remains insufficient. I encourage the authors to provide more comprehensive experiments to strengthen the paper's claims and overall credibility.

Critical strengths
  • The decomposition of reconstruction errors into transverse and radial components provides a clear geometric motivation for the proposed global-to-local optimization framework.

  • RayCon demonstrates promising reconstruction performance across multiple benchmarks without requiring additional training, highlighting its practical effectiveness as a post-optimization framework.

Critical weaknesses
  • All experiments rely on LingBot-Map, providing insufficient evidence of generalizability. Evaluations on other backbones, such as VGGT-Long, are necessary to support broader applicability claims.

  • Equal-weight trajectory fusion lacks reliability estimation, potentially degrading accurate trajectories when one reconstruction fails. Robustness under asymmetric tracking failures remains unexplored.

  • Table 9 evaluates SCR components on a single best-performing region, with most component replacements yielding identical results. Broader ablations are needed to establish their effectiveness.

  • Table 9 introduces component abbreviations that differ from the main text, making the ablation study difficult to interpret without carefully revisiting Section 3.4.

Questions for authors
  • Could the authors evaluate RayCon on other reconstruction backbones, such as VGGT-Long, to demonstrate its generalizability beyond LingBot-Map?

  • How does the proposed equal-weight trajectory fusion handle cases where one reconstruction suffers from severe tracking failures while the other remains accurate?

  • Could the authors provide broader ablation studies across multiple regions to clarify the effectiveness of individual SCR components, given that most replacements yield identical results? Additionally, could the authors clarify the correspondence between the component abbreviations in Table 9 and the terminology used in Section 3.4?

5 comments

We thank the reviewer for recognizing the geometric motivation behind our tangential–radial error decomposition and the effectiveness of RayCon in improving long-sequence reconstruction without retraining. We provide additional analyses and experiments addressing cross-backbone generalization, the reliability of equal-weight fusion, and SCR ablations.


W1 & Q1: Can RayCon generalize to other reconstruction backbones?

A: To eliminate the influence of backbone differences, we conduct controlled same-backbone comparisons on all 11 KITTI sequences (00–10).

  • Original VGGT backbone: Compared with VGGT-Long, adding RayCon post-optimization reduces mean ATE from 24.804 m to 19.629 m (20.86%), with error reductions of approximately 69% and 68% on large-scale sequences such as 01 and 08, respectively.
  • LongStream backbone: Adding our optimization to the LongStream + VGGT-Long baseline reduces mean ATE from 86.625 m to 66.926 m (22.74%).
  • LingBot-Map backbone: Compared with LingBot-Map + VGGT-Long, RayCon reduces mean ATE from 22.471 m to 9.588 m (57.33%).

These controlled experiments demonstrate that RayCon's improvements do not simply result from using a stronger backbone. Our global-to-local post-optimization consistently reduces mean trajectory error by 20.8%–57.3% across different visual geometry backbones.

Table 1: Cross-Backbone Evaluation on KITTI 00–10 (ATE ↓, m)

MethodMean ATE ↓0001020304050607080910
Native VGGT-Long (VGGT)24.80410.81666.19755.9385.5573.92310.1246.9492.82057.16031.17922.178
w/ Our Optimization19.62922.59720.57076.7304.6140.57119.4319.5814.79918.29928.57810.148
LongStream + native VGGT-Long86.625119.124132.143252.4994.6034.995133.118136.80526.37486.20826.14430.864
w/ Our Optimization66.92694.59268.202232.2144.5912.57085.26317.66913.92592.39497.91926.852
Native LingBot-Map31.9124.2483.6365.752.501.0274.777.519.8719.4542.5519.69
LingBot-Map + native VGGT-Long22.4717.63984.49668.2966.6452.6504.4655.9373.25236.92712.10514.769
RayCon (Ours)9.5883.52326.92331.6801.9030.9363.4102.3442.47116.54410.8944.839

W2 & Q2: How does dual-window fusion behave when one reconstruction fails severely while the other remains accurate?

A: Our dual-window fusion exploits complementary errors between staggered window partitions, reducing mean ATE from 11.074 m to 9.588 m across KITTI under normal conditions.

To examine asymmetric branch failures, we conduct two perturbation tests on KITTI 07. Under moderate perturbation, the affected branch obtains 9.825 m ATE, while fusion achieves 4.955 m, close to the intact branch (4.744 m). Under severe corruption, the affected branch degrades to 58.851 m, and fusion reaches 26.796 m.

Table 2: Dual-Window Fusion Under Asymmetric Branch Perturbations (KITTI 07)

ConfigurationClean Branch (90)Perturbed Branch (75)Fusion
Moderate Perturbation4.7449.8254.955
Severe Branch Failure4.74458.85126.796

These results demonstrate that dual-window fusion effectively exploits complementary estimates under moderate perturbations, while severe single-branch failures can propagate into the fused trajectory.

W3 & Q3: SCR effectiveness and identical ablation results.

A: Our post-optimization primarily aims to eliminate surface ghosting and layered artifacts introduced by multi-window reconstruction.

Global and local geometry improvements. In full-scene reconstruction, point-cloud quality and camera poses are strongly coupled. With our global optimization, the mean CD across all 11 KITTI sequences decreases from 9.0131 m to 2.4973 m, a 72.29% reduction, as shown in Table 4.

Local refinement. SCR applies only centimeter-scale corrections along viewing rays to ghosted points, leaving other points unchanged. Consequently, the mean CD across eight regions decreases only from 0.0973 m to 0.0938 m. However, in a representative ghosting region, F1@10 cm increases from 9.49% to 22.32%, as shown in Table 3. Such subtle point-cloud misalignments have limited impact on overall CD but directly cause visible surface layering and ghosting, making these improvements difficult to quantify. Figures 1, 3, and 4 visually demonstrate substantial removal of these artifacts. This shows that SCR specifically targets local ghosting that affects reconstruction quality.

Identical SCR ablation results. H determines the target surface, whereas D, J, and M constrain admissible corrections. Since the inferred depth adjustments already fall within these constraints, removing them produces nearly identical results in the evaluated region.

Table 3: Local Surface Deghosting Evaluation with Fixed Cameras

Evaluation SettingCD ↓ (m)F1@10 cm ↑
7-Scenes Heads, All 8 Regions0.0973 → 0.093843.40% → 45.61%
Table 9 Representative Ghosting Region0.1531 → 0.13199.49% → 22.32%
Table 9 Representative Region: Effect of H0.1339 → 0.131920.32% → 22.32%

Table 4: Global Dense Point-Cloud Evaluation with Joint Pose and Geometry Optimization on KITTI

Metric / Evaluation ScopeBase IntegrationOursImprovement
Mean CD ↓, All 11 Sequences (m)9.01312.4973−72.29%
Mean F1@10 cm ↑, All 11 Sequences4.48%6.52%+2.04 pp
CD ↓, KITTI 01 (m)60.96464.1960−93.12%
CD ↓, KITTI 02 (m)12.94716.5902−49.10%
CD ↓, KITTI 08 (m)6.04402.4740−59.07%
CD ↓, KITTI 10 (m)4.02361.9747−50.92%
CD ↓, KITTI 04 (m)5.03403.0126−40.16%

Q3: What do the component abbreviations in Table 9 represent, and how do they correspond to Section 3.4?

A: We will use consistent component names in Section 3.4 and Table 9: D — Directional Bounds, which constrain allowable geometric errors; J — Joint Track Compatibility, which enforces multi-view consistency within each track; H — Held-out Track Scoring, which selects the correct target surface; and M — Minimum Ray-Depth Change, which minimizes unnecessary point movements. We will also improve the method flowchart to clearly illustrate the surface selection and depth correction process.

As the discussion period is coming to an end, we would appreciate it if the reviewers could take a moment to review our rebuttal and additional experiments, and share any remaining concerns. We welcome further discussion and hope the new evidence can be considered in the final evaluation.

Reviewer_8RCcOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RayCon, a training-free post-optimization framework that improves the output of windowed feed-forward models on long videos, using a frozen LingBot-Map as the backbone. The authors decompose pointmap error into a transverse component perpendicular to the ray and a radial component along it, and note that once the cameras are fixed, only depth along the ray can change. The global stage smooths the camera trajectory within each window, runs the video twice with different window overlaps, and aligns and averages the two trajectories, all on top of the chunk alignment and loop closure of VGGT-Long. The local stage first filters out points contradicted by neighbouring frames. It then models observation uncertainty with ellipsoids elongated along the ray, uses held-out tracks to choose between one-plane and two-plane hypotheses, and corrects depth only along the ray through a second-order cone program. The experiments report trajectory error on KITTI and Oxford Spires and surface quality on local regions of 7-Scenes and KITTI.

Critical strengths

Without retraining the backbone, the mean ATE on KITTI drops from 31.91 m to 9.59 m and every sequence improves, which is of practical interest for deploying feed-forward reconstruction models. The local refinement is also carefully designed: selecting hypotheses on held-out tracks and applying a minimum-change convex correction is better grounded than simply averaging or deleting points.

Critical weaknesses

The main problem is that the large trajectory improvement does not come from the components this paper proposes. Table 4 shows that removing dual-window fusion only raises the KITTI mean ATE from 9.59 m to 11.07 m. Most of the drop from 31.91 m to about 11 m therefore comes from placing LingBot-Map inside the VGGT-Long chunk alignment and loop closure pipeline plus trajectory smoothing, all of which are existing methods. In Table 1, LingBot-Map is reported without loop closure, so comparing it directly with RayCon, which has loop closure, is not a like-for-like comparison. SCR, the most technically involved module, has no effect on trajectory metrics at all. To show what RayCon really contributes, the paper needs at least a LingBot-Map baseline with VGGT-Long loop closure and a separate ablation of smoothing. Moreover, most baseline numbers are copied from other papers, and the authors acknowledge that the protocols differ, so the claim of the lowest mean ATE is not convincing.

Second, the gains of the local stage are small and evaluated on a narrow set. Over eight regions, mean Chamfer distance only drops from 0.0973 m to 0.0938 m and F1 rises from 43.4% to 45.6%. Only one 7-Scenes region is shown in detail, and TRCF lowers recall from 16.4 to 14.2. The method depends on many hand-set thresholds, including ellipsoid size, distance bounds, depth-change limits, grazing-angle limits, several margins and voxel percentiles, and it only supports planar hypotheses, which the authors admit fail on curved and weakly textured regions. The radial and transverse decomposition is basic multi-view geometry, and the guarantee in Eq. 11 requires the true depths to lie in the feasible set, which cannot be verified in practice. Runtime and overhead are not given in the main text, even though the dual-window scheme runs the frontend at least twice.

Questions for authors
  1. What is the ATE of LingBot-Map inside the same VGGT-Long chunking and loop closure pipeline, without smoothing or dual-window fusion? Could you report results step by step, adding loop closure, then smoothing, then the second window? This question matters most for my assessment.
  2. Were the baselines in Tables 1 and 2 obtained with the same alignment, frame stride and evaluation protocol? Could you rerun VGGT-Long, LongStream, HorizonStream and VGGT-SLAM under a single protocol?
  3. How were the 90 and 75 overlaps and the SCR thresholds chosen, were they tuned on the test sequences, and how sensitive are the results to them?
  4. What are the runtime and memory cost of each stage relative to the backbone?
5 comments

We thank the reviewer for the rigorous and detailed assessment. We appreciate the recognition of the practical value of our method for long-sequence feed-forward reconstruction and the sound design of our local convex optimization. Your comments also highlight that our description of component contributions in the manuscript was insufficiently clear. We address these concerns directly with clarifications and additional experiments below.


W1a & Q1: Contributions

A: Our description in Section 3.2 was imprecise: we summarized global chunk alignment and loop closure as following VGGT-Long, which may have given the impression that the large ATE reduction from 31.91 m to 11.07 m was entirely due to the existing framework, with only a minor contribution from our method. We will clarify this distinction in the revision.

In fact, integrating LingBot-Map into the original VGGT-Long pipeline, including chunk alignment and loop closure, yields a mean ATE of 22.47 m, rather than approximately 11 m. Building on this baseline:

  • Ray-guided trajectory post-optimization (22.47 → 11.07 m): Our ray-guided post-optimization addresses transverse errors by suppressing within-window pose jitter and alignment residuals, reducing mean ATE by 50.72%, from 22.47 m to 11.07 m. This demonstrates the substantial contribution of our post-optimization.
  • Staggered dual-window fusion (11.07 → 9.59 m): While smoothing reduces within-window jitter, it cannot eliminate partition-dependent boundary errors. We therefore use staggered windows so that boundary frames in one partition become interior frames in another, providing complementary trajectory estimates. This further reduces mean ATE by 13.42%, from 11.07 m to 9.59 m.

This controlled comparison directly quantifies the gains contributed by each component. The step-by-step ablation results on all 11 KITTI sequences are shown below.

Table 1: Step-by-Step Ablation of Global Trajectory Optimization on KITTI 00–10

ConfigurationMean ATE ↓ (m)
Native LingBot-Map31.91
LingBot-Map + VGGT-Long (Chunk Alignment + Loop Closure), w/o Our Optimization22.471
w/ Our Ray-Guided Trajectory Smoothing Post-Optimization11.0738
w/ Staggered Dual-Window Fusion9.5879

Same-backbone comparisons across reconstruction architectures. To fully eliminate the influence of backbone differences, we conduct controlled same-backbone comparisons on all 11 KITTI sequences (00–10).

  • Same original VGGT backbone: Compared with the original VGGT-Long, adding RayCon post-optimization reduces mean ATE from 24.804 m to 19.629 m (20.86%), with error reductions of approximately 69% and 68% on long-distance, large-scale sequences such as 01 and 08, respectively.

  • Same LongStream backbone: Adding our optimization to the LongStream + VGGT-Long baseline similarly reduces mean ATE from 86.625 m to 66.926 m (22.74%).

  • Same LingBot-Map backbone: Compared with the LingBot-Map + VGGT-Long baseline (22.471 m), RayCon reduces mean ATE to 9.588 m (57.33%).

Together, these experiments demonstrate the strong effectiveness and methodological novelty of our proposed ray-guided global-to-local optimization framework, developed to address transverse and radial inconsistencies and surface ghosting in long-sequence 3D reconstruction.

Table 2: Same-Backbone Evaluation on KITTI 00–10 (ATE ↓, m)

MethodMean ATE ↓0001020304050607080910
Native VGGT-Long (VGGT)24.80410.81666.19755.9385.5573.92310.1246.9492.82057.16031.17922.178
w/ Our Optimization19.62922.59720.57076.7304.6140.57119.4319.5814.79918.29928.57810.148
LongStream + native VGGT-Long86.625119.124132.143252.4994.6034.995133.118136.80526.37486.20826.14430.864
w/ Our Optimization66.92694.59268.202232.2144.5912.57085.26317.66913.92592.39497.91926.852
Native LingBot-Map31.9124.2483.6365.752.501.0274.777.519.8719.4542.5519.69
LingBot-Map + native VGGT-Long22.4717.63984.49668.2966.6452.6504.4655.9373.25236.92712.10514.769
RayCon (Ours)9.5883.52326.92331.6801.9030.9363.4102.3442.47116.54410.8944.839

W1b & Q2: Baseline evaluation protocols.

A: RayCon uses the same windowing parameters, trajectory alignment method, and ATE computation in Tables 1 and 2. Input sampling is dataset-specific: stride 1 for KITTI and stride 12 for Oxford Spires. We previously reproduced most baselines and obtained results close to those reported in their original papers. To maintain consistency with the published comparisons, we retained the officially reported baseline values. We will include both the published and reproduced results in the revision.

We additionally reran VGGT-Long, HorizonStream, LongStream, and VGGT-SLAM on KITTI under a unified input sampling, trajectory alignment, and ATE evaluation protocol, using stride 1. The table below reports results on all 11 KITTI sequences. Under this unified protocol, RayCon achieves the lowest mean ATE of 9.588 m.

Table 3: Unified-Protocol Evaluation on KITTI 00–10 (ATE ↓, m)

MethodMean ATE ↓0001020304050607080910
VGGT-Long24.8010.8266.2055.945.563.9210.126.952.8257.1631.1822.18
HorizonStream17.8714.7123.0971.335.621.839.856.234.0022.1824.8412.84
LongStream65.01105.0159.73146.7215.9914.1398.9937.4028.2674.9198.7335.20
VGGT-SLAM108.23151.95149.49315.4132.2429.2856.3742.5141.48263.1963.7944.77
RayCon (Ours)9.593.5226.9231.681.900.943.412.342.4716.5410.894.84

W2a & Q3a: Parameter Selection and Sensitivity

A: All hyperparameter settings were determined before evaluation, without tuning on the test sequences. The trajectory smoothing weight λ\lambda is automatically selected through generalized cross-validation (GCV), while the reprojection tolerance τ=3.954\tau=3.954 px is calibrated using the 90th percentile of reprojection errors from independent calibration tracks. The remaining SCR thresholds were estimated from observed offsets between ghosted surface layers and fixed as specified in Appendix Table 5. We further tested 0.5× and 2× variations of the main SCR thresholds, with relatively stable CD and F1 results, as shown in Table B.

For dual-window inference, we selected 150-frame windows with overlaps of 90 and 75 frames to introduce staggered boundaries. Table A evaluates six alternative window configurations across all 11 KITTI sequences. Our default single-window configuration is not the best-performing one, yet all tested configurations substantially outperform the VGGT-Long baseline (22.47 m), with mean ATE ranging from 10.02 m to 12.01 m. Dual-window fusion further reduces mean ATE to 9.59 m, demonstrating that our improvements do not depend on a narrowly optimized window configuration.

Table A: Window Parameter Sensitivity on KITTI 00–10

ConfigurationWindow / OverlapMean ATE ↓ (m)
VGGT-Long Baseline150 / 9022.471
Single-Window 1150 / 7511.3369
Single-Window 2 (Default)150 / 9011.0738
Single-Window 3150 / 10510.0170
Single-Window 4150 / 11810.5256
Single-Window 5165 / 10511.3005
Single-Window 6300 / 7512.0146
RayCon (Dual-Window)150 / 75 + 150 / 909.5879

Table B: SCR Threshold Sensitivity on Eight Fixed Regions

SCR SettingCD ↓ (m)F1@10 cm ↑
Input (w/o SCR)0.097343.40%
κ=1\kappa=10.094944.70%
κ=4\kappa=40.097343.40%
η=0.0125ρ0\eta=0.0125\rho^00.094644.90%
η=0.05ρ0\eta=0.05\rho^00.092945.92%
Δ=0.075\Delta=0.0750.094744.80%
Δ=0.30\Delta=0.300.093445.67%
Margin = 0.0150.093545.82%
Margin = 0.060.094445.10%
Default SCR0.093845.61%

W2 & Q3b: Geometric Consistency, Surface Deghosting, and SCR Ablation Results

A: Our method primarily aims to eliminate surface ghosting introduced by multi-window reconstruction.

  • Global and local geometry improvements: In full-scene reconstruction, point-cloud quality and camera poses are strongly coupled. With our global optimization, the mean CD across all 11 KITTI sequences decreases from 9.0131 m to 2.4973 m, a 72.29% reduction, as shown in Table D.

  • Local refinement: SCR applies only centimeter-scale corrections along viewing rays to ghosted points, leaving other points unchanged. Consequently, the mean CD across eight regions decreases only from 0.0973 m to 0.0938 m. However, in a representative ghosting region, F1@10 cm increases from 9.49% to 22.32%, as shown in Table C. Such subtle point-cloud misalignments have limited impact on overall CD but directly cause visible surface layering and ghosting, making these improvements difficult to quantify. Figures 1, 3, and 4 visually demonstrate substantial removal of these artifacts. This shows that SCR specifically targets local ghosting that affects reconstruction quality.

  • Identical SCR ablation results: H determines the target surface, whereas D, J, and M constrain admissible corrections. Since the inferred depth adjustments already fall within these constraints, removing them produces nearly identical results in the evaluated region.

  • TRCF filtering trade-off: TRCF removes temporally inconsistent points, reducing recall from 16.43% to 14.20%, while improving CD from 4.112 m to 3.861 m and F1@10 cm from 4.03% to 4.40%. This reflects the intended trade-off between point retention and geometric consistency.

Table C: Local Surface Deghosting with Fixed Cameras

Evaluation SettingCD ↓ (m)F1@10 cm ↑
7-Scenes Heads, All 8 Regions0.0973 → 0.093843.40% → 45.61%
Table 9 Representative Region0.1531 → 0.13199.49% → 22.32%
Table 9: Effect of Held-out Validation0.1339 → 0.131920.32% → 22.32%

Table D: Global Dense Reconstruction Quality on KITTI

Metric / Evaluation ScopeBaselineOursImprovement
Mean CD ↓, All 11 Sequences (m)9.01312.4973−72.29%
Mean F1@10 cm ↑, All 11 Sequences4.48%6.52%+2.04 pp
CD ↓, KITTI 01 (m)60.96464.1960−93.12%
CD ↓, KITTI 02 (m)12.94716.5902−49.10%
CD ↓, KITTI 08 (m)6.04402.4740−59.07%
CD ↓, KITTI 10 (m)4.02361.9747−50.92%
CD ↓, KITTI 04 (m)5.03403.0126−40.16%

Q4: Runtime and GPU memory overhead relative to the backbone.

A: We measured runtime and GPU memory usage on KITTI using an RTX 5090 GPU. Native LingBot-Map, native VGGT-Long, LingBot-Map + VGGT-Long, our single-window optimization, and full RayCon achieve 34.0, 17.2, 21.3, 19.3, and 11.9 FPS, respectively, with peak GPU memory usage of 31, 28, 20, 20, and 20 GB.

Since the two window partitions are processed sequentially with model caches cleared between windows, full RayCon introduces no additional peak GPU memory usage compared with our single-window configuration. Local post-optimization runs on the CPU and adds no GPU memory overhead. The main additional cost is inference time, while full RayCon still achieves 11.9 FPS.

Table 4: Runtime and GPU Memory Overhead on KITTI (RTX 5090)

MethodFPS ↑Peak GPU Memory
Native LingBot-Map34.031 GB
Native VGGT-Long17.228 GB
VGGT-Long (LingBot-Map)21.320 GB
Our Single-Window Optimization (150/90)19.320 GB
RayCon (150/75 + 150/90)11.920 GB
Reviewer_wt4FOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RayCon, a training-free optimization framework for long-sequence feed-forward 3D reconstruction. It improves camera trajectories using trajectory smoothing and staggered dual-window inference, and temporal-range consistency filtering and surface-consensus refinement to reduce inconsistent geometry. Experimental results show improvements in trajectory accuracy and geometric consistency.

Critical strengths
  1. The paper addresses an important problem in long-sequence feed-forward reconstruction. The decomposition of traverse and radial components provides a clear motivation for the proposed approach.

  2. The method achieves substantial performance improvements in camera trajectory accuracy without retraining the backbone. The paper also provides experiments on local geometric refinement and ablation studies.

Critical weaknesses

1.The paper substantially improves trajectory accuracy over Lingbot-map (31.91 m to 9.59 m), but its global optimization builds on the existing VGGT-Long framework. While the ablation demonstrates the benefit of dual-window fusion, it does not isolate the effect of trajectory smoothing or the existing global optimization components. Moreover, the comparison with VGGT-Long uses different backbones, making it difficult to assess the improvement from the proposed optimization method itself. A comparison using the same backbone would help clarify the contribution of the proposed components.

  1. Although reducing surface ghosting is a central motivation, the improvements in local geometric accuracy are relatively small and are evaluated on only eight regions from 7-Scenes Heads. Furthremore, several SCR component ablations produce identical results, making it difficult to determine the contribution of individual design choices.
Questions for authors
  1. Can the authors provide an ablation that separates the effects of the existing VGGT-Long global optimization framework, trajectory smoothing, and the proposed dual-window fusion?

  2. Can the authors compare RayCon with VGGT-Long using the same backbone to better isolate the contribution of the proposed optimization method?

  3. Why do several SCR components ablations produce identical results?

4 comments

We thank the reviewer for recognizing the importance of long-sequence feed-forward 3D reconstruction, the clear motivation provided by transverse–radial error decomposition, and the substantial trajectory accuracy improvements. We also appreciate the reviewer's insightful concerns regarding the independent contributions of individual components, the evaluation of local deghosting, and the interpretation of SCR ablation results. Below, we address each of the reviewer's concerns in detail.


W1a & Q1: Contributions.

A: In Section 3.2, our brief description of following the VGGT-Long divide-and-conquer framework did not clearly distinguish the original VGGT-Long alignment pipeline from our ray-guided global-to-local optimization mechanism specifically designed based on ray-error analysis. We will explicitly clarify this distinction in the revised manuscript.

To precisely quantify the independent contribution of each component, we conduct controlled experiments on all 11 KITTI sequences, fixing the LingBot-Map weights, inputs, and evaluation protocol. (Note that the rerun baseline results differ slightly from the third-party reported values directly cited in Table 1.)

  1. Ray-guided trajectory post-optimization (22.47 → 11.07 m): Starting from the 22.47 m VGGT-Long baseline, we introduce our Ray-Guided trajectory smoothing post optimization, designed to address transverse error characteristics. These effectively suppress within-window pose jitter and alignment residuals, substantially reducing mean ATE by 50.72% to 11.07 m. This demonstrates the significant contribution of our proposed post-optimization module.

  2. Staggered dual-window fusion (11.07 → 9.59 m): Although within-window smoothing suppresses internal jitter, it cannot eliminate window-boundary truncation errors that depend on window partitioning. We therefore introduce staggered dual-window inference, placing boundary frames from one partition inside the windows of another to obtain complementary trajectory estimates. This further reduces mean ATE by 13.42%, from 11.07 m to 9.59 m. This controlled comparison directly quantifies the gains contributed by each component.

Table 1: Step-by-Step Ablation of Global Trajectory Optimization on KITTI 00–10

ConfigurationMean ATE ↓ (m)
Native LingBot-Map31.91
LingBot-Map + VGGT-Long (Chunk Alignment + Loop Closure)22.471
+ Our Ray-Guided Trajectory Smoothing Post-Optimization11.0738
+ Staggered Dual-Window Fusion9.5879

W1b & Q2: Same-backbone comparison between RayCon and VGGT-Long.

A: To fully eliminate the influence of backbone differences, we conduct controlled same-backbone comparisons on all 11 KITTI sequences (00–10).

  • Same original VGGT backbone: Compared with the original VGGT-Long, adding RayCon post-optimization reduces mean ATE from 24.804 m to 19.629 m (20.86%), with error reductions of approximately 69% and 68% on long-distance, large-scale sequences 01 and 08, respectively.

  • Same LongStream backbone: Adding our optimization to the LongStream + VGGT-Long baseline similarly reduces mean ATE from 86.625 m to 66.926 m (22.74%).

  • Same LingBot-Map backbone: Compared with the LingBot-Map + VGGT-Long baseline (22.471 m), RayCon reduces mean ATE to 9.588 m (57.33%).

These three experiments consistently demonstrate that RayCon's performance gains do not simply result from replacing the backbone with a stronger model. Our global-to-local post-optimization mechanism delivers substantial and consistent trajectory accuracy improvements of 20.8%–57.3% across different types of visual geometry backbones.

Table 2: Same-Backbone Evaluation on KITTI 00–10 (ATE ↓, m)

MethodMean ATE ↓0001020304050607080910
Native VGGT-Long (VGGT)24.80410.81666.19755.9385.5573.92310.1246.9492.82057.16031.17922.178
w/ Our Optimization19.62922.59720.57076.7304.6140.57119.4319.5814.79918.29928.57810.148
LongStream + native VGGT-Long86.625119.124132.143252.4994.6034.995133.118136.80526.37486.20826.14430.864
w/ Our Optimization66.92694.59268.202232.2144.5912.57085.26317.66913.92592.39497.91926.852
Native LingBot-Map31.9124.2483.6365.752.501.0274.777.519.8719.4542.5519.69
LingBot-Map + native VGGT-Long22.4717.63984.49668.2966.6452.6504.4655.9373.25236.92712.10514.769
RayCon (Ours)9.5883.52326.92331.6801.9030.9363.4102.3442.47116.54410.8944.839

W2 & Q3: Local refinement, global–local coupling, and identical SCR ablation results.

A: Our method primarily aims to eliminate surface ghosting introduced by multi-window reconstruction.

1. Global and local geometry improvements. In full-scene reconstruction, point-cloud quality and camera poses are strongly coupled. With our global optimization, the mean CD across all 11 KITTI sequences decreases from 9.0131 m to 2.4973 m, a 72.29% reduction, as shown in Table 4.

2. Role of SCR. SCR applies only centimeter-scale corrections along viewing rays to ghosted points, leaving other points unchanged. Consequently, the mean CD across eight regions decreases only from 0.0973 m to 0.0938 m. However, in a representative ghosting region, F1@10 cm increases from 9.49% to 22.32%, as shown in Table 3. Such subtle point-cloud misalignments have limited impact on overall CD but directly cause visible surface layering and ghosting, making their improvement difficult to quantify. Figures 1, 3, and 4 visually demonstrate substantial removal of these artifacts. This shows that SCR specifically targets local ghosting that affects reconstruction quality.

3. Identical SCR ablation results. H determines the target surface, whereas D, J, and M constrain admissible corrections. Since the inferred depth adjustments already fall within these constraints, removing them produces nearly identical results in the evaluated region.

Table 3: Local Surface Deghosting Evaluation with Fixed Cameras

Evaluation SettingCD ↓ (m)F1@10 cm ↑
7-Scenes Heads, all 8 regions0.0973 → 0.093843.4% → 45.6%
Table 9 representative ghosting region0.1531 → 0.13199.49% → 22.32%
Table 9 representative region: effect of H0.1339 → 0.131920.32% → 22.32%

Table 4: Global Dense Point-Cloud Evaluation with Joint Pose and Geometry Optimization on KITTI

Metric / Evaluation ScopeBase IntegrationOursImprovement
Mean CD ↓, all 11 sequences (m)9.01312.4973−72.29%
Mean F1@10 cm ↑, all 11 sequences4.48%6.52%+2.04 pp
CD ↓, KITTI 01 (m)60.96464.1960−93.12%
CD ↓, KITTI 02 (m)12.94716.5902−49.10%
CD ↓, KITTI 08 (m)6.04402.4740−59.07%
CD ↓, KITTI 10 (m)4.02361.9747−50.92%
CD ↓, KITTI 04 (m)5.03403.0126−40.16%

Thank you to the authors for the additional experiments. The new ablations address my concerns regarding trajectory optimization. However, Table 2 indicates that for Native VGGT-Long w/ optimization, the mean improvement is largely driven by a few sequences with large errors. In addition, SCR's local geometric accuracy is evaluated only on the 7-Scenes Heads, which remains a concern regarding generalizability.

Thank you for your further feedback.

The varying improvements across sequences reflect differences in backbone error characteristics and their interactions with existing global optimization. Importantly, accumulated drift remains a major bottleneck for deploying feed-forward 3D reconstruction in real-world long-sequence scenarios, making substantial gains on challenging sequences particularly valuable. RayCon also improves all 11 KITTI sequences with LingBot-Map and 9 of 11 with LongStream, demonstrating effectiveness beyond a few sequences or a specific backbone.

RayCon is a complete global-to-local reconstruction optimization framework. Across all 11 KITTI sequences, it reduces mean ATE by 57.33% and global point-cloud CD by 72.29%. The fixed-pose evaluation on 7-Scenes independently validates SCR's effectiveness in reducing local surface ghosting. Together, these results demonstrate substantial improvements in both trajectory accuracy and geometric consistency, two critical requirements for practical long-sequence reconstruction.

Given these significant gains achieved without retraining, and that the additional experiments have addressed the primary concerns regarding trajectory optimization, we hope the reviewer will reconsider the rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.