acceptodds
Withdrawn from ICLR 2027

MV3R: Long-Sequence Multi-View 3D Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reconstructing 3D scenes from long-horizon multi-camera streams requires processing thousands of images without sacrificing global geometric consistency. Existing feed-forward methods, however, typically process all input views in a single forward pass and therefore scale poorly to long sequences, while streaming methods rely on a bounded state that may discard fine-grained cross-view evidence and accumulate drift over time. To overcome this trade-off, we introduce MV3R, a scalable window-based framework that processes a sequence as overlapping local windows and propagates context through their shared observations, thereby preserving continuity without jointly processing the entire sequence. Building on this windowed design, MV3R incorporates patch-level Plücker ray encodings to expose the spatial configuration of the fixed camera array and improve reconstruction within each window. The shared observations between adjacent windows then serve a second purpose by establishing deterministic correspondences for connecting local reconstructions. To robustly exploit these correspondences, a fixed-capacity scene memory predicts their reliability and guides a two-stage Sim(3) registration that performs confidence-weighted coarse alignment followed by residual refinement. Together, these designs enable MV3R to scale to long multi-camera sequences with bounded memory while bringing all local reconstructions into a geometrically consistent global coordinate system. To address the lack of benchmarks for indoor long-sequence reconstruction with fixed multi-camera arrays, we further introduce MV3D for evaluating geometric reconstruction quality and long-term trajectory stability. Extensive experiments show that MV3R significantly outperforms existing methods in reconstruction and trajectory estimation across four indoor datasets. Specifically, relative to the strongest competing result for each reconstruction metric on each test subset, MV3R reduces point-cloud Accuracy and Completeness errors by 22.5% and 22.1% on average, respectively. For trajectory estimation, MV3R reduces average ATE from 0.204 for the strongest baseline to 0.095, corresponding to an approximately 53.4% reduction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Reconstruction, Multi-View Reconstruction, Streaming 3D Reconstruction, Long-Sequence Reconstruction, Visual Geometry

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AWJVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MV3R, a window-based framework for long-sequence 3D reconstruction from synchronized multi-camera streams. The method combines overlapping feature propagation and array-aware Plücker-ray encoding for local reconstruction with memory-conditioned correspondence weighting and residual Sim(3) correction for inter-window alignment. The authors also introduce MV3D, an indoor multi-camera benchmark. Experiments demonstrate improvements in geometric reconstruction and long-sequence trajectory estimation, with ablations supporting the benefits of array-aware encoding and scene memory.

Critical strengths
  1. Coherent framework. MV3R integrates camera-array geometry, overlapping observations, and scene memory for streaming reconstruction.

  2. Strong empirical results. MV3R leads on 11 of 12 reconstruction metrics and all eight long-sequence trajectories.

Critical weaknesses
  1. Long-sequence reconstruction and efficiency remain insufficiently validated. Table 1 evaluates reconstruction on only the first 50 time steps, while Table 2 assesses trajectory accuracy rather than reconstruction quality. A direct comparison of long-sequence reconstruction against baselines is still missing. Given that scalability is a central motivation, reporting runtime, latency, or GPU memory would also help.

  2. The contribution of long-term memory is not yet well isolated. While Sec. 3.3.1 defines local and global memory branches, Table 3(b) only compares with vs. without scene memory, without removing either branch separately. Weighted alignment (Sec. 3.3.2) and residual Sim(3) correction (Sec. 3.3.3) are also not ablated separately.

Questions for authors
  1. Could the authors provide a direct comparison of full-sequence reconstruction quality against the main baselines, to clarify whether the trajectory improvements translate into more consistent accumulated geometry?

  2. Could the authors report inference FPS (throughput) and peak GPU memory, including their dependence on sequence length?

  3. Could the authors provide the additional ablations and clarifications requested in Weakness 2? Addressing these would help clarify the paper’s contribution.

  4. The left wall in Figure 1 appears to exhibit layered or duplicated surfaces. Could the authors clarify whether this reflects reconstruction inconsistency, scene structure, or visualization effects?

Reviewer_JgoPOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Summary:

This paper deals with reconstructing 3D point maps and recovering camera pose from long-horizon multi-camera streams. Prior feed-forward reconstruction approaches scale poorly to long sequences because they process all frames together, while streaming techniques suffer from potential drift of the state. Windowed solutions prevent this cost explosion but suffer from disruption of cross-window feature-level communication. To address these challenges the authors introduce MV3R, which processes sequences via overlapping windows (with shared observations) which allow for the establishment of correspondences for connecting the per-window local reconstructions. Plucker ray encodings are used to encode camera configuration information while a fixed-capacity scene memory performs a confidence weighted alignment of the local correspondences. Additionally, the authors introduce MV3D - a benchmark for evaluating reconstruction quality and trajectory stability over long sequences. Empirical evaluations are conducted on multiple indoor datasets of long sequence videos.

Critical strengths

The strengths of the paper are as follows:

  1. Different from standard feed-forward reconstruction approaches that look at the input merely as a collection of images, MV3R jointly explores synchronized, rigidly, mounted camera feeds across time which is valuable.
  2. Unlike overlap alignment based only on confidence from the current pair of windows, MV3R also leverages historical scene features for reliability prediction as well.
  3. The MV3D benchmark is a useful contribution to the community.
Critical weaknesses

The main weaknesses of the paper are as follows:

  1. Several prior work have independently looked at the 3 main components of the proposed approach: (i) overlapping-window reconstruction (VGGT-Long), (ii) ray-based geometric encoding (RayRoPE [d]), and (iii) memory-assisted registration (LoGeR [b]). This work seems to be more about putting together these ideas.
  2. The method does not explicitly enforce loop closure or global consistency across nonadjacent windows. Alignment is only performed among adjacent ones.
  3. Comparison with several established 3D reconstruction methods have not been shown [a, b, c].
  4. Very few qualitative results and no videos of the reconstructions have been presented.

References:

[a] Elflein, S., Li, R., Agostinho, S., Gojcic, Z., Leal-Taixé, L., Zhou, Q. and Osep, A., 2026. VGG-T 3^ 3: Offline Feed-Forward 3D Reconstruction at Scale. arXiv preprint arXiv:2602.23361.

[b] Zhang, J., Herrmann, C., Hur, J., Sun, C., Yang, M.H., Cole, F., Darrell, T. and Sun, D., 2026, September. Loger: Long-context geometric reconstruction with hybrid memory. In European Conference on Computer Vision (pp. 263-281). Cham: Springer Nature Switzerland.

[c] Chen, Z., Qin, M., Yuan, T., Liu, Z. and Zhao, H., 2025, October. Long3r: Long sequence streaming 3d reconstruction. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 5273-5284). IEEE.

[d] Wu, Y., Jeon, M., Chang, J.H.R., Tuzel, O. and Tulsiani, S., 2026, September. Rayrope: Projective ray positional encoding for multi-view attention. In European Conference on Computer Vision (pp. 192-210). Cham: Springer Nature Switzerland.

Questions for authors

Comments/Questions:

  1. What happens when camera observations have pure rotations, very small translations, etc.?
  2. How does the model performance scale to really long time steps? The paper emphasizes sequences containing up to 3,336 images but seems like in terms of time steps, it is only up to 556. Datasets, such as [b], have 10,000+ frame sequences.

References:

[b] Zhang, J., Herrmann, C., Hur, J., Sun, C., Yang, M.H., Cole, F., Darrell, T. and Sun, D., 2026, September. Loger: Long-context geometric reconstruction with hybrid memory. In European Conference on Computer Vision (pp. 263-281). Cham: Springer Nature Switzerland.

Reviewer_yQDrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MV3R addresses long-sequence indoor 3D reconstruction from a fixed camera array. It partitions the input sequence into overlapping windows of a fixed size, encodes the relative camera geometry through Plücker rays, and performs local reconstruction using a VGGT backbone. Contextual features are propagated across overlapping observations, while inter-window registration consists of coarse alignment followed by residual calibration. The authors also introduce a six-camera benchmark rendered from four sources of indoor scene scans, and evaluate MV3R against competing reconstruction methods on this benchmark.

Critical strengths
  1. Online reconstruction from a fixed multi-camera array is a practically valuable problem, particularly for mobile robots and autonomous driving.
  2. MV3R substantially improves over the evaluated feed-forward 3D reconstruction baselines on the benchmark using 50 timesteps and 6 synchronized cameras. It also demonstrates clear advantages in trajectory estimation on the eight longer test sequences.
Critical weaknesses
  1. The title is broader than the demonstrated scope. The method is explicitly designed for fixed multi-camera arrays, but the title does not make this restriction clear and suggests broader applicability to long-sequence 3D reconstruction.
  2. Generalization beyond the evaluated configuration remains unverified. All experiments use 6-camera arrays in indoor environments, and no results are provided for other camera counts. Although the paper explicitly targets indoor scenes, its VGGT backbone already supports outdoor reconstruction, and newer backbones such as Pi3X and DA3 may offer stronger outdoor capabilities. It is therefore important to clarify whether MV3R preserves this broader applicability and what limits its generalization to outdoor scenes.
  3. Evaluation is limited to sequences rendered from camera viewpoints selected by the authors. Results on established datasets would provide stronger evidence of applicability beyond the proposed data setup. Under a binocular configuration, relevant stereo baselines such as FoundationStereo could be considered. Existing datasets with more than 6 fixed cameras, such as Eyeful Tower, NCLT, and Oxford RobotCar, also offer potential evaluation settings.
Questions for authors
  1. Can you provide evidence of generalization to different camera counts and outdoor environments? In particular, how does MV3R perform with fewer cameras, such as a stereo pair, or with larger arrays of a dozen or more cameras?
  2. How does MV3R compare with the baselines in computational efficiency? Please report the input resolution, hardware, peak GPU memory usage, inference latency, and FLOPs. Please also specify the GPU memory capacity underlying the OOM region in Figure 5.
  3. What is the training cost? Please report the number and type of GPUs, memory capacity per GPU, total training time, and the datasets and data volumes used for training.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.