acceptodds
Withdrawn from ICLR 2027

Lap3R: Large-Scale Panoptic 3D Reconstruction from Aerial Imagery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large-scale panoptic 3D reconstruction from aerial imagery requires globally consistent geometry and persistent object identities across distant views. Existing feed-forward methods struggle with both: depth inconsistencies distort local reconstructions, repetitive layouts confound loop closure, and cross-view instance supervision is scarce. We present Lap3R, a framework that combines semantic-prior geometry rectification, spatially and temporally verified loop mining, and cross-view co-visibility mask linking. Lap3R first rectifies local geometric distortions using semantic priors. It then verifies long-range loop candidates to retain genuine revisits while rejecting false matches caused by repetitive aerial layouts. To establish persistent instance identities, Lap3R aggregates mask agreement across co-visible views, without temporal tracking or manual cross-view annotations. On large scenes, Lap3R achieves the lowest trajectory error and Chamfer distance among the evaluated methods, while substantially improving semantic and instance reconstruction over feed-forward baselines. We further use Lap3R to generate pseudo-labels from aerial videos; a model trained solely on these labels surpasses the evaluated panoptic baselines on all reported metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Panoptic Segmentation, 3D Reconstruction, Large-Scale

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rt3iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents LAP3 - method for 3D geometry and panoptic reconstruction from aerial imagery. It utilizes strong backbones such as SAM3 and VGGT to create a pipeline for the large scale aerial reconstruction, based on panoptic prediction to get better reconstruction and alignment.

Critical strengths
  1. The method seems to work well on the specific use case, get’s SOTA results on the validation set and the qualitative results show a very good reconstruction quality.
  2. The method is detailed and the pipeline is well explained. The ablation study justifies each step of the pipeline, which is the core novelty of the paper.
Critical weaknesses
  1. Validation - Validation is done on 6 scenes which is extremely small validation set. Moreover, the validation set is out of distribution for any other model then the presented method, while it is from the exact distribution of the training set of the method. Thus, it is not clear if the improvement comes from the new pipeline or from the training and validation set distribution. Also missing strong baselines - MapAnything, Pi3, and DepthAnything. Missing an analysis of runtime and memory, which are the bottlenecks of large scene reconstruction.
  2. Although the pipeline is detailed and the parts of the pipeline are justified in the ablation, it has limited novelty and it is not clear if the full pipeline is necessary or it can be learned without this pipeline at all using an existing method on the training dataset.
Questions for authors

Can you explain the validation set choice? There is a test set for your training dataset (UAVFF3D), why not using it too? UAVFF3D paper shows that MapAnything, DepthAnything, VGGT and Pi3 works well on this kind of data, especially after finetuning.

Reviewer_DNeAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets large-scale panoptic 3D reconstruction from aerial imagery and proposes Lap3R, which builds on a VGGT backbone with a multi-view panoptic prediction head adapted from PanSt3R. The method is evaluated on six large-scale scenes and reports consistently strong results.

Critical strengths
  1. The main table is comprehensive, with simultaneous improvement on both metrics. The proposed method achieves the best ATE and CD on all six scenes at the same time.
  2. The instance-level improvement is substantial.
  3. Training and implementation details are disclosed fairly thoroughly. Appendices A and B provide the panoptic-head architecture, the complete optimization schedule, and the loss weights.
Critical weaknesses
  1. No strategy is proposed for the reconstruction drift inherent to VGGT itself. The paper builds directly on the backbone's output without addressing the geometric offset that VGGT already exhibits.
  2. The ablations cover only a single scene. Both Table 3 and Table 4 are run on Longhua alone, so the ablation evidence is insufficient.
  3. The baseline comparison is not entirely fair. The three baselines in Table 2 (PE3R, PanSt3R, IGGT) are all designed for bounded scenes, have no cross-chunk mechanism, and nearly fail on large-scale scenes; meanwhile the geometric baselines in Table 1 (nine methods) have no panoptic capability at all. The two comparisons each cover only half of the problem.
  4. Runtime and memory costs are entirely unreported. The paper reports neither the cost of SRLM (matching with up to 8,192 SIFT features per image, RANSAC fundamental-matrix estimation, pose graph optimization) nor of CCML (projecting and evaluating each mask pair across ≥τ_vis = 5 co-visible views, plus iterative graph updates), nor the number of chunks; nor does it compare inference time against VGGT-Long or TALO.
Questions for authors
  1. How does the proposed method correct the reconstruction error that VGGT itself exhibits?
  2. Could you provide more thorough ablation results (for example, repeated on more than one scene)?
  3. Were PE3R, PanSt3R, and IGGT in Table 2 also retrained on UAVFF3D using CCML-constructed labels under the same optimization schedule?
  4. What are the inference time and memory footprint of SRLM and CCML respectively? How do they scale with sequence length, and how does the total runtime compare with VGGT-Long and TALO on the same hardware?
Reviewer_RKppOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LAP3R for panoptic 3D reconstruction from aerial imagery. The authors introduce 1. semantic-prior rectification that correct local geometry given the predicted semantics. 2 verified loop mining (SRLM) that verifies based on spatial feature verification and a temporal-neighborhood check. 3. cross-view co-visibility mask linking (CCML) that checks if objects are filtered or merged based on masks from different observations.

Critical strengths
  1. The paper presents a coherent solution to the meaningful panoptic 3D reconstruction from aerial imagery, and the reasoning behind each component is easy to follow.
  2. The proposed method achieves strong empircal gains compared to other baseline methods in geometric correctness and semantic metrics.
  3. The ablation results in Table 3 and 4 support each component.
Critical weaknesses
  1. The proposed method have multiple thresholds such as minimum projected support, mask association, cross-view consensus and minimum co-visible views. Are these thresholds maintained throughout the experiments and how sensitive they are? particularly, 5% image-area requirement could exclude small objects, and the five co-visble views and 90% agreement could be restrictive.
  2. For the qualitative semantic reconstruction in Figure 7, one visible difference between the proposed method and PE3R is that PE3R lacks terrain and road, which could contribute to part of the results and deserves further analysis. There is also gain from the aerial panoptic training.
  3. The training dataset needs more details.
  4. From my understanding, the geometry metrics gains came from the inference semantic rectifications, verified loop mining, and cross-view co-visibility mask linking, and there is no direct geometry training, which worth clarify better in the revised manuscript. How does this approach compared to training-based method, e.g. AerialMegaDepth [1]
  5. What are the computation costs during training and inference?
  6. A qualitative comparison on ablated component would be helpful to demonstrate their effectiveness better.

[1] Vuong, Khiem, et al. "Aerialmegadepth: Learning aerial-ground reconstruction and view synthesis." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

Questions for authors

Please see comments above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.