acceptodds
Withdrawn from ICLR 2027

ARGUS-GS: Sensor-Guided Aerial-Ground Gaussian Splatting for 3D City Modeling with Model-to-World Alignment

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

3D city modeling requires both broad scene coverage and accurate alignment with the physical world (i.e., model-to-world alignment), for which image-based reconstruction techniques, such as Gaussian Splatting (GS), provide an effective solution. Vehicle and UAV imagery provide complementary coverage, but their large viewpoint gap complicates cross-stream fusion, while accumulated estimated pose errors along the trajectory can further distort the reconstruction away from its true physical location during coordinate alignment. We present ARGUS-GS, a sensor-guided ground-aerial collaborative 3D city modeling system that produces a unified Gaussian representation with model-to-world alignment. Standard onboard sensor measurements anchor each stream's estimated poses and sparse geometry into the shared physical-world metric frame, while revisit handling limits duplicate Gaussian insertion. We then initialize a shared Gaussian field using inverse-depth search with range-adaptive searching windows, maintaining consistent relative search coverage across viewing distances. Camera poses and Gaussian parameters are then jointly optimized using photometric losses and soft sensor-derived poses, with periodic reseeding of poorly reconstructed views. Simulated urban experiments show  dB higher PSNR over baselines at ground-truth camera poses, with  % lower depth MAE and  % higher surface agreement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, Urban 3D Reconstruction, Aerial-Ground Collaboration, Model-to-World Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_arjoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents ARGUS-GS, a sensor-guided 3D Gaussian Splatting framework designed for joint aerial-ground urban reconstruction with physical model-to-world alignment. The method utilizes onboard GNSS/IMU measurements to anchor vehicle and UAV camera poses into a shared metric world frame, bypassing the need for explicit cross-stream visual feature matching. It initializes Gaussians using range-adaptive inverse-depth search with Depth Anything V2 priors, refines camera poses and Gaussian attributes jointly under soft sensor-derived pose penalties, and incorporates a revisit handling mechanism alongside error-guided reseeding to suppress duplicate primitive insertion. Evaluated on 20 simulated trajectory sequences in a PX4-Gazebo-Unity platform, the authors report substantial improvements in PSNR, depth MAE, and surface agreement under ground-truth camera poses.

Critical strengths

S1. Practical Problem Formulation: Reconstructing large-scale urban scenes by coupling aerial and ground perspectives while anchoring geometry into an absolute metric world coordinate system is an important and practically grounded problem for digital twin generation, robotics, and low-altitude surveying.

S2. Cohesive System Architecture: The end-to-end framework systematically touches upon several core challenges in heterogeneous visual mapping, specifically tackling disparate view ranges via range-adaptive search, mitigating cross-stream initialization discrepancies with monocular depth scaling, and addressing trajectory overlap artifacts via history-aware revisit gating.

S3. Extensive Ablation and Diagnostic Breakdown: The synthetic experimental section provides detailed component-level ablations in Table 3, quantifying the relative impact of the inverse-depth window scheme, the sensor pose loss term, the revisit insertion filter, and targeted reseeding on both pose error and surface reconstruction metrics.

Critical weaknesses

W1. Severe Protocol Bias in Baseline Evaluation: The central claim of a 5.8 dB PSNR gain over baselines is fundamentally confounded by the evaluation protocol. In Section 4.1 and Section 4.2, all models are rendered and evaluated strictly at the Unity ground-truth world-frame camera poses. Pure-vision baselines (e.g., 3DGS, VastGaussian, CityGaussianV2, UrbanGS) are georeferenced via a single global Sim(3) alignment without local pose refinement against world sensors, leaving trajectory drift uncorrected (as shown by their 0.5+ m position errors in Table 1). In large-scale scenes, even sub-degree rotation or sub-meter positional offsets produce severe pixel-space phase shifts that drastically degrade PSNR down to 16–17 dB regardless of the actual quality of local scene geometry and texture. Consequently, Table 1 primarily demonstrates that ARGUS-GS achieves better camera pose georeferencing, not necessarily a superior radiance field representation. To establish genuine reconstruction superiority, baselines must be evaluated either in their estimated camera frames or with test-time local pose alignment (e.g., local Umeyama alignment per evaluation chunk).

W2. Critical Absence of Empirical Real-World Validation: Although the experimental setup mentions real-world aerial-ground captures and references the Horizon-GS setup, the actual experimental validation in Tables 1, 2, and 3 is conducted entirely on synthetic data simulated in PX4-Gazebo-Unity. Synthetic sensors generate idealized, well-behaved Gaussian noise or smooth random walks. This fails to simulate real urban canyon conditions, where onboard GNSS suffers from severe non-line-of-sight (NLOS) multipath jumps, IMUs exhibit unmodeled temperature-dependent bias drifts, camera sensors present rolling-shutter artifacts and large dynamic range discrepancies, and real scenes contain transient moving objects. Claiming robust "model-to-world alignment from standard onboard sensors" without a single quantitative or qualitative evaluation on actual real-world vehicle/UAV sensor logs represents an unresolved empirical gap.

W3. Missing Core Engineering Baseline (Sensor-Aided SfM + SOTA 3DGS): The benchmark compares ARGUS-GS against pure-vision methods and GeoRefGS. However, the standard and widely adopted engineering paradigm for this problem is a two-stage pipeline: running Bundle Adjustment / Factor Graph Optimization with GNSS/IMU pose priors (e.g., COLMAP with GPS priors or Ceres-based pose graph optimization) to obtain drift-free metric camera poses, followed by training modern urban 3DGS models such as CityGaussianV2 or UrbanGS. Without comparing against this baseline, it is impossible to determine whether the complex joint photometric-pose gradient backpropagation (Equation 7) and heuristic reseeding provide any tangible benefit over decoupled sensor fusion followed by state-of-the-art splatting.

W4. Disconnect Between Revisit Claims and Quantitative Geometric Metrics: Section 3.6 emphasizes preventing "duplicate or layered Gaussians" when loops or revisits occur. However, the evaluation in revisited areas (Figure 5) relies strictly on photometric error and rendered depth maps. Photometric loss alone is not an adequate indicator of duplicate suppression, as volumetric Gaussians with adjusted opacities can easily fit RGB views despite harboring redundant, overlapping internal surfaces. The paper lacks direct geometric verification such as point cloud thickness, Chamfer distance, or primitive redundancy ratios in overlapped sub-regions. Furthermore, while the Abstract highlights "87.5% higher surface agreement", this metric is not defined in Section 4.1 (deferred entirely to Appendix B), leaving a noticeable gap between the abstract's headline claim and the main text's formal analysis.

W5. Conflated Impact of Monocular Foundation Model Priors: The range-adaptive initialization relies heavily on Depth Anything V2 (DDA V2), a massive pretrained vision foundation model possessing strong semantic geometric priors. The ablation in Table 3 only compares a range-adaptive window against a fixed inverse-depth window, failing to isolate how much of the geometric initialization improvement originates from the external foundation model prior versus the proposed geometric search formulation.

Questions for authors

Q1. Regarding Table 1's evaluation protocol: If the pure-vision baselines (e.g., CityGaussianV2, UrbanGS) are evaluated at their own estimated camera poses or subjected to test-time local pose registration before computing PSNR/SSIM, how much of the reported 5.8 dB PSNR advantage remains? Does the visual representation itself outperform baselines when decoupled from pose registration error?

Q2. Why are there no quantitative or visual results on the real-world dataset (such as the Horizon-GS real capture mentioned in the setup)? Can the authors present quantitative metrics or side-by-side visual comparisons on a real-world aerial-ground sequence demonstrating that the soft sensor constraint remains stable under real urban GNSS multipath spikes and IMU bias drift?

Q3. How does the proposed joint photometric-pose optimization compare against a two-stage pipeline where camera poses are pre-optimized using standard sensor-fusion bundle adjustment (e.g., COLMAP with GNSS priors) and then used to train UrbanGS or CityGaussianV2? What specific geometric failure modes does joint optimization solve that sensor-aided BA cannot?

Q4. Can the authors provide direct quantitative evidence demonstrating the suppression of duplicate Gaussians in revisit zones, such as local point cloud thickness profiles, Chamfer distance against ground-truth meshes, or primitive count comparisons in overlapped areas?

Q5. What is the computational and memory overhead of the revisit landmark matching and error-guided reseeding stages during training relative to standard 3DGS densification?

Reviewer_CHc1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ARGUS-GS aim to improve joint aerial–ground reconstruction in a common metric world frame. The system combines sensor-derived pose anchors, range-adaptive inverse-depth initialization, joint camera/Gaussian optimization, revisit validation, and reseeding. Evaluation covers 20 paired sequences from 12 simulated scenes with demonstrated improvements over representative baseline methods. Validation remains simulation-only; robustness to realistic sensor error remains insufficiently characterized.

Critical strengths

The pipeline is well motivated by practical aerial–ground capture challenges. ARGUS-GS addresses limited cross-platform overlap, heterogeneous viewing distances, pose drift, and revisited surfaces through complementary registration, initialization, refinement, and insertion-control designs. Ablations support the evaluated components, with sensor pose regularization exhibiting the largest effect. These findings substantiate the design within the simulated setting.

Critical weaknesses

A central weakness is the reliance on limited number of simulation-only evaluation, which leaves performance under real-world sensor noise, dynamic objects, and appearance changes untested. Sensor-derived references establish the metric frame and constrain pose refinement (Sections 3.3 and 3.5). Although Table 3 demonstrates their importance, removing these constraints does not establish robustness to inaccurate references. This limits the evidence for practical city modeling; a systematic analysis of GNSS/IMU noise and bias would help substantiate the alignment claims.

Questions for authors
  1. Baseline configuration. How were the baselines adapted to use sensor measurements, and what training budgets and convergence criteria were used? Section 4.1 describes initial sensor-derived Sim(3) alignment for locally defined methods; did any baseline also use sensor constraints during optimization? This would help assess whether the reported gains primarily reflect continued sensor supervision or the other proposed components.

  2. Figure3 evaluation. How the reported PSNR values computed? The large inter-method gaps (about 16 vs 30) and low baseline scores warrant clarification, as I think the visual quality looks similar in certain cases.

  3. Sensor accuracy and real-world evidence. What GNSS/IMU noise, bias, and drift are included in the simulator, and how accurate are the resulting sensor-derived camera references relative to ground truth? Are any real-world or sensor-perturbation results available? Since these references anchor and regularize the reconstruction, this evidence would materially affect my assessment of the world-frame alignment claim.

Reviewer_C897OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents ARGUS-GS, a sensor-guided framework for aerial-ground 3D city reconstruction with model-to-world alignment. The method uses GNSS/IMU measurements to align independently registered UAV and vehicle observations into a shared metric coordinate system. It further introduces range-adaptive inverse-depth initialization, joint Gaussian and camera-pose optimization with sensor-derived constraints, revisit-aware Gaussian insertion, and error-guided reseeding. The framework is evaluated on 20 paired UAV-vehicle sequences across 12 simulated urban scenes. Experiments report substantial improvements in rendering quality, camera-pose accuracy, and surface reconstruction over several Gaussian Splatting baselines.

Critical strengths
  1. The paper addresses the practical challenge of combining aerial and ground observations while maintaining accurate model-to-world alignment in large-scale urban reconstruction.

  2. The framework integrates sensor-guided registration, adaptive initialization, and joint optimization, achieving substantial improvements in rendering quality and geometric accuracy on the simulated benchmarks.

Critical weaknesses
  1. The experimental protocol raises significant concerns about the validity and generalizability of the reported model-to-world alignment improvements. The paper emphasizes accurate alignment with the physical world using standard onboard GNSS/IMU measurements. However, all evaluations are conducted in simulated urban environments, and the paper does not adequately characterize the accuracy, noise distributions, calibration uncertainty, or systematic biases of the simulated sensor measurements. This omission is particularly important because the proposed method directly anchors camera poses to sensor-derived references through Equation (2) and the regularization terms in Equation (7). Consequently, the achievable model-to-world accuracy may depend heavily on the quality of these external references rather than on the effectiveness of the proposed reconstruction framework. Table 1 reports extremely small camera-pose errors of 0.013 m and 0.012 degrees, substantially outperforming all baselines. Without specifying the accuracy of the underlying GNSS/IMU references, it is difficult to determine whether these results reflect robust recovery from uncertain sensor measurements or optimization under nearly ideal pose priors. Moreover, the proposed soft sensor constraints do not independently guarantee physical alignment when the sensor references contain systematic errors or trajectory-dependent biases. The evaluation protocol introduces an additional concern. Section 4.1 states that the sequences are captured at 60 fps, with odd-numbered frames used for training and even-numbered frames reserved for testing. This produces highly correlated training and test observations with potentially very small viewpoint differences. Such a protocol primarily evaluates interpolation between nearby observations rather than reconstruction under substantially unseen viewpoints or trajectories. Although evaluation at ground-truth camera poses measures alignment-sensitive rendering quality, it does not resolve these concerns about sensor reliability and test-view diversity. The absence of real-world experiments or controlled sensor-noise studies substantially weakens the paper's central claim of reliable physical-world alignment under practical conditions.

  2. The experimental comparisons do not adequately isolate the proposed methodological contributions, and several important algorithmic assumptions remain insufficiently justified. The reported performance improvements are difficult to attribute specifically to the proposed initialization, revisit handling, or reseeding strategies because the competing methods do not appear to receive equivalent sensor-constrained optimization. Section 4.1 states that conventional Gaussian Splatting baselines are transformed into the shared world frame using sensor-derived Sim(3) alignment before optimization. In contrast, ARGUS-GS continuously incorporates per-frame sensor position and orientation constraints into joint Gaussian and camera-pose optimization. These represent substantially different levels of access to sensor information during optimization, even when the same raw measurements are provided. This distinction is particularly consequential because all methods are evaluated at ground-truth camera poses. As the authors acknowledge in Section 4.2, relatively low rendering scores for conventional baselines may largely result from world-frame pose misalignment rather than inferior local reconstruction quality. Therefore, the substantial improvements in Table 1 cannot establish how much of the advantage originates from sensor-constrained pose refinement rather than the other proposed components. The methodological formulation also leaves several questions unresolved. Equation (3) introduces a range-relative inverse-depth search window, but its practical behavior depends on the monocular depth alignment and the selected search bounds. The stated interval does not explicitly guarantee positive inverse-depth candidates, and the treatment of invalid candidates is unspecified. Similarly, the revisit-validation and Gaussian-insertion procedures rely on multiple geometric consistency checks whose detailed criteria and thresholds are not sufficiently defined to establish their reliability or reproducibility. Although Table 3 provides component ablations, these experiments are conducted within the same simulated sensor setting and do not disentangle the contributions of improved pose references, reconstruction initialization, and geometric refinement under matched experimental conditions. In particular, the ablation without the sensor pose loss still retains sensor-anchored initialization, preventing a complete assessment of the dependence on external pose measurements.

Questions for authors
  1. How accurately do the simulated GNSS/IMU measurements represent real-world sensor uncertainty? The proposed method achieves position and rotation errors of only 0.013 m and 0.012 degrees, respectively. What are the actual position and orientation errors of the sensor-derived reference poses before optimization? How are measurement noise, systematic bias, calibration uncertainty, and temporal drift simulated? Can the authors demonstrate that the reported advantages persist under different sensor-noise levels, particularly when the sensor references contain realistic accumulated errors?

  2. Are the baseline comparisons sufficiently controlled to isolate the benefits of the proposed reconstruction strategy? The baselines are globally aligned through sensor-derived Sim(3) transformations, whereas ARGUS-GS additionally employs per-frame sensor constraints during joint optimization. Would the reported improvements persist if the baselines were equipped with comparable per-frame sensor-pose regularization and camera-pose optimization under matched computational budgets? Additionally, can the authors provide rendering comparisons under each method's estimated camera poses to distinguish local reconstruction quality from errors caused by world-frame misalignment?

  3. Does the evaluation protocol adequately demonstrate generalization beyond closely neighboring observations? The paper uses alternating frames from 60 fps recordings for training and testing. What are the actual translational and rotational differences between training and test viewpoints? How does the method perform under temporally separated test sequences, held-out trajectory segments, or substantially different aerial-ground viewpoints? Can the reported model-to-world alignment improvements be substantiated beyond the current simulated environment?

  4. How are the proposed initialization and revisit-handling mechanisms validated independently of sensor-based pose refinement? Equation (3) defines a range-relative inverse-depth search interval, but the implementation details of the search bounds, invalid inverse-depth handling, and candidate selection remain unclear. How sensitive are the results to these choices and to monocular depth alignment errors? Furthermore, how are revisit consistency and duplicate Gaussian insertion quantitatively evaluated? Can the authors demonstrate that these mechanisms provide improvements beyond those already obtained from sensor-anchored initialization and per-frame pose constraints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.