acceptodds
Under review as a conference paper at ICLR 2027

AeroGround: Language Grounding in Large-Scale Aerial Gaussian Scenes

Abstract

Grounding natural-language queries in large-scale aerial 3D scenes requires distinguishing visually similar structures and recovering their complete 3D support. Large viewpoint changes and incomplete observations make both tasks challenging. We present AeroGround, a query-time framework that couples active view exploration with evidence-guided structural refinement. Active Query-guided View Exploration (AQVE) maintains competing cross-view target hypotheses and selects informative views to resolve ambiguity. It associates observations across views and accumulates visibility-aware positive and negative evidence to guide subsequent exploration. Evidence-Guided Structural Refinement (ESR) consolidates reliable foreground and background observations into Gaussian-level target posteriors and combines them with local geometric consistency to recover missing support while suppressing foreground leakage. AeroGround operates directly on standard 3D Gaussian Splatting without scene-specific semantic feature optimization or storage. We also introduce AeroRef3D, a benchmark covering spatial–attribute, commonsense, functional, and hypothetical queries with zero-to-many targets across six aerial scenes. Experiments show that AeroGround substantially outperforms the evaluated baselines on AeroRef3D and achieves competitive results on two aerial open-vocabulary segmentation benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, 3D Grounding and Segmentation, Aerial Scene Understanding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vULCOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The submission addresses language-guided grounding in large-scale 3D Gaussian scenes. The method operates at query time without learning scene-specific semantic features. In the first stage, Active Query-guided View Exploration (AQVE), capture observations across views, accumulates visibility-aware positive and negative evidence, and then selects subsequent views based on the current hypotheses. In the second stage, Evidence-Guided Structural Refinement (ESR), combines multi-view foreground/background evidence with local geometric constraints to refine the selected Gaussian support. The submission also introduces AeroRef3D, a new benchmark covering four query types. The final results show significant improvements over the evaluated baselines. And the additional alblation experiments examine component contributions, backbone choices, and computational costs.

Critical strengths
  1. The method addresses two distinct sources of issue rather than treating grounding as a single retrieval problem.

    Distinguish similar targets and recovering their spatial extent require different evidence. The separation between AQVE and ESR is well motivated, and the cumulative ablation in Tab. 3a supports this design: hypothesis maintenance and active acquisition improve target grounding, while posterior fusion and geometric refinement provide further gains.

  2. Visibility-aware negative evidence is useful in the design.

    A failed detection in an aerial image is not sufficient evidence that a target is absent. Conditioning negative updates on expected visibility is a sensible way to avoid penalizing occluded hypotheses. Besides, the sensitivity experiment in Tab. 6 also provide evidence that negative updates matter: removing them reduces pre-ESR mIoU on SZIIT from 83.35 to 79.60.

  3. The reported improvements are substantial and extend beyond the proposed benchmark.

    AeroGround leads the evaluated methods on all six AeroRef3D scenes and also perform strong on the two open-vocabulary segmentation benchmarks. The latter results are important, because they show that the gains are not limited to the authors’ query construction procedure. Sharing the reasoning backbone across MLLM-based baselines is also a useful experimental control.

Critical weaknesses
  1. I am unsure how much of the improvement comes from AQVE and ESR, rather than the stronger MLLM + SAM 3 pipeline.

    Tab. 3b shows that image-conditioned MLLM localization significantly outperforms text-only segmentation. And, the main baselines retain their method-specific segmentation and preprocessing components, rather than sharing AeroGround’s SAM 3 configuration. This is reasonable for comparing complete systems, but it will leave uncertainty about how much of the large headline improvement comes from AQVE and ESR versus stronger initial masks. Tab. 3a helps, but a more explicit matched-backbone / matched-mask comparison would make it clearer.

  2. The evidence for active disambiguation is less conclusive than the evidence for reduced view consumption.

    In Tab. 5, AQVE achieves nearly the same accuracy as fixed ranking while using fewer views, which support an efficiency claim, but the comparison changes both view selection and stopping behavior. Tab. 3a provides a matched-view-count ablation, yet do not directly show whether AQVE resolves competing candidates more reliably. A fixed-ranking policy with the same stopping rule, together with results on genuinely ambiguous queries, would better show the value of hypothesis-guided acquisition. Besides, Eq. 12 is an uncertainty-and-visibility heuristic rather than an explicit expected information-gain calculation.

  3. The claim of recovering complete 3D support is only indirectly evaluated.

    The quantitative evaluation measures rendered masks at held-out camera poses. This is useful, but it does not directly establish the correctness of unobserved or weakly observed parts of the selected Gaussian support. Point-cloud visualization provide qualitative evidence, not a quantitative completeness measure. The conclusions should distinguish improved novel-view segmentation from demonstrated recovery of complete 3D object extent.

  4. The pipeline relies on many hand-designed decisions whose individual contributions are unclear.

    The appendix includes several thresholds and special cases for association, representative-view fusion, topology-dependent support retention, graph construction, component acceptance, and rendering calibration, which improve reproducibility, but make it difficult to find which principles are responsible for the gains. The sensitivity study covers only a small subset of these choices on one scene. In particular, the unchanged results across the tested graph weights raise the question of how much the pairwise graph term contribute beyond unary evidence and the subsequent acceptance rules.

Questions for authors
  1. Can you separate adaptive selection from early stopping?

    In particular, how does fixed ranking perform with AQVE’s stopping criterion, and how does adaptive selection perform without early stopping at matched view counts?

  2. What are the benchmark counts and results for zero-, single-, and multi-target queries, each reasoning type, and empty versus nonempty evaluation masks?

    It's better to report an always-empty baseline and a scene-level target-presence metric.

  3. How were the many thresholds and special-case rules selected?

    Was a separate development set used, and were all settings frozen before evaluating the six benchmark scenes? Please also clarify how often the relation-supported fallback and topology-dependent rules are activated.

3 comments
AuthorsOpenReview ↗

To reviewer vULC

We appreciate your thoughtful review and encouraging assessment. We address your concerns below.

1. Matched-Backbone Comparison

We conduct a controlled comparison on UrbanBIS-Ref using the same query planner, Qwen3.5-35B-A3B + SAM 3 frontend, 3DGS geometry, and evaluation protocol. Unlike the direct-query SAM 3 baseline in Table 3a, our fixed-retrieval controls use the same MLLM-assisted localization as AeroGround. We compare fixed retrieval and AQVE, each with and without ESR, to isolate their respective contributions.

MethodmIoU ↑mBIoU ↑
Fixed Retrieval + MLLM + Mask Lifting70.5159.69
Fixed Retrieval + MLLM + ESR73.8461.82
Full AQVE74.4762.56
Full AeroGround78.4968.22

AQVE improves mIoU/mBIoU over matched fixed retrieval by 3.96/2.87 points without ESR. ESR adds 3.33/2.13 points to fixed retrieval and 4.02/5.66 to AQVE. Overall, AeroGround gains 7.98/8.53 points over matched fixed retrieval.

2. Active Disambiguation

Eq. (12) uses uncertainty and visibility, not exact expected information gain.

Controlled ablation. We compare pre-ESR view selection on 21 ambiguous SZIIT instance annotations (N = 21), with identical candidate pools, budget (B = 24), hypothesis updates, and stopping rules when enabled. Target Acc. is the percentage of annotations with the correct target selected.

View SelectionEarly StopmIoU ↑mBIoU ↑Target Acc. ↑
Fixed RankingNo69.1949.5233.33
Fixed RankingYes71.5153.3642.86
AQVENo70.9751.1738.10
AQVEYes76.1754.4866.67

Even without early stopping, AQVE improves mIoU/mBIoU over fixed ranking by 1.78/1.65 points. When both use the same stopping rule, AQVE gains 4.66/1.12 points and improves Target Acc. from 42.86% to 66.67%, highlighting the benefit of active view selection beyond early stopping.

3. Scope of 3D Support Evaluation

Following query-mask annotation paradigms in ReasoningGD (ReasonGrounder [1]) and REALM3D (REALM [2]), AeroRef3D pairs queries with human-verified 2D masks. As in Gaussian Grouping [3] and Unified-Lift [4], we evaluate Gaussian support via held-out-view rendered masks (mIoU/mBIoU). All baselines, including ReasonGrounder [1], ReLaGS [5], and REALM [2], follow the same protocol.

This measures novel-view segmentation, not exact 3D completeness, especially in unobserved regions. ESR improves mIoU/mBIoU from 74.47/62.56 to 78.49/68.22 (Table 3a); 3D visualizations suggest better spatial coverage. We will replace “complete 3D support” with “improved spatial coverage and consistency of Gaussian support” and acknowledge the lack of direct 3D completeness evaluation.

4. Heuristic Design and Graph Contribution

All following ablations use UrbanBIS-Ref.

Special-case rule ablation. We evaluate the relation-supported fallback and non-enclosed target retention rules by measuring their activation frequencies and performance changes after removal.

VariantActivationChanged OutputsmIoU ↑mBIoU ↑
Without Relation-supported Fallback10.4%9.0%78.4168.10
Without Non-enclosed Target Retention11.9%11.9%78.2667.95
Without Both Rules——78.1567.84
Full AeroGround——78.4968.22

Both rules activate in a minority of cases, and removing them together reduces mIoU/mBIoU by only 0.34/0.38 points. This indicates that AeroGround's overall improvements are not primarily driven by special-case heuristics.

ESR ablation. We conduct a cumulative ablation starting from AQVE, progressively adding Gaussian posterior fusion, unary/component refinement, local geometric evidence, and pairwise graph regularization.

VariantmIoU ↑mBIoU ↑
Full AQVE74.4762.56
+ Gaussian Posterior Fusion76.7666.88
+ Unary / Component Refinement77.8967.53
+ Local Geometric Evidence78.2467.95
+ Pairwise Graph Regularization (Full AeroGround)78.4968.22

ESR improves mIoU/mBIoU over AQVE by 4.02/5.66 points. Posterior fusion contributes 2.29/4.32 points, while later refinements add 1.73/1.34 points, including 0.25/0.27 points from pairwise graph regularization.

References

[1] Liu et al., “ReasonGrounder: LVLM-Guided Hierarchical Feature Splatting for Open-Vocabulary 3D Visual Grounding and Reasoning,” CVPR, 2025.

[2] Shi et al., “REALM: An MLLM-Agent Framework for Open World 3D Reasoning Segmentation and Editing on Gaussian Splatting,” CVPR, 2026.

[3] Ye et al., “Gaussian Grouping: Segment and Edit Anything in 3D Scenes,” ECCV, 2024.

[4] Zhu et al., “Rethinking End-to-End 2D to 3D Scene Segmentation in Gaussian Splatting,” CVPR, 2025.

[5] Xie et al., “ReLaGS: Relational Language Gaussian Splatting,” CVPR, 2026.

Thank you for the careful responses and additional experiments, and sorry for my delayed reply. I also met some deadlines these a few days.

I have carefully read all your responses, and most of my concerns have been addressed. For example, the matched-backbone comparison provide strong evidence for the contributions of AQVE and ESR. The view-selection ablations also provide clearer evidence for AQVE’s benefits beyond early stopping. The additional analyses of the special-case rules and ESR components is also reasonable, which help to clarify their individual contributions. The full responses are supported by experiments that strengthen the authors’ explanations. Although the ambiguous-query experiment remains limited in scale, the additional evidence sufficiently strengthens the submission. I will therefore raise my rating to weak acceptance.

AuthorsOpenReview ↗

To reviewer vULC

Thank you for taking the time to carefully read our responses and additional experiments. We really appreciate your thoughtful and constructive feedback. We will present these additional experiments in greater detail in the revised manuscript.

Reviewer_mTP3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents AeroGround for language grounding in aerial 3D Gaussian scenes. It adaptively selects existing views to resolve target ambiguity, then refines the target’s 3D extent using multi-view evidence and geometric constraints. The authors also introduce AeroRef3D, a referring and reasoning benchmark built on six aerial scenes. Overall, the strong empirical results and benchmark contribution support acceptance, although the advance over existing association and refinement methods could be better established.

Critical strengths
  • AeroGround substantially outperforms the evaluated baselines on grounding and open-vocabulary segmentation. The component ablations support its design choices, while AQVE combined with early stopping processes fewer views with comparable accuracy to fixed ranking.
  • AeroRef3D extends aerial evaluation beyond category-level segmentation to referring and reasoning queries with zero, single, or multiple targets. Its human-reviewed and corrected annotations provide a useful benchmark for this setting.
Critical weaknesses
  • The component ablations support the proposed pipeline, but its advance over existing association and refinement methods (e.g., MaskClustering [1]) is less clear. Comparisons using the same input observations would help establish the value of the proposed components beyond their incremental additions.
  • Table 3a supports hypothesis-guided exploration with matched view counts. However, Table 5 compares AQVE against fixed ranking without early stopping, so the respective contributions of adaptive selection and early termination to the reduction in processed views remain unclear.

[1] Yan et al., "MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation", CVPR, 2024.

Questions for authors
  • Have you evaluated existing association or refinement methods (e.g., MaskClustering) using the same MLLM/SAM 3 observations? Such a comparison would help assess the contribution of the proposed components.
  • In Table 5, how does fixed ranking perform when it uses the same hypothesis updates and stopping criterion as AQVE?
3 comments
AuthorsOpenReview ↗

To reviewer mTP3

We are sincerely encouraged by your positive assessment of our work and grateful for your constructive feedback.

1. Association and Refinement Comparison

We conduct a controlled comparison with MaskClustering on UrbanBIS-Ref to better isolate the contributions of cross-view association and structural refinement.

Matched-observation comparison. We reuse the query-conditioned views and Qwen3.5-grounded SAM 3 masks from the original ablation, keeping them identical across all variants. We independently vary the cross-view association method and refinement strategy while using the same 3DGS geometry and evaluation protocol.

AssociationRefinementmIoU ↑mBIoU ↑
MaskClusteringMask Lifting68.0656.79
MaskClusteringESR70.8459.22
Hypothesis maintenanceMask Lifting71.5859.83
Hypothesis maintenanceESR74.4962.79

Under identical observations, our hypothesis maintenance improves mIoU/mBIoU over MaskClustering by 3.52/3.04 points with Mask Lifting and 3.65/3.57 points with ESR. ESR further improves both association variants, confirming the contributions of hypothesis maintenance and structural refinement.

2. Disentangling Active View Selection and Early Stopping

We compare fixed ranking with and without early stopping against AQVE, using the same hypothesis-update rules and stopping criterion when enabled. Results on GauU-Ref are shown below.

View SelectionEarly StopViews ↓Qwen Calls ↓mIoU ↑mBIoU ↑
Fixed RankingNo23.5924.5980.4769.59
Fixed RankingYes18.8919.8980.3269.44
AQVEYes13.2814.2880.8969.76

Early stopping alone reduces fixed-ranking views by 19.9%, with only a 0.15-point decrease in mIoU/mBIoU. Under the same stopping criterion, AQVE further reduces views and Qwen calls by 29.7% and 28.2%, respectively, while improving mIoU/mBIoU by 0.57/0.32 points. This confirms the benefits of active view selection beyond early stopping.

Thank you for the response. I've read the rebuttal as well as other reviewers' comments, and found that most of my concerns including comparison with MaskClustering are resolved. Therefore, I'll maintain my positive rating.

AuthorsOpenReview ↗

To reviewer mTP3

Thank you for your thoughtful follow-up and for carefully considering our rebuttal. We are glad that the additional experiments and analyses helped address your concerns. We sincerely appreciate the time and effort you have put into reviewing our work.

Reviewer_5cDxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper achieves language localisation in 3D aerial scenes by using AQVE to actively select views and maintain cross-view target hypotheses, and then employing ESR to fuse evidence and refine Gaussian support; it also proposes the AeroRef3D benchmark comprising six scenes.

Critical strengths
  1. Table 1 shows that the model outperforms the baseline across all six scenarios, with an mIoU advantage ranging from 16.69 to 50.07 percentage points; this presents an ablation study on the number of matched views and a comparison with the unified Qwen backbone.
  2. Under the default budget, the number of views and model invocations are reduced by 43.70% and 41.93% respectively, whilst point estimates of accuracy remain comparable.
  3. The paper provides verification via human annotation, sets aside test views, and analyses parameters and costs, whilst also presenting examples of missed scores and leakage failures.
Critical weaknesses
  1. On pages 13–14, image–query pairs are aggregated for scoring, with both-side empty masks counted as 1. Grouped results are missing for zero/single/multiple targets, four-class reasoning and non-empty samples; therefore, the overall high scores cannot individually demonstrate reliability in rejecting recognition, detecting all multiple objects, or handling certain types of reasoning. The statistical denominators should be supplemented, and the existing prediction reports should be utilised to group results, scene-level false positives and object omissions. This does not imply that the scores are artificially inflated.
  2. Equation (13) on page 15 accepts all hypotheses that meet the threshold, but the condition ‘retain only the highest-ranked candidate’ appears only in the stop list. It is not made clear whether, when both candidates meet the threshold, only one is ultimately output. Pseudocode for the final selection and execution traces for the dual-candidate scenario should be provided. This currently constitutes a specification ambiguity, not a proven implementation error.
  3. The sources for the candidate pool, multi-scale regions, specific partitioning and threshold parameter selection remain insufficient to allow for accurate verification. Priority should be given to providing existing configuration, implementation and development records. APR and ORC have explicitly rated this as ‘major’, whilst PJ has not separately listed the severity. There is no evidence to support an allegation of test leakage.
Questions for authors

Detail in Weaknesses

2 comments
AuthorsOpenReview ↗

To reviewer 5cDx (Part 1: Evaluation)

Thank you for recognizing the strong empirical results, efficiency improvements, and benchmark contribution of our work. We appreciate your constructive feedback and address each concern below.

1. Grouped Evaluation and Failure Analysis

We report target-cardinality results on all 336 annotations and reasoning-type results on 275 nonempty-GT annotations (mIoU/mBIoU). Results follow the scene-wise averaging protocol in Appendix A, and N denotes the number of evaluated annotations.

Target CardinalityNBest BaselineAeroGround
Zero-target61LEGS: 93.75 / 93.75100.00 / 100.00
Single-target217REALM: 32.29 / 25.5074.83 / 60.61
Multi-target58AG²S: 35.95 / 27.6877.54 / 66.90

On 275 nonempty-GT annotations, AeroGround achieves 75.13/61.83, outperforming the strongest baseline (REALM: 30.46/24.36) by 44.67/37.47 points. Thus, the improvement is not driven by empty-mask agreement.

Reasoning Type (Nonempty GT)NBest BaselineAeroGround
Spatial–Attribute175Unified-Lift: 31.35 / 25.3774.54 / 61.00
Commonsense19Unified-Lift: 47.55 / 37.3875.00 / 62.16
Functional50REALM: 46.03 / 35.7678.50 / 63.70
Hypothetical31ReLaGS: 36.91 / 31.9270.17 / 58.30

The best baseline in each group is selected by mIoU, with its corresponding mBIoU. AeroGround consistently outperforms the strongest baseline across all four reasoning types, with mIoU gains of 27.45–43.19 points, demonstrating improvements across diverse reasoning categories.

Failure analysis. We distinguish false positives, complete-mask failures, and individual target omissions. All 61 zero-target annotations yield empty predicted masks, with no false positives. Among 275 nonempty-GT annotations, 5 produce entirely empty predictions. Of the 58 multi-target annotations, 4 contain at least one missed target, including one complete-mask failure; the remaining 54 have no entirely missed targets. We count a target as missed only when it is entirely absent from the prediction, excluding incomplete boundaries of otherwise detected objects. Observed omissions include small or heavily occluded structures.

AuthorsOpenReview ↗

To reviewer 5cDx (Part 2: Method Details)

2. Hypothesis Selection

Each hypothesis maintains one or more lifted observations and does not necessarily correspond to a distinct physical object. For single-target queries, the top-one restriction applies only to the stopping list, not to final selection. At termination, all maintained hypotheses satisfying Eq. (13) are accepted and their Gaussian supports are fused.

Selection procedure:

Accept(h) := (p_h >= 0.62) AND (n_h >= 2 OR J_h = 1)

During exploration:
    S_t <- {h in Top-M(H) | Accept(h)}
    if single-target:
        S_t <- Top-1(S_t)
    if budget exhausted or candidate pool empty or early stopping is triggered:
        break
    Acquire next view and update H

At termination:
    H_q <- {h in H | Accept(h)}
    return Fuse(H_q) if nonempty, else empty support

Verified execution trace. We examine the SZIIT query “Locate the irregular landscaped pond that winds between the campus towers.” H1 covers the main water surface, while H2 covers a smaller portion beyond the footbridge. Both correspond to the same annotated pond, as confirmed by the saved observations and annotations.

StepH1 (p, n, J)H2 (p, n, J)Stopping listFinal accepted set
1(0.6610, 1, 1)Not createdH1—
2(0.7388, 2, 0)(0.6610, 1, 1)H1—
6(0.8637, 6, 0)(0.6610, 1, 1)H1H1, H2

Here, p denotes the hypothesis posterior, n the number of supporting views, and J the single-view acceptance gate based on anchor-level confidence and other criteria. Membership in the stopping list does not itself trigger termination; the stopping criterion is evaluated separately. At step 6, both H1 and H2 satisfy the final acceptance condition. They cover complementary regions of the same annotated pond and are therefore fused.

3. Implementation Details

View partition and candidate pool. Training and evaluation views are disjoint; only training images enter retrieval and verification. Multi-cue CLIP scores are smoothed along the image sequence and fitted with a two-component Gaussian mixture. We retain the ranked prefix ending at the lowest-ranked view whose posterior for the higher-mean component is at least 0.5. After suppressing views within four frame positions of retained views in the same acquisition segment, up to 30 candidates remain. AQVE verifies at most B = 24 views.

Multi-scale regions. Each W × H image supplies 15 CLIP regions: the full image, six crops of size 0.62W × 0.78H on a 3 × 2 grid, and eight square crops of side min(0.50W, 0.75H) on a 4 × 2 grid. Crop origins are evenly spaced over valid positions. This construction is independent of queries and ground-truth annotations.

Parameter settings and reproducibility. The AQVE and ESR hyperparameters are specified in Sec. 4.1 (Implementation Details), with further details in Appendix B and sensitivity analyses in Table 6. We selected a single global configuration using training-view observations and qualitative inspection. We will include the additional retrieval details above in the revised appendix. We also plan to release our code upon acceptance.

Comments on the paper

2 comments
Area_Chair_W68zOpenReview ↗

Authors' rebuttal

Dear Reviewers,

Authors have worked hard to address your concerns. Please check the rebuttals, and I encourage you to discuss with the authors to address any open points.

Regards, AC for this article.

Dear AC,

Thank you for your kind reminder. I have read the authors’ rebuttal and updated my responses accordingly.

Best,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.