acceptodds
Withdrawn from ICLR 2027

AdaGSFormer: Complexity-Adaptive Gaussian Evolution for 3D Semantic Occupancy Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Gaussian-based representations offer an efficient alternative to dense volumetric features for 3D semantic occupancy prediction. Existing methods adapt the Gaussian population mainly through progressive densification or allocation under a predefined final budget, limiting their ability to increase capacity in structurally complex regions while removing redundancy elsewhere. We propose AdaGSFormer, a complexity-adaptive Gaussian evolution framework that adjusts the population size according to local scene structure without prescribing its final cardinality. AdaGSFormer predicts Gaussian-wise complexity scores using supervision derived from local semantic heterogeneity and uses scene-relative thresholds to guide population restructuring. It splits Gaussians in complex regions and removes redundant primitives through contribution-aware pruning and moment-preserving merging. To decode the resulting variable-size population, we formulate Gaussian contributions and Gaussian-to-voxel splatting in additive optical-density space. On the SurroundOcc dataset, AdaGSFormer achieves state-of-the-art performance in semantic occupancy prediction, while offering approximately faster inference and 57.8% lower GPU memory usage compared to representative Gaussian methods. These results demonstrate the effectiveness of complexity-adaptive Gaussian evolution for accurate and efficient 3D semantic occupancy prediction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Semantic Occupancy Prediction, 3D Gaussians, Efficient 3D Representation, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jBZFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AdaGSFormer, a Gaussian-based framework for camera-based 3D semantic occupancy prediction that adapts its primitive population during feed-forward decoding. Starting from a structured Gaussian initialization, the model predicts per-Gaussian complexity scores supervised by local semantic heterogeneity. Scene-relative thresholds guide learned splitting in complex regions, while contribution-aware pruning and moment-matched merging compress negligible or redundant primitives. The resulting variable-size Gaussian set is decoded through additive optical-density Gaussian-to-voxel aggregation. On the SurroundOcc validation set, the method reports 21.13 mIoU and 31.79 IoU with an average final population of 14.3k Gaussians.

Critical strengths

Good motivation and a coherent approach to local capacity allocation. The framework combines complexity-guided expansion with contribution- and redundancy-aware compression. In particular, low semantic heterogeneity alone does not trigger removal: informative primitives can be retained, while negligible or redundant ones are pruned or merged. This is a sensible design for balancing detailed representation and spatial coverage.

Critical weaknesses
  1. The paper’s citation style does not conform to the ICLR template.
  2. The source of the reported efficiency gains is insufficiently characterized.
  3. Only evaluate on one benchmark.
Questions for authors

Q1: The experiments support the effectiveness of the complete restructuring framework, but do not sufficiently isolate the benefit of complexity-guided selection. In particular, the larger fixed-capacity baseline in Table 2 helps rule out a simple increase in static capacity, whereas the “w/o Split” ablation in Table 3 tests the presence of splitting rather than the criterion used to select primitives for splitting. A comparison against random or shuffled-score selection, with matched per-stage operation counts and otherwise identical settings, would clarify whether the proposed complexity signal improves allocation. This distinction matters because complexity-guided allocation is a central methodological contribution.

Q2: The source of the reported efficiency gains is insufficiently characterized. The headline approximately 2.5× speedup and 57.8% memory reduction are obtained by comparing the complete system against GaussianFormer-2. This comparison also involves differences in initialization and potentially other architectural or implementation choices, so it does not isolate the contribution of adaptive population restructuring. The same-framework comparison in Table 2 is useful and shows approximately 16% lower latency and 10% lower memory than the fixed-16.3k variant. However, a component-wise runtime (for example, gaussian-to-voxel module)and memory breakdown is needed to explain the larger cross-method improvement. Since efficiency is a central motivation and claimed benefit, the current evidence leaves uncertainty about how much of the headline advantage is attributable to the proposed adaptive mechanism.

Q3: Under what scene conditions does the full model produce more or fewer Gaussians than the initial population? Table 6 reports average final counts above the initial 10k for all three complexity groups. Does the full model ever produce fewer than 10k Gaussians? And

Q4: Could you also provide visualizations of representative scenes showing the initial, intermediate, and final Gaussian populations, with splitting, pruning, and merging highlighted and the corresponding counts annotated at each stage?

Q5: Could you evaluate the method on additional occupancy benchmarks to assess its applicability beyond SurroundOcc?

1 comment

Thank you for the constructive feedback. We address each concern and identify the revisions below.

W1. Citation style.

The revised PDF uses author-year in-text citations throughout.

Revised: in-text citations throughout the manuscript.

W2. Sources of efficiency gains.

We agree that the 2.5x speedup and 57.8% memory reduction versus GaussianFormer-2 include initialization and other system differences. We therefore foreground the same-framework comparison.

Table 1. Efficiency comparison.

MethodmIoULatency (ms)Memory (MB)
AdaGSFormer-Fixed (10.0k)20.20189.272826.66
AdaGSFormer-Fixed (16.3k)20.52245.012839.82
AdaGSFormer-Adaptive21.13205.902550.17

Against fixed-16.3k, adaptation gains 0.61 mIoU points with 16.0% lower latency and 10.2% lower memory. Against fixed-10k, it incurs 8.8% extra latency for a 0.93-point gain. These controls establish the measured trade-off within AdaGSFormer; Q2 examines its runtime source.

Revised: Abstract; Sec. 4.2, Table 3.

W3. Evaluation limited to one benchmark.

We added Occ3D-nuScenes and SSCBench-KITTI-360. Q5 provides the results.

Revised: Secs. 4.1-4.2; Appendix A.

Q1. Does complexity-guided selection outperform random selection at matched per-stage counts?

Table 2 compares complexity-guided and uniform-random split-parent selection under the same settings. Random results are mean +/- sample SD over three inference-selection seeds.

Table 2. Split-parent selection comparison.

Selection policymIoUIoUMean final #G
Uniform random20.546 +/- 0.00330.556 +/- 0.00417351.7
Complexity-guided21.13531.81117351.7
Delta+0.589+1.254--

Guidance improves mIoU by 0.589 and IoU by 1.254 points at the same reported mean final population. This supports the selection criterion beyond simply producing a larger final representation.

Revised: Sec. 4.3, Table 5.

Q2. Component-wise runtime and memory attribution.

Table 3 reports a separate instrumented profile. Population control is included in decoder time.

Table 3. Component runtime (ms).

ModuleFixed (16.3k)AdaGSFormer
Image Encoder117.431120.311
Lifter0.0160.014
Decoder131.26886.321
Population control0.00023.178
Remaining decoder131.26863.143
G2V3.2703.315

Control includes complexity/threshold evaluation, splitting, pruning, and merging. Remaining decoder computation falls by 68.125 ms, exceeding the 23.178-ms control cost and yielding a net 44.947-ms decoder saving. G2V time is nearly unchanged, so the measured saving is concentrated in decoder processing. This explains the within-framework runtime benefit. Table 1 reports whole-model memory; module-wise memory attribution is not provided.

Revised: Appendix C.2, Table 11.

Q3. Which scenes produce more or fewer Gaussians, and does any finish below 10k?

Across all validation frames:

Table 4. Final Gaussian-count statistics.

MinP10MedianMeanP90Max
11723.0013064.8014326.0014308.5315468.2016918.00

No evaluated frame finishes below 10k. Manuscript Fig. 6 illustrates this variation. The low-count example has a narrow road corridor and few foreground objects; substantial peripheral support is removed, ending at 11,744. The two median examples show a tree-lined road and an open junction, ending at 14,324/14,325: different layouts can require similar total capacity. The maximum-count frame contains vehicles, vegetation, and roadside structures around a junction; continued splitting and limited compression produce 16,918 Gaussians. Cardinality therefore reflects both local refinement demand and compressible support. Intermediate contraction (e.g., 12,153 to 10,988) does occur.

Revised: Appendix D.3, Fig. 6.

Q4. Representative stage-wise visualizations and annotated counts.

Manuscript Fig. 6 shows inputs, initialization, three restructuring stages, final populations, and occupancy GT. Retain/split/prune/merge are distinguished, with counts at each stage. The rows cover a random sample from the lowest-cardinality group, two near-median frames, and the maximum-count frame. Counts satisfy Nout = Nin + S - P - M, where S/P count split/pruned parents and M counts merged pairs, making expansion and compression directly traceable.

Revised: Appendix D.3, Fig. 6.

Q5. Results beyond SurroundOcc.

Table 5. Additional benchmark results.

MethodOcc3D-nuScenes mIoUSSCBench-KITTI-360 mIoU
GaussianFormer35.5112.92
GaussianFormer-236.5413.90
AdaGSFormer37.4314.14

AdaGSFormer exceeds GaussianFormer-2 by 0.89 and 0.24 points, respectively. Occ3D changes occupancy annotations on nuScenes; monocular SSCBench-KITTI-360 uses independent driving data. These results support applicability across occupancy settings and a separate dataset.

Revised: Sec. 4.2, Table 2; Appendix A, Table 6.

Reviewer_gJVPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents AdaGSFormer for 3D semantic occupancy prediction. It predicts Gaussian-wise scene complexity and adaptively splits, prunes, or merges Gaussians during decoding. An optical-density formulation maps the Gaussians to 3D occupancy predictions. Experiments on SurroundOcc show improved accuracy over fixed-population baselines.

Critical strengths
  1. The motivation is clear: allocating more Gaussians to complex regions while reducing redundancy elsewhere is reasonable.
  2. The experiments support the method’s effectiveness, improving mIoU from 20.20 to 21.13 over its fixed-population baseline with a modest latency increase.
Critical weaknesses
  1. Limited evaluation across datasets. The experiments cover only SurroundOcc, whereas GaussianFormer and QuadricFormer also evaluate on SSCBench-KITTI-360. Results on another dataset are encouraged to establish whether the proposed method works consistently across different settings.

  2. Incomplete comparison with SOTA. Table 1 omits QuadricFormer’s strongest performance. With 12800 primitives, QuadricFormer achieves 21.11 mIoU and 32.13 IoU, compared with AdaGSFormer’s 21.13 mIoU and 31.79 IoU. QuadricFormer also reports 179 ms latency, although measured on different hardware. The paper should compare accuracy and efficiency with this configuration under the same conditions. The current results do not establish a clear performance–efficiency advantage over QuadricFormer.

  3. Additional inference cost. Compared with its fixed-population baseline, latency increases from 189.27 ms to 205.90 ms. Although modest, this overhead reduces the practical benefit of the accuracy improvement for applications in driving scenarios with strict latency requirements.

Questions for authors

Please address the weaknesses above and clarify the following:

  1. In Figure 3, most visible Gaussians appear to undergo splitting, pruning, or merging, with very few retained. Is this distribution encouraged by the design or an outcome of training? Please explain why so few Gaussians are retained.

  2. Could the authors show where splitting improves predictions, particularly around object boundaries and semantically complex regions? Visual comparisons with the variant without splitting, or per-class IoU changes for small objects such as pedestrians and bicycles, would help demonstrate these benefits.

Overall, I would be willing to raise my score if these concerns are adequately addressed.

1 comment

Thank you for the constructive feedback. We address each concern and identify the revisions below.

W1. Does the method remain effective beyond SurroundOcc?

We added SSCBench-KITTI-360. Table 1 summarizes the Gaussian-based comparisons.

Table 1. SSCBench-KITTI-360 results.

MethodIoUmIoU
GaussianFormer35.3812.92
GaussianFormer-238.3713.90
QuadricFormer38.8913.63
AdaGSFormer38.3514.14

AdaGSFormer improves mIoU over GaussianFormer-2 and QuadricFormer by 0.24 and 0.51 points, with completion IoU close to GaussianFormer-2. This supports applicability beyond SurroundOcc, although completion IoU remains below QuadricFormer. We also added Occ3D-nuScenes: 37.43 mIoU versus 36.54 for GaussianFormer-2.

Revised: Sec. 4.2, Table 2; Appendix A, Table 6.

W2. Accuracy-efficiency comparison with the strongest QuadricFormer.

We now include QuadricFormer with 12,800 primitives. Table 2 reports measurements under the same conditions.

Table 2. Efficiency comparison.

MethodmIoULatency (ms)Memory (MB)
QuadricFormer (12.8k)21.11205.193409.35
AdaGSFormer-Fixed (10.0k)20.20189.272826.66
AdaGSFormer-Fixed (16.3k)20.52245.012839.82
AdaGSFormer-Adaptive21.13205.902550.17

Semantic accuracy and latency are comparable, while memory decreases by 25.2%. We acknowledge that occupancy IoU is lower (31.79 versus 32.13). The supported advantage is lower memory at comparable semantic accuracy and latency.

Revised: Sec. 4.2, Tables 1 and 3.

W3. Does added latency limit practical value under strict deadlines?

Relative to fixed-10k, adaptation gains 0.93 mIoU points at an additional 16.63 ms (8.8%). This overhead matters under strict deadlines. However, increasing the fixed population to 16.3k raises latency to 245.01 ms while reaching only 20.52 mIoU. Adaptation achieves higher accuracy with 16.0% lower latency and 10.2% lower memory than this larger fixed version. Thus, adaptive allocation offers a better measured trade-off than simply increasing static capacity. Fixed-10k remains the faster option when latency takes priority.

Revised: Sec. 4.2, Table 3; Sec. 5, limitations.

Q1. Why do few Gaussians appear retained, and is this designed or learned?

Retention is not discouraged by design. Complexity scores are learned, while explicit rules use scene-relative thresholds without a fixed retained/edited quota. Complexity identifies restructuring candidates; contribution and redundancy determine compression or retention. Low complexity alone does not require removal.

Updated manuscript Fig. 3 makes retention visible across the main scene structure. Table 3 quantifies actual decisions.

Table 3. Local complexity and retention.

ComplexityRetained
Low80.26%
Medium57.35%
High20.37%

Retention is common in low- and medium-complexity regions; high-complexity regions undergo more splitting. Groups use GT-derived complexity only for post-hoc analysis; inference uses predicted scores. Retention means no topology edit at that stage; features and positions may still be refined.

Revised: Sec. 4.4, Fig. 3; Appendix B.1, Table 7.

Q2. Where does splitting help, including small objects and boundaries?

We added a class-wise ablation. Response Table 4 reproduces manuscript Table 10.

Table 4. Class-wise effects of splitting.

Class / metricw/o SplitAdaGSFormerDelta
Barrier19.0221.74+2.72
Bicycle13.3512.20-1.15
Bus28.4328.84+0.41
Car30.1432.03+1.90
Construction vehicle8.549.55+1.00
Motorcycle15.0414.51-0.53
Pedestrian12.8614.21+1.35
Traffic cone10.1111.55+1.44
Trailer11.3413.42+2.08
Truck23.0124.29+1.28
Driveable surface41.9142.52+0.61
Other flat25.8427.72+1.88
Sidewalk25.4427.53+2.09
Terrain23.9025.06+1.15
Manmade8.4611.24+2.79
Vegetation19.6721.58+1.91
mIoU19.8221.12+1.31
IoU29.8131.81+2.00

Splitting improves 14 of 16 classes. Barrier (+2.72) and sidewalk (+2.09) show gains beyond small objects; occupancy IoU also rises by 2.00 points. Pedestrian and traffic cone gain 1.35 and 1.44 points, but bicycle and motorcycle lose 1.15 and 0.53 points. Thus, splitting benefits several small-object categories without uniformly improving all of them. The gains are consistent with added local representational flexibility; class-wise IoU does not directly isolate boundary accuracy.

Revised: Appendix C.1, Table 10.

Reviewer_BKavOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors proposed AdaGSFormer, a complexity-adaptive Gaussian evolution framework for 3D semantic occupancy prediction. The method estimates local structural complexity to dynamically adjust Gaussian populations through splitting, pruning, and merging, without requiring a predefined final Gaussian budget. It further introduces Optical-Density Gaussian-to-Voxel Splatting for semantic occupancy decoding. Experiments on SurroundOcc demonstrate improved segmentation accuracy and computational efficiency compared with existing Gaussian-based methods.

Critical strengths
  • The paper addresses the limitations of fixed-size Gaussian representations by dynamically allocating computational resources according to local scene complexity.
  • The combination of splitting, pruning, and merging provides a reasonable approach to balancing representation capacity and computational efficiency.
Critical weaknesses
  • The proposed framework mainly combines established Gaussian splitting, pruning, and merging strategies with a complexity prediction module. Similar population adaptation mechanisms have been extensively explored in 3D Gaussian Splatting, while recent occupancy methods such as PG-Occ and SAGFormer also investigate adaptive Gaussian allocation. Although the integration is meaningful, the paper could better clarify its novelty over these closely related approaches.
  • As shown in Table 1, AdaGSFormer achieves 21.13% mIoU, improving over GaussianFormer-2 and QuadricFormer by only 1.11% and 1.01%, respectively. Furthermore, its overall occupancy IoU of 31.79% remains below SplatSSC's 32.19%. These results suggest that the accuracy improvements are relatively modest despite the additional population restructuring mechanisms.
  • As shown in Table 3, removing pruning slightly improves mIoU from 21.13% to 21.14%, while removing merging leaves mIoU unchanged at 21.13%. Although these components improve computational efficiency, their contributions to segmentation accuracy are limited. The authors should further clarify whether the proposed framework requires all three population restructuring operations and provide a more detailed analysis of their individual effectiveness.
  • Although the paper discusses PG-Occ and SAGFormer as closely related approaches, neither method is included in the main experimental comparisons in Table 1. Since both methods investigate adaptive Gaussian population allocation, direct comparisons under consistent experimental settings would be valuable for demonstrating the advantages of the proposed complexity-adaptive evolution strategy.
Questions for authors

Please refer to the weaknesses.

1 comment

Thank you for the constructive feedback. We clarify the contribution and evidence below.

W1. How does AdaGSFormer differ from PG-Occ and SAGFormer?

Our contributions include: 1) coordinating local expansion and compression using learned complexity and scene-relative thresholds; 2) separating representation demand from compressibility; 3) linking compression to occupancy decoding through a shared optical-density formulation.

Conventional 3DGS adjusts density through scene-specific optimization, whereas AdaGSFormer predicts restructuring within feed-forward occupancy decoding. PG-Occ emphasizes progressive query addition; we jointly expand and compress local capacity while retaining informative support. SAGFormer globally selects candidates under a fixed final budget; our final count emerges from local complexity, contribution, and redundancy decisions. The contribution is therefore coordinated occupancy-conditioned control and decoding, rather than new restructuring operators.

We revise the introduction, Sec. 2, and Sec. 3 of the manuscript, respectively.

W2: Do modest mIoU gains and lower occupancy IoU justify the added restructuring mechanisms?

AdaGSFormer improves mIoU by 1.11% over GaussianFormer-2 and achieves comparable semantic accuracy to the strongest QuadricFormer configuration, although its occupancy IoU is lower.

Table 1. Efficiency comparison under the same conditions.

MethodmIoULatency (ms)Memory (MB)
GaussianFormer-220.02511.416037.81
QuadricFormer21.11205.193409.35
AdaGSFormer21.13205.902550.17

Compared with GaussianFormer-2, inference is approximately 2.5x faster and memory usage is 57.8% lower. Against the strongest QuadricFormer configuration, AdaGSFormer achieves similar semantic mIoU and latency with 25.2% lower memory. Its value therefore includes competitive semantic accuracy and runtime with lower GPU memory consumption.

W3. Are all three operations necessary, and what does each contribute?

Splitting increases local capacity, pruning removes negligible support, and merging consolidates redundant support.

Table 2. Ablation of the three operations.

VariantmIoULatency (ms)Avg. #G
w/o Split19.81183.298.2k
w/o Prune21.14266.4617.0k
w/o Merge21.13232.1614.6k
AdaGSFormer21.13205.9014.3k

Disabling splitting reduces mIoU from 21.13 to 19.81. Enabling it raises latency from 183.29 to 205.90 ms, showing an accuracy gain at additional computational cost.

Removing pruning slightly raises mIoU to 21.14 but increases latency to 266.46 ms. Removing merging leaves mIoU at 21.13 while increasing latency to 232.16 ms. Pruning and merging therefore primarily reduce computational cost while preserving nearly the same observed accuracy.

The separate profile in manuscript Table 11 shows that pruning costs 0.353 ms and merging 11.421 ms, yet Full reduces total profiled runtime by 22.0% and 7.4% relative to w/o Prune and w/o Merge. Their downstream savings outweigh their measured costs. All three operations are retained for their complementary accuracy and efficiency roles.

W4. How does AdaGSFormer compare with PG-Occ and SAGFormer?

We provide the requested comparisons, distinguishing the supervision settings. PG-Occ targets open-vocabulary occupancy using image-derived depth and feature supervision; AdaGSFormer uses 3D occupancy supervision.

Table 3. Comparison with PG-Occ on Occ3D-nuScenes.

MethodmIoU
PG-Occ15.15
AdaGSFormer37.43

These scores involve different supervision settings and do not isolate allocation quality.

Table 4. Comparison with SAGFormer on SurroundOcc.

MethodmIoULatency (ms)Memory (MB)
SAGFormer23.67380.422645.86
AdaGSFormer21.13205.902550.17

SAGFormer achieves higher mIoU. AdaGSFormer reduces latency by 45.9% (1.85x faster inference) and memory by 3.6%, at a cost of 2.54 mIoU points. The main measured benefit is substantially lower latency, with a modest memory reduction. These results position AdaGSFormer as an alternative when inference cost is prioritized, while acknowledging SAGFormer's accuracy advantage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.