acceptodds
Under review as a conference paper at ICLR 2027

DGA-Occ: Distance-Guided and Gated-Adaptive Fusion for Multimodal 3D Semantic Occupancy Prediction

Abstract

Existing multimodal 3D semantic occupancy prediction methods typically perform feature fusion only in local voxel space or global BEV space, failing to account for cross-modal spatial biases and neglecting the complementary advantages of global and local collaborative fusion. To address this, this paper proposes a global-local collaborative multimodal fusion framework based on distance guidance and gating adaptation. Specifically, the gating-adaptive local fusion(GALF) achieves reliable local feature interaction in geometrically aligned voxel space through bidirectional cross-modal nearest neighbor association and learnable gating modulation; the dual-geometric interactive fusion(DGIF) combines cross-modal interaction alignment, camera-LiDAR dual-source geometric priors, and distance-aware attention to achieve spatially adaptive global fusion. Furthermore, the Mamba Split Block jointly models global structure and local details through frequency-aware decomposition, and the local-global fusion module(LGFM) further promotes bidirectional collaboration between camera BEV semantics and LiDAR 3D geometry. Experimental results show that the proposed method achieves mIoU of 54.46 and 25.9 on the Occ3D-nuScenes and nuScenes-Occupancy datasets, respectively, validating the effectiveness of the proposed framework and its components.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
3D Semantic Occupancy Prediction Cross-Modal Alignment Mamba Distance-Aware Attention Gated Feature Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_FVNiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DGA-Occ. The method performs local camera-LiDAR fusion in voxel space and global interaction in BEV space, using GALF, DGIF, Mamba Split Block, and LGFM. Experiments on Occ3D-nuScenes and nuScenes-Occupancy show competitive performance.

Critical strengths

1.This paper explores the complementary use of local voxel-level fusion and global BEV-level fusion for multimodal occupancy prediction.

Critical weaknesses
  1. The methodological novelty is limited. The overall framework mainly combines existing techniques such as KNN association, gating, self-attention, cross-attention, Mamba, ASPP, and bidirectional feature fusion.

  2. The frequency-aware interpretation of the Mamba Split Block is not well justified. The proposed split only rescales the same feature map, so the low-frequency and high-frequency branches are not explicitly separated in the frequency domain.

  3. The key distance-guided mechanism is insufficiently validated. The ablation study does not isolate the contributions of distance encoding and the dual-source geometric prior. The RDE-Gate is only illustrated in Fig. 4, while its exact formulation and implementation are not clearly described in the main text.

Questions for authors
  1. Could the authors clarify why the Mamba Split Block is described as frequency-aware? Since the two branches are obtained by rescaling the same feature map, it is unclear how they explicitly correspond to low- and high-frequency components.

  2. Could the authors provide a clearer formulation of the distance-guided mechanism, particularly the RDE-Gate, and additional ablations that isolate the effects of the distance encoding and the dual-source geometric prior?

  3. Why Fig.6 ground truth voxel is even worse than the prediction results? Is this the right GT used for visualization?

Reviewer_YxscOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes DGA-Occ, a global-local collaborative framework for multimodal 3D semantic occupancy prediction using camera and LiDAR inputs. The framework consists of four components: (1) a Mamba Split Block for modeling global structures and local details through frequency-aware feature processing; (2) Gated Adaptive Local Fusion (GALF) for bidirectional cross-modal neighborhood aggregation; (3) Dual-Geometry Interactive Fusion (DGIF) for distance-aware BEV fusion using geometric priors; and (4) a Local-Global Fusion Module (LGFM) for integrating camera semantics and LiDAR geometry. Experiments on Occ3D-nuScenes and nuScenes-Occupancy report 54.46% and 25.9% mIoU, respectively, supported by component-level ablations and qualitative comparisons.

Despite these results, several concerns remain. The proposed components largely integrate existing techniques, raising questions about methodological novelty and architectural complexity. The frequency-aware decomposition lacks sufficient mathematical justification, while the claimed robustness to calibration errors and effectiveness of distance-guided fusion are not directly validated. Furthermore, the ablation studies provide limited evidence of the individual contributions and interactions of the proposed mechanisms, and computational efficiency is not adequately evaluated. These limitations weaken the overall technical contribution and experimental justification of the framework.

Critical strengths

This paper proposes DGA-Occ, a multimodal 3D semantic occupancy prediction framework that integrates camera and LiDAR features through global-local collaborative fusion. The proposed framework consists of four main components: the Mamba Split Block for frequency-aware feature modeling, GALF for gated local cross-modal fusion, DGIF for distance-guided global fusion, and LGFM for bidirectional local-global feature interaction. The method reports 54.46% mIoU on Occ3D-nuScenes and 25.9% mIoU on nuScenes-Occupancy.

Critical weaknesses
  1. Limited methodological novelty and an overly complicated architecture.

The proposed framework primarily combines existing techniques, including K-nearest neighbor feature aggregation, learnable gating, cross-attention, Mamba-based feature modeling, and multi-scale convolutional fusion. While their integration produces empirical improvements, the paper does not clearly establish which component introduces a fundamentally new mechanism beyond existing multimodal fusion approaches. Furthermore, the framework introduces four major modules, each containing several additional operations. The resulting architecture is considerably complicated, but the paper lacks a convincing explanation of why this particular combination is necessary. The improvements may largely originate from increased model capacity and repeated feature interactions rather than from a genuinely novel fusion principle.

  1. The frequency-aware decomposition in the Mamba Split Block lacks mathematical justification.

In Section 3.1, Eq. (1) defines the frequency decomposition. However, this formulation merely scales the original features using a learnable coefficient. Assuming a scalar coefficient, both branches preserve the same relative frequency distribution, rather than explicitly separating low-frequency and high-frequency components. Although subsequent convolution and pooling operations may extract different spatial patterns, this does not establish that the proposed initial decomposition is genuinely frequency-aware. The authors provide neither theoretical analysis nor frequency-domain visualizations to support this claim. Consequently, one of the central methodological motivations of the paper remains insufficiently justified.

  1. The GALF module contains ambiguous technical definitions that raise concerns about its correctness.

Section 3.2 describes identifying non-empty voxels by summing camera and LiDAR features along the channel dimension and checking whether the resulting values are nonzero. However, a nonzero channel sum is not a reliable indicator of voxel validity, since valid feature activations may cancel each other. Moreover, the description does not clearly explain how modality-specific non-empty voxel sets are independently recovered from a combined intensity map. The proposed neighborhood retrieval is also described as K-nearest neighbor matching, but the actual implementation involves farthest point sampling and ball queries. These operations do not necessarily guarantee exact K-nearest neighbor associations for every query voxel. Important details, including the neighborhood radius, sampling strategy, and handling of insufficient neighbors, are omitted. Additionally, Eq. (5) does not clearly specify how the K neighboring features are aggregated to ensure dimensional compatibility with the original voxel features. These ambiguities substantially weaken the technical clarity and reproducibility of GALF.

  1. The central claims of distance guidance and robustness are not directly validated.

The paper repeatedly emphasizes that GALF alleviates calibration errors and that DGIF improves feature fusion across different observation distances. However, the experiments do not directly evaluate these claims. For calibration robustness, no controlled perturbation experiments are presented under varying extrinsic calibration errors. Similarly, the effectiveness of distance-guided fusion is not evaluated using distance-stratified occupancy metrics, such as performance at near, medium, and far ranges. Moreover, DGIF combines cross-modal alignment, dual-source geometric priors, and distance-aware attention, but the ablation study does not isolate the effects of the distance encoding and the individual geometric priors. Without these evaluations, it remains unclear whether the improvements actually result from the proposed distance-guided and gated-adaptive mechanisms or simply from additional network capacity.

  1. Insufficient efficiency analysis and limited performance advantages on the second benchmark.

The architecture introduces multiple computationally demanding operations, including bidirectional neighborhood retrieval, global BEV attention, dual-source geometric processing, and repeated cross-modal attention. However, the paper provides no quantitative analysis of parameter count, FLOPs, inference latency, or GPU memory consumption. This is particularly concerning because global self-attention over flattened BEV tokens can introduce substantial computational and memory overhead. Furthermore, on nuScenes-Occupancy, the proposed approach improves mIoU from 25.3 (MS-Occ) to 25.9, representing a relatively modest gain of 0.6 percentage points. Without computational comparisons or matched experimental settings, it is difficult to determine whether the added architectural complexity is justified.

  1. Important implementation details are missing, limiting reproducibility.

Several critical hyperparameters and design choices are insufficiently specified, including the number of sampled points in GALF, neighborhood search parameters, the architecture of the gating networks, the mathematical implementation of distance-aware attention, and the construction and normalization of geometric priors.

Questions for authors

Please mention the above weaknesses.

Q1: How does the scalar feature partition in Eq. (1) achieve genuine frequency decomposition rather than simple feature rescaling?

Q2: How are modality-specific non-empty voxel coordinates obtained reliably, and how are the K retrieved neighborhood features aggregated in Eq. (5)?

Q3: Can the authors provide controlled robustness experiments under calibration perturbations and distance-stratified evaluation results?

Q4: What are the individual contributions of distance encoding, camera geometric priors, and LiDAR geometric priors within DGIF?

Q5: What are the parameters, FLOPs, inference latency, and GPU memory requirements compared with strong multimodal occupancy baselines?

Reviewer_jCivOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents DGA-Occ, a camera–LiDAR framework for 3D semantic occupancy prediction. The framework combines local voxel-space fusion with global BEV-space interaction. Its main components are a Mamba Split Block for global and local feature modeling, Gated Adaptive Local Fusion (GALF) for bidirectional cross-modal neighborhood interaction, Dual-Geometry Interactive Fusion (DGIF) for incorporating geometric priors and distance information into BEV fusion, and a Local-Global Fusion Module (LGFM) for integrating the resulting representations. The paper reports 54.46 mIoU on Occ3D-nuScenes and 35.1 IoU / 25.9 mIoU on nuScenes-Occupancy, together with module-level ablations and qualitative comparisons. Both benchmarks are derived from nuScenes, with different occupancy annotation and evaluation protocols.

Critical strengths
  1. Promising reported performance with a relatively modest image backbone and input resolution.

The reported 54.46 mIoU on Occ3D-nuScenes uses ResNet-50 and 256 × 704 images. This is 2.80 percentage points above the SDGOcc entry in Table 1. Subject to clarification of the comparison protocols discussed below, this suggests that the proposed combination could be useful for multimodal occupancy prediction.

  1. A plausible multilevel fusion design supported by actual component ablations.

Combining local geometric associations with global semantic interaction is a reasonable design choice. Tables 3–7 provide evidence that several architectural additions improve the reported baseline. In particular, Table 7 reports an increase from 50.79 to 54.46 mIoU when GALF, DGIF, and LGFM are progressively introduced. These experiments provide evidence for the utility of the overall system, although they do not yet establish the specific mechanisms claimed to explain the gains.

Critical weaknesses
  1. The empirical comparison contains substantial inconsistencies in benchmark protocols, model descriptions, and baseline selection.

The most concerning example is GaussianWorld in Table 2. It is listed as a camera–LiDAR model with an R101/VoxelNet backbone and scores of 33.4 IoU / 22.1 mIoU. However, the original GaussianWorld paper, Section 4 and Table 1, reports the corresponding 33.40 / 22.13 scores for camera-only prediction using SurroundOcc annotations with 0.5 m voxels. This differs from the 0.2 m nuScenes-Occupancy setup described in this submission. Unless the authors performed a separate, protocol-matched reimplementation, this row mixes incompatible configurations and should not be presented as a directly comparable result.

The baseline selection also does not reflect stronger published variants. OccMamba’s revised manuscript, Table 1, reports 35.7 IoU / 26.2 mIoU for OccMamba-384 on OpenOccupancy v0.0, whereas this submission lists 33.7 / 25.1. Similarly, EFFOcc’s revised manuscript, Table II, reports 52.82 mIoU with R50 and 54.08 with Swin-B, while Table 1 includes only the R18 result of 49.29. These variants have different configurations, so a controlled comparison is needed; nevertheless, omitting them weakens the claimed performance context.

There is also an attribution problem: the OccFusion row with 32.4 IoU / 22.4 mIoU in Table 2 matches Zhang et al.’s depth-estimation-free OccFusion, but is attributed to Ming et al.’s different OccFusion paper.

Together with the insufficient specification of annotation versions and training/evaluation masks, these issues prevent a reliable assessment of the empirical advance.

  1. The distinctive fusion mechanisms are insufficiently specified and experimentally isolated.

The paper needs a more precise comparison with its closest predecessors. Co-Occ already uses non-empty voxel neighborhood association and a learnable KNN gate, while Ming et al.’s OccFusion already combines dynamic fusion of local 3D and global BEV features. The contribution of GALF’s bidirectionality and DGIF’s distance conditioning therefore needs to be established beyond the general idea of gated local/global fusion.

However, Section 3.3 does not provide an explicit formulation of the distance encoding or the RDE-Gate shown in Figure 4. The construction, normalization, and incorporation of the geometric priors into attention are also insufficiently specified. Table 5 compares whole fusion modules and the addition of CMIA, but does not isolate distance encoding, either geometric prior, or the distance-dependent gate. The claims regarding calibration robustness and distance adaptation are not directly evaluated through controlled calibration perturbations or distance-stratified results.

The frequency interpretation has a related limitation: for scalar α, Equation (1) rescales the same feature map rather than separating frequency bands. Subsequent pooling and residual subtraction may still provide useful multiscale processing, but the current evidence does not establish the claimed frequency decomposition.

Finally, the supplied appendix does not contain the detailed protocols promised by the reproducibility statement. Important information, including the occupancy prediction head, training losses, neighborhood aggregation details, and feature transformations between BEV and 3D, remains missing. No quantitative computational-cost comparison is provided. Consequently, it is difficult to reproduce the method or determine whether the improvements arise from the proposed mechanisms rather than additional capacity and repeated feature interactions.

Questions for authors

Q1 — Comparison validity and result provenance. Was GaussianWorld reimplemented and evaluated under the same annotation and voxelization protocol as DGA-Occ, or was its result taken from the original SurroundOcc evaluation? Please provide a corrected comparison identifying the source of each result, annotation version, sensor inputs, model variant, temporal inputs, and training/evaluation masks. Please also clarify the OccFusion attribution and the selection of OccMamba and EFFOcc variants.

Q2 — Definition and evidence for distance guidance. Please provide the exact formulation of the distance encoding, RDE-Gate, geometric priors, and their incorporation into attention. Which controlled ablations demonstrate the individual effects of distance encoding and the camera/LiDAR priors? If calibration robustness and distance adaptation remain central claims, what evidence supports them under calibration perturbations and across distance ranges?

Q3 — Contribution relative to the closest fusion methods. What is the precise methodological advance over Co-Occ’s gated neighborhood fusion and Ming et al.’s dynamic local/global fusion? Can the authors provide a controlled comparison that separates the benefit of their proposed interaction mechanism from additional model capacity, together with parameter count and measured inference cost?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.