acceptodds
Withdrawn from ICLR 2027

OccWeave: Structure-Guided Refinement and Fusion for Cooperative Semantic Occupancy

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cooperative semantic occupancy prediction combines observations from connected vehicles to recover dense scene geometry and semantics. Yet collaboration can leave distorted boundaries, missing thin structures, and fragmented local regions in the final occupancy prediction. Preserving these structures requires both informative local features and effective integration of observations from different agents. We propose OccWeave, a framework that introduces visual structural priors to connect local feature refinement with cooperative evidence weighting. Structure-Prior Feature Refinement (SPFR) associates prior features with each agent's local 3D representation and incorporates them through a gated residual. Structure-aware Cooperative Reliability Fusion (SCRF) uses the refined features, prior-induced residuals, cross-agent discrepancies, and relative pose to estimate local source weights. We realize OccWeave through a tri-plane-to-voxel architecture following CoHFF and a Gaussian-based architecture following VOGS-CP. On Semantic-OPV2V, the voxel and Gaussian implementations improve overall mIoU by 4.35 and 2.39 percentage points over their respective baselines. Both implementations also improve semantic prediction on ego-invisible non-empty voxels, in class-boundary bands, and across the thin-structure class group relative to their respective baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
cooperative perception, semantic occupancy prediction, structural priors, feature fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aHhwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OccWeave, which adopts a frozen SAM images features as priors to improve cooperative semantic occupancy prediction, which currently suffers from boundary distortion, broken continuity, thin-structure omission, and thin-structure thickening. Structure-Prior Feature Refinement (SPFR) associates these features with local 3D representations and adds them through a learned gate. Structure-aware Cooperative Reliability Fusion (SCRF) then weights collaborators using their refined features, refinement residuals, feature discrepancies, and relative poses. OccWeave is implemented and has shown gains on both CoHFF’s voxel representation and VOGS-CP’s Gaussian representation.

Critical strengths
  1. Clear motivation that current CSOP methods suffer from boundary distortion, broken continuity, thin-structure omission, and thin-structure thickening.
  2. OccWeave is generalizable to both voxel and Gaussian representations, and diverse foundation model priors including CLIP/DINOv2/SAM.
Critical weaknesses
  1. VQSOP also falls into Voxel category, yet OccWeave is not implemented and evaluated upon. It would be useful to assess how OccWeave performs upon a stronger Voxel method.
  2. Overhead analysis is missing. CSOP is communication and latency sensitive. OccWeave extracts foundation model priors offline which is not practical in real-world. In addition, SCRF requires collaborators’ refined features, refinement residuals, feature discrepancies, and relative poses, further adds communication overhead.
Questions for authors
  1. Could the authors clarify how much communication and latency overhead does OccWeave have on voxel and Gaussian representations?
  2. Could the authors specify what exact information is transmitted in OccWeave?
Reviewer_fprnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper looks at cooperative semantic occupancy and argues that even after fusing multiple vehicles, predictions still mess up boundaries, thin structures, and continuity. The fix has two parts. SPFR projects frozen SAM ViT-B image embeddings onto each agent’s 3D elements and adds them back through a scalar gated residual. SCRF then scores each CAV at each location using a concatenation of the ego and CAV features, their difference, the prior residual, and the relative pose, and does a softmax over CAVs.

Critical strengths

The problem is interesting. “Which agent should I trust at this location” hasn’t gotten much attention in cooperative occupancy, and the failure cases in Fig. 1 are convincing.

It works on two pretty different representations, voxels and Gaussians, so it’s not just tuned to one baseline.

The ablations are fine-grained, and I appreciated the prior-source comparison (Table A3). Honestly, I think that’s the most informative part of the paper. It shows that the choice of foundation features matters a lot, and that just reusing backbone features doesn’t help.

Critical weaknesses

The novelty is thin, and the core idea doesn’t carry much of the gain. SPFR is basically “lift a strong frozen 2D feature into 3D and add it with a gate.” SCRF is per-location attention-style fusion.

Communication cost isn’t reported. Please report the per-agent communication volume, and ideally compare at a matched budget. SAM inference cost per camera per agent should also be reported.

The Gaussian version starts from a pretrained cooperative VOGS-CP model and is fine-tuned further, but the baseline numbers are the published ones. A “VOGS-CP plus the same extra fine-tuning” control would help.

Questions for authors

Can you report single-agent results?

What’s the per-agent communication volume for both versions, and how do the results look at a budget matched to the baselines?

Reviewer_bRAPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents OccWeave for cooperative semantic occupancy prediction. It uses frozen SAM features to refine local 3D representations through gated residual updates, then learns spatially varying weights for cross-agent fusion. The framework is implemented on CoHFF and VOGS-CP and evaluated on Semantic-OPV2V.

Critical strengths

The framework is evaluated with both voxel and Gaussian representations, showing improvements over the corresponding baselines.

Critical weaknesses
  1. Unfair comparison with the baselines. SPFR contributes most of the IoU gains: 4.90 from SPFR vs. 1.31 from SCRF in the voxel model, and 0.57 from SPFR vs. 0.12 from SCRF in the Gaussian model. However, the comparison does not control for the additional SAM features. It is unclear whether SPFR’s gains come from its design or mainly from the pretrained SAM features. The paper lacks a baseline that uses the same SAM features with simple addition or concatenation.

  2. Limited technical novelty. The method mainly adds gated feature refinement and learned fusion weights to existing cooperative occupancy models. These changes are incremental, and the experiments do not clearly establish their contribution beyond the use of pretrained SAM features.

  3. Limited performance on 3D occupancy. The best OccWeave variant achieves 39.83 mIoU, below VQSOP’s 41.54. Since the paper focuses on cooperative occupancy prediction, Table A1 should be the main comparison in the main paper, with BEV segmentation serving as a supplementary evaluation.

  4. Insufficient validation of the learned weights. The paper lacks direct evidence that the two modules learn appropriate weights. Occupancy performance also drops for some classes, such as Terrain and Bridge, without a clear explanation. The authors should examine whether ineffective weighting contributes to these regressions.

  5. Presentation needs improvement. Clearer figures and explanations would make the paper easier to understand.

Questions for authors

Please address the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.