acceptodds
Under review as a conference paper at ICLR 2027

SGDet3D++: Geometry-Grounded Semantics for 4D Radar-Camera 3D Object Detection

Abstract

4D radar complements dense image semantics with long-range geometry and radial motion, but existing radar–camera detectors largely solve where to align the modalities while leaving whether a piece of evidence supports an evolving object hypothesis implicit. An image token may describe an occluder, a nearby radar return may belong to another object, and a pose-aligned memory slot may carry incompatible motion. We formulate hypothesis-conditioned evidence grounding, which separates candidate access from evidence use: semantic, geometric, or temporal evidence is filtered or conditioned by the evolving 3D state before updating the corresponding query. instantiates this principle through Anchor-Grounded Semantic Retrieval (AGR), which conditions deformable image retrieval on pooled anchor-consistent radar support; Geometry-Consistent Anchor Refinement (GCR), which attentively aggregates individual associated returns; and Doppler-Verified Correspondence (DVC), which replaces history only when current radial motion contradicts it. improves the strongest compared method by 3.82 mAP and 6.82 ODS on OmniHD-Scenes and by 6.82 mAP and 9.22 NDS on ManTruckScenes, while also leading the listed methods in the TJ4DRadSet test comparison. Mechanism-targeted evaluations show that AGR improves strict AP in every projected-occlusion bin, the yaw-aligned box gate raises target-return purity from 29.95% to 58.87%, and DVC preserves 96.11% of motion-consistent history while retaining 75.90% contradiction recall. Code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D radar

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_R9byOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SGDet3D++ advances 4D radar-camera fusion from scene-level BEV alignment to object-level integration conditioned on dynamic 3D hypotheses. It employs AGR, GCR, and DVC to constrain semantic retrieval, geometric refinement, and temporal memory inheritance, respectively, yielding performance gains on the OmniHD-Scenes, ManTruckScenes, and TJ4DRadSet datasets.

Critical strengths

(1) Advancing 4D radar-camera fusion from scene-level BEV alignment to object-level fusion conditioned on dynamic 3D hypotheses.

(2) Achieving strong performance on the OmniHD-Scenes, ManTruckScenes, and TJ4DRadSet datasets.

Critical weaknesses

(1) The proposed components have already been widely used in camera-LiDAR fusion methods. For example, OPEN [1] projects LiDAR points to generate sparse GT depth map for camera backbone and uses 3D object queries to detect 3D objects. SparseFusion [2] adopts bi-directional LiDAR and camera modality fusion strategy. Sparse4D [3] adopts 3D deformable attention to aggregate local features. I don not think the proposed method is novel compared to previous methods.

(2) The paper lacks some implementation details. For example, there is not the description of image and radar backbones. This makes a fair comparison with previous methods impossible, as it is unclear whether the backbone scale and the models are the same.

(3) The authors identify RaCFormer and CVFusion as the most relevant methods for object-level fusion. However, RaCFormer is absent from all primary tables, ablation studies, and robustness experiments for both OmniHD-Scenes and ManTruckScenes, while CVFusion is evaluated only on the TJ4DRadSet dataset. Given that the paper’s core argument is that existing query/proposal-based methods fail to explicitly verify evidence–hypothesis compatibility, the omission of these two architecturally similar baselines is a significant gap. The authors should include results for these two methods on the two primary datasets.

(4) Table 5 presents the mean and standard deviation derived from three independent runs. However, for the results in Tables 1–3 and those concerning TJ4DRadSet, it is not explicitly stated whether each method’s results represent a single run, a specific checkpoint, or the mean across multiple random seeds.

[1] OPEN: Object-wise Position Embedding for Multi-view 3D Object Detection

[2] Sparse4D v3: Advancing End-to-End 3D Detection and Tracking

[3] SparseFusion: Fusing Multi-Modal Sparse Representations for Multi-Sensor 3D Object Detection

Questions for authors

(1) What are the differences between this method and existing camera-LiDAR fusion-based 3D object detection methods?

Reviewer_4vchOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SGDet3D++, a 3D object detection method using 4D radar and cameras that focuses on consistency between object hypotheses and observations. SGDet3D++ uses AGR to adjust image feature retrieval weights, GCR to aggregate radar returns and update bounding boxes, and DVC to replace historical states that conflict with Doppler observations. The authors evaluated its performance on OmniHD-Scenes, ManTruckScenes, and TJ4DRadSet.

Critical strengths
  • A strength is the unified design of image feature retrieval, radar return aggregation, and temporal inheritance based on consistency with object hypotheses. The method goes beyond spatial alignment and provides concrete design guidelines for selecting useful information for each object.
  • On OmniHD-Scenes, the method improves mAP from 40.43 for the common baseline to 48.87. DVC also improves mAP by 2.75 points in a comparison with the same memory capacity. These controlled experiments support the effectiveness of the added modules.
Critical weaknesses
  • Radar-return purity is evaluated only for true positives with a center error within 1 m. It remains unclear how many target returns are retained when the anchor is misaligned, or whether subsequent updates can correct the error. There is insufficient evidence that the method can select appropriate information in failure cases.
Questions for authors
  • Which decoder layer’s anchors are used to calculate the purity in Table A6, and are they taken before or after the update? Please also clarify whether the same cases are evaluated across the compared settings.
  • Could you also report the target-return retention rate and the error reduction after the update, grouped by the error before the update? Please include failure cases in this analysis.
Reviewer_WjDDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SGDet3D++, a 4D radar-camera fusion framework for 3D object detection. Building upon SGDet3D, the method introduces a sparse object-query decoder and proposes hypothesis-conditioned evidence grounding. Three modules, AGR, GCR, and DVC, are designed to guide image feature retrieval using radar information, refine 3D bounding boxes, and verify the motion consistency of historical object states, respectively. The method achieves competitive performance on OmniHD-Scenes, ManTruckScenes, and TJ4DRadSet, with ablation studies, robustness evaluations, and efficiency analyses supporting its effectiveness.

Critical strengths
  1. The method achieves notable improvements in detection performance on OmniHD-Scenes and ManTruckScenes, while maintaining relatively low inference latency and computational cost, demonstrating its practical potential.

  2. The experimental evaluation is comprehensive. In addition to standard ablation studies, the paper includes analyses of occlusion, radar-return association, temporal memory, and robustness, which help clarify the contributions of individual components.

Critical weaknesses
  1. Table A6 reports a motion-conflict detection precision of only 5.70%, although DVC improves overall detection performance. Further analysis would help clarify how much of this improvement comes specifically from Doppler-based verification.

  2. Although Appendix I discusses the differences from existing object-centric fusion methods, the benefit of recomputing anchor-consistent radar support at each decoder layer could be more clearly demonstrated.

Questions for authors
  1. How does DVC compare with random memory refresh at the same replacement rate? Does Doppler verification provide additional benefits?

  2. What is the benefit of recomputing anchor-consistent radar support at each decoder layer compared with keeping it fixed across layers?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.