acceptodds
Under review as a conference paper at ICLR 2027

RCQFusion: Sparse Radar-Camera Query Fusion for Robust 3D Detection

Abstract

Autonomous driving requires reliable perception under diverse and challenging environments. While multi-sensor fusion has become the standard paradigm, exist- ing radar–camera fusion methods largely inherit LiDAR-centric dense BEV-fusion architectures, struggling with radar sparsity, information loss, cross-modal feature misalignment, and high computational cost from dense BEV representations. To address these issues, we present RCQFusion, a sparse radar-camera query fusion framework for 3D detection. Our method introduces a density-aware transformer encoder, a sparse radar-camera fusion module, a multi-modal query generation mechanism, and a hybrid matching module to offer a scal- able and efficient solution for long-range, robust 3D perception in real-world autonomous-driving scenarios. Extensive experiments on nuScenes demonstrate that RCQFusion achieves state-of-the-art radar-camera 3D detection performance, with clear improvements for long-range, dynamic-object, and adverse-condition perception while preserving practical real-time efficiency. Additional evaluation on MAN TruckScenes further shows the generalization ability of RCQFusion to 4D radar sensing and long-range detection scenarios.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Multimodal Learning, Radar–Camera Fusion, 3D Object Detection, Sparse Query Learning, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hWYSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RCQFusion is a sparse radar-camera query-based 3D detector that avoids dense BEV fusion. It combines a density-aware radar Transformer, sampling-enhanced radar-to-image fusion, radar-/camera-conditioned multi-modal query generation, and dynamic one-to-many auxiliary matching. The paper reports strong nuScenes results, long-range/motion/weather analyses, matched runtime comparisons, and additional MAN TruckScenes evaluation.

Critical strengths
  1. The sparse radar-centric design is well motivated. Avoiding dense BEV grids is a natural match for sparse radar measurements, and the matched runtime/memory comparison provides concrete evidence for a useful accuracy-efficiency trade-off.
  2. The paper includes informative distance-, velocity-, weather-, and long-range analyses, which show where radar cues help rather than relying only on aggregate mAP/NDS.
Critical weaknesses
  1. The novelty over RaCFormer is not sufficiently strong under the most controlled comparison. With the same ResNet-50 backbone and 704×256 input, RCQFusion improves mAP only modestly while NDS is identical. Since both methods are query-based radar-camera detectors, the paper needs a sharper explanation of what fundamentally changes beyond a more elaborate radar/query initialization pipeline.
  2. The most relevant sparse radar-camera baseline, SpaRC, is not quantitatively compared even though the paper explicitly discusses it as a sparse non-BEV alternative. For a work whose central claim is sparse radar-camera fusion, omitting a controlled SpaRC comparison is a major gap.
  3. The radar-camera related-work coverage is incomplete. Recent works SGDet3D (RAL 2024), SIFormer (TMM 2026) studies cross-view instance-aware 4D radar-camera fusion and reports results on nuScenes; RaGS (CVPR 2026) also represent advanced rc fusion paradigm. SIFormer is particularly important because a nuScenes comparison is feasible.
  4. The main ablations do not cleanly isolate the claimed contributions. The final row bundles multi-source query generation with hybrid matching, while the density-aware radar encoder is validated mainly in a simplified radar-only setting. Therefore, the paper does not establish how much each major component contributes inside the full RCQFusion detector.
  5. The strongest test-set 'SOTA' number is confounded by backbone/resolution escalation. The 0.640/0.686 result uses a high-resolution ViT/EVA-02 configuration, whereas the matched VoV-99 result is weaker than RICCARDO in NDS and the matched ResNet-50 gain over RaCFormer is small. The paper should separate method gains from backbone scaling and avoid presenting the best configuration as evidence of architectural superiority.
  6. More experiments on view-of-delft, omnihd-scenes, tj4dradset should be conducted, if so, i will improve my score.
Questions for authors
  1. Can the authors provide a controlled comparison with SpaRC under matched backbone, resolution, training schedule, and evaluation settings?
  2. Can the authors separately ablate multi-source query generation and hybrid matching in the full detector, and report the density-aware radar encoder's contribution inside the complete fusion pipeline?
  3. How much of the test-set improvement remains under the same backbone/resolution as the strongest prior radar-camera methods? Please distinguish architectural gains from backbone scaling.
  4. Can the authors conduct more experiments on more datasets, such as view-of-delft, omnihd-scenes, tj4dradset.
2 comments
AuthorsOpenReview ↗

Part I — Architectural Novelty, SpaRC, and Ablations

R1 To W1. Architectural Novelty vs. RaCFormer

Although both methods employ query decoders, their core contributions and fusion architectures differ substantially.

RaCFormer focuses on cross-perspective query-based fusion, introducing circular query initialization, radar-aware depth prediction, and an implicit dynamic catcher to enhance BEV representations. Its decoder repeatedly samples image, radar-BEV, and camera-BEV features, including interactions with seven historical feature frames.

RCQFusion instead develops a unified sparse radar–camera detection pipeline through four connected designs: (1) density-aware point-level radar encoding; (2) radar-anchored camera sampling and sparse fusion before decoding, avoiding dense radar–camera BEV construction; (3) physics-guided heterogeneous queries initialized from radar measurements and radar–camera associations, alongside camera and historical queries; and (4) Dynamic-K matching to address assignment competition among heterogeneous queries.

Unlike RaCFormer's spatially distributed circular queries, our radar-originated queries represent measurement-grounded object hypotheses. Our sparse fusion and query-memory design also avoids repeated interactions with dense historical feature maps.

Under the same ResNet-50 backbone and 704×256 resolution, RaCFormer achieves 54.1% mAP / 61.3% NDS using seven historical feature frames, while RCQFusion achieves 55.0% / 61.3% without historical feature-map interactions. RCQFusion further reduces latency from 216.0 to 56.1 ms and peak memory from 4.55 to 2.76 GB, achieving 3.9× faster inference with 39% lower memory.

For detailed efficiency profiling, see SuzS-R3/Q1. Our framework-level novelty is further explained in rBvM-R5/W3 (Part II).

R2 To W2/Q1. SpaRC Comparison

During our submission preparation, SpaRC was mainly available as an arXiv preprint, with its official publication appearing on September 30, 2026. Our attempted reproduction reached only 26.3% mAP / 43.6% NDS due to missing official code and implementation details, preventing a reliable controlled comparison. See SuzS-R2/W2 for details.

As requested, we include SpaRC's originally reported nuScenes validation results. With R50 (704×256), RCQFusion achieves 55.0%/61.3% vs. SpaRC's 54.5%/62.0% (mAP/NDS); with R101 (1408×512), 57.4%/63.7% vs. 57.1%/64.4%. RCQFusion has slightly higher mAP, while SpaRC has higher NDS.

R3 To W3. Recent Radar–Camera Methods

Our work primarily focuses on sparse 3D radar and compares against methods designed for the same sensing configuration. Unlike 4D radar, conventional 3D radar produces sparse, noisy, and non-uniformly distributed measurements with limited spatial accuracy and elevation information. These characteristics motivate our density-aware encoding, radar-anchored sparse fusion, and physics-guided query generation.

Recent methods such as SGDet3D, SIFormer, CVFusion, and RaGS primarily target 4D radar with richer spatial measurements. SGDet3D employs geometric depth completion and Radar PillarNet; SIFormer introduces depth-guided localization and cross-view instance enhancement; CVFusion uses radar-guided BEV fusion and two-stage proposal refinement; and RaGS constructs Gaussian representations and renders BEV features. In contrast, RCQFusion preserves sparse radar point-level geometry and constructs heterogeneous object queries without dense BEV fusion.

On nuScenes validation, RCQFusion achieves 55.0% mAP / 61.3% NDS, compared with SIFormer's 46.0% / 56.8% under the same ResNet-50 backbone and 704×256 resolution (+9.0 mAP / +4.5 NDS points). Additional comparisons on 4D radar datasets are provided in Part II (W6/Q4).

R4 To W4/Q2. Independent Ablations

The final row of Table 3 is ambiguously labeled: “Multi-source Queries” refers to query sources already introduced in preceding rows, rather than a new module.

Table 3 progressively evaluates radar queries (+1.0 mAP / +0.5 NDS), radar–camera 2D queries (+0.8 / +0.3), and Dynamic-K hybrid matching (+1.7 / +1.7, from 53.9%/60.4% to 55.6%/62.1%). We will rename the final row “+ Dynamic-K Hybrid Matching.” For a detailed breakdown of each component's contribution, please refer to SuzS-R5/Q3.

Our density-aware encoder is structurally coupled with downstream sparse fusion and query generation, making direct replacement with pillar-based encoders impractical without redesigning these modules. Table 6 therefore provides controlled radar-only ablations, where density-aware encoding achieves 21.21%/29.54%, versus 20.31%/27.23% for distance-aware attention (mAP/NDS). For these structural dependencies, see rBvM-R5/W3 (Coherent Representation-to-Prediction Design).

AuthorsOpenReview ↗

Part II. Matched Comparisons, Cross-Dataset Generalization, and Architectural Coherence

R5 To W5/Q3. Backbone Effects and Matched Comparisons

As stated in the main paper, we primarily conduct fair comparisons on the nuScenes validation set, where backbones, input resolutions, and training settings can be better aligned, as many methods do not publicly disclose their complete test-set configurations.

Our strongest nuScenes test result (64.0% mAP / 68.6% NDS) uses EVA-02 ViT at 1600×800. We distinguish this configuration from matched-backbone comparisons to clarify architectural gains.

With VoVNet-99 at 1600×640, RCQFusion achieves 61.2% mAP / 66.3% NDS on nuScenes test, compared with RaCFormer's 59.2% / 65.9%, using Train-only data without future-frame inference.

RICCARDO reports 63.0% / 69.5% on test using Train+Val (~34K frames), whereas our 61.2% / 66.3% model uses Train only (~28K frames). Under the matched Train-only validation protocol with ResNet-101 at 1408×512, RCQFusion achieves 57.4% / 63.7%, compared with RICCARDO's 54.4% / 62.2% (+3.0 mAP / +1.5 NDS points).

These matched comparisons, together with the efficiency results in Part I, demonstrate the accuracy–efficiency advantages of RCQFusion beyond backbone scaling. For RICCARDO's training-only future radar sweeps and the detailed comparison protocol, see rBvM-R2/W2/Q2.

R6 To W6/Q4. Generalization to 4D Radar: TruckScenes and VoD

RCQFusion is primarily designed for sparse and noisy radar measurements, particularly those from widely deployed conventional 3D radars. Its point enhancement and sparse fusion modules specifically address low point density and the absence of elevation information in such radars. We further evaluate its generalization to 4D radar datasets as reported in the supplementary material of our original submission.

MAN TruckScenes. RCQFusion achieves 23.9% mAP / 36.0% NDS with 900 queries, increasing to 24.7% / 38.6% with 1600 queries. For comparison, Horizon3D (arXiv submitted June 30, 2026) reports CRT-Fusion (16.9% / 28.8%), BEVFusion (18.2% / 30.4%), and SpaRC stated (22.5% / 35.4%) on TruckScenes.

View-of-Delft (VoD). VoD has only one front camera, providing limited coverage that is unfavorable for our projection-based radar–camera fusion, as many radar points cannot be associated with camera features. Nevertheless, with only a simple adaptation, RCQFusion achieves competitive results using ResNet-50 and 24 training epochs, retaining its core sparse fusion and heterogeneous query design.

Under the AP11 protocol, our new results are:

RegionCar APPed. APCyc. AP3D mAP
Entire49.56%54.12%71.72%58.47%
ROI69.26%64.59%89.15%74.34%

VoD Entire 3D mAP (ResNet-50):

Method3D mAP
RCFusion49.65%
RCBEVDet49.99%
RaCFormer54.44%
RCQFusion (Ours)58.47%
SGDet3D59.75%
SIFormer60.18%
CVFusion60.29%

SGDet3D uses geometric depth completion and Radar PillarNet; SIFormer combines depth-guided localization and cross-view instance enhancement; and CVFusion adopts radar-guided BEV fusion and two-stage proposal refinement.

RCQFusion achieves 58.47% Entire mAP / 74.34% ROI mAP through a simple adaptation of our original sparse 3D radar–camera framework, outperforming existing 3D radar–camera fusion methods, including RCFusion, RCBEVDet, and RaCFormer.

Notably, these results are achieved without 4D radar-specific tuning for higher point density or elevation information, complex temporal feature interactions, multi-stage pretraining or fine-tuning, or auxiliary LiDAR-based depth supervision. Together with TruckScenes, these results demonstrate the generalization of RCQFusion across different radar sensing configurations.

VoD adaptation is ongoing. We will release the code and weights via an anonymous link before the rebuttal deadline.

TJ4DRadSet and OmniHD-Scenes. Similar to TruckScenes, both provide 4D radar data, but access approval may take over a week. We are applying for access and will add results in the revision once available.

R7. Architectural Coherence and Revision

RCQFusion forms a unified sparse pipeline: density-aware encoding preserves radar geometry and motion cues; radar-anchored fusion incorporates camera semantics; heterogeneous queries generate complementary object hypotheses; and Dynamic-K matching resolves assignment competition.

We will add approximately one page to the revised PDF's main text to clarify these structural connections and distinguish RCQFusion from dense-BEV fusion, RaCFormer, query-based methods, and recent 4D radar fusion and refinement approaches.

Reviewer_rBvMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RCQFusion, a Sparse Radar–Camera Query Fusion framework for 3D Object Detection, aiming to address the high computational cost and cross-modal feature misalignment associated with existing dense BEV-based fusion methods when processing sparse radar point clouds. The framework consists of four main components: a Density-Aware Transformer Backbone, Sample Enhancement and Sparse Radar–Camera Feature Fusion, Multi-Modal Query Generation, and Multi-Source Query Hybrid Matching. By combining radar geometric and motion cues with visual semantic features, the method constructs multi-source Object Queries and employs Dynamic-K 1vN auxiliary supervision to improve training. Experiments on nuScenes and MAN TruckScenes demonstrate competitive detection accuracy and inference efficiency, with notable improvements in long-range and moving-object detection.

Critical strengths
  1. The proposed method leverages radar geometric priors to construct multi-modal Object Queries without introducing an additional dense BEV fusion representation. It substantially reduces inference latency and memory consumption while maintaining competitive detection accuracy, demonstrating practical value.

  2. The paper conducts extensive evaluations on nuScenes and MAN TruckScenes, including comparisons with existing methods, module-wise ablations, runtime analysis, and detection performance under different distance ranges, motion states, and weather conditions. These experiments provide useful evidence supporting the effectiveness of the proposed framework.

Critical weaknesses
  1. The proposed Multi-Modal Query Generation and Dynamic-K Hybrid Matching are closely related to existing approaches, including Far3D, RaCFusion, H-DETR, and Group DETR. Although the ablation studies demonstrate the effectiveness of individual components, the paper does not sufficiently clarify their key technical distinctions from prior methods or identify the independently novel contributions.

  2. The performance advantage on the nuScenes test set and the fairness of the experimental comparisons require further clarification. In Table 2, under the same VoVNet-99 backbone and input resolution, RCQFusion achieves lower mAP/NDS (61.2%/66.3%) than RICCARDO (63.0%/69.5%), which raises questions about the claimed SOTA performance. Furthermore, RICCARDO uses future radar sweeps for constructing training supervision, while inference relies on current and historical radar sweeps. The (+6) notation in the paper may therefore be misleading. Additionally, Appendix D shows that the inference latency difference between RCQFusion and RaCFormer mainly comes from the decoder. The contribution of the sparse fusion design itself to the efficiency improvement remains unclear.

  3. The paper does not include a dedicated Related Work section in the main text, placing the systematic discussion in Appendix B instead. Although the Introduction cites several existing studies, the technical distinctions between RCQFusion and related Sparse Radar–Camera Fusion, Query-based Detection, and Hybrid Matching approaches are not sufficiently discussed in the main text, making it harder to assess the method's positioning and novelty.

Questions for authors
  1. Novelty and Related Work. The proposed Multi-Modal Query Generation and Dynamic-K Hybrid Matching are technically related to Far3D, RaCFusion and Group DETR. Could the authors clarify the key innovations of RCQFusion compared with these approaches and provide a more systematic discussion of the technical differences in the main text?

  2. Table 2 shows that RCQFusion achieves lower mAP and NDS than RICCARDO under the same VoVNet-99 backbone and input resolution. How do the authors interpret this result, and what are the main advantages of RCQFusion over RICCARDO? Furthermore, RICCARDO is marked with (+6), which is described as indicating the use of future frames. However, the original RICCARDO paper indicates that inference uses current and historical radar sweeps. Could the authors clarify the meaning of this notation and specify the temporal information and radar sweep configurations used by both methods during training and inference?

  3. Appendix D shows that the inference latency difference between RCQFusion and RaCFormer primarily comes from the decoder. Could the authors clarify the differences in query count, decoder layers, and attention implementation, and further analyze their respective contributions to inference efficiency? Additionally, Table 9 in Appendix C reports only 0.2% mAP for Far3D's camera 2D queries, compared with approximately 50.3% mAP for the proposed multi-modal queries. Could the authors explain this substantial gap and clarify whether the two query configurations are comparable in terms of query count, training supervision, and inference conditions?

2 comments
AuthorsOpenReview ↗

Part I Query Novelty, RICCARDO Fairness, Decoder Efficiency, and Architectural Coherence

R1 To W1/Q1. Novelty relative to Far3D, RaCFusion, H-DETR, and Group DETR.

Physics-guided heterogeneous queries. Far3D initializes 3D queries from camera 2D proposals and estimated depth; RaCFusion uses radar priors to enhance camera-centric queries and refine velocity. RCQFusion instead also generates independent radar-originated object hypotheses using measured range, Doppler, RCS, and temporal consistency. It further constructs radar–camera association queries by selecting peaks from radar-conditioned depth distributions within camera 2D boxes, alongside camera-only, historical, and learnable queries. Radar therefore serves not merely as an auxiliary feature for camera proposals, but as a direct source of object hypotheses. Relative to the camera-only baseline, nuScenes validation mAP increases from 8.4% to 16.0% for 40–60 m objects and 25.0% to 33.7% for moving objects.

Matching heterogeneous queries. H-DETR introduces auxiliary one-to-many supervision; Group DETR uses multiple query groups during training. Our heterogeneous query sources can produce competing hypotheses for the same GT, with different geometric priors and confidence. RCQFusion applies quality-adaptive Dynamic-K assignments to the existing query set instead of requiring additional query groups or a fixed number of positives per GT. With matched queries and training settings, 1v1 Hungarian, naive 1vN, and Dynamic-K yield 53.9%/60.4%, 55.1%/61.2%, and 55.6%/62.1% mAP/NDS. N-group + Dynamic-K reaches 56.0%/62.7%, demonstrating complementarity; additional groups increase our training time by approximately 35–45%. We do not claim to invent one-to-many matching. The contribution is its quality-adaptive use for physics-grounded heterogeneous radar–camera queries within a unified sparse detection pipeline.

R2 To W2/Q2. RICCARDO: test fairness and temporal inputs.

We previously misunderstood RICCARDO's six future radar sweeps as inference inputs, by analogy with temporal fusion methods such as RaCFormer and RCBEVDet. The reviewer correctly notes that they are used only to construct training supervision. We will relabel (+6) as training-only future radar sweeps. At inference, neither method uses future frames; RCQFusion uses current/past radar measurements, current camera features, and limited historical query memory.

Importantly, RICCARDO's reported 63.0% mAP / 69.5% NDS test model was trained on Train+Val (~34K frames), whereas our 61.2% / 66.3% model used Train only (~28K frames). RICCARDO does not report a Train-only test result. Under the Train-only validation protocol, with the same ResNet-101 backbone and 1408×512 input, RCQFusion achieves 57.4%/63.7%, versus RICCARDO's 54.4%/62.2% (+3.0/+1.5 percentage points). Additional Val training data may contribute to the test gap, so the test comparison is not a controlled architecture comparison. We are training a high-resolution VoVNet-99 model on Train+Val; however, official nuScenes submissions online test are limited, and in our experience test results take 1–2 weeks to return, so new results may not arrive within rebuttal.

R3 To W2/Q3(a). Decoder efficiency VS RaCFormer.

As detailed in our SuzS-Q1 response, RaCFormer's six-layer decoder repeatedly samples image, radar-BEV, and LSS-BEV features, including temporal interactions with seven historical feature frames. Our AMP FP16 profiling measures 178.77 ms decoder latency, including 56.99/53.52/33.55 ms for the three sampling operations (144.06 ms total) versus 4.39 ms for self-attention. RCQFusion performs radar-anchored sparse fusion and query preparation before decoding (9.0 ms) and uses limited historical query memory rather than repeatedly sampling dense historical maps. Both use comparable query counts and decoder depth; the main difference is repeated dense multi-modal/temporal feature access, not ordinary attention alone. Our matched profiling reports 56.1 ms / 2.76 GB for RCQFusion versus 216.0 ms / 4.55 GB for RaCFormer.

R4 To Q3(b). Queries Compare in Table 9.

The reported 0.2% mAP is not Far3D's full result: its complete query set achieves 50.7% mAP. The 0.2% entry evaluates predictions from the additional camera-2D query source separately. Likewise, RCQFusion's multi-source query subset achieves 50.3% mAP, while its full query set achieves approximately 55.6%. Both experiments retain learnable/historical queries and the training supervision; Table 9 evaluates source-specific predictions, not the marginal gain of removing/retraining query sources. We will explicitly clarify the Top-K selection and source-wise evaluation protocol.

To W3. Related-work positioning.

The technical distinctions above are currently detailed mainly in Supplementary B. We will move a concise comparison into the revised PDF's main text, including the architectural rationale described in Part II below.

AuthorsOpenReview ↗

Part II Additional Clarification on Architectural Coherence and Framework-Level Novelty

R5 To W3. Related-work positioning and framework-level novelty

Coherent representation-to-prediction design. RCQFusion is not simply a collection of a radar encoder, feature sampler, query initializer, and matching loss. It is designed around one constraint:sparse, noisy, nonuniform radar measurements specially for 3d radar provide physically meaningful range, Doppler, and RCS cues, yet do not justify constructing and repeatedly processing a dense radar–camera BEV representation. We therefore preserve radar point-level geometry and carry its priors all the way from sensing to object-level prediction.

  1. Representation: Density-aware point encoding retains radar geometry, local density, and motion cues as anchors for downstream association; it does not require dense radar–camera BEV construction.
  2. Cross-modal alignment: Radar-anchored virtual sampling locations gather camera semantics near uncertain radar projections; these locations are sampling supports, not fabricated radar measurements. Sparse residual fusion supplements the original radar representation rather than replacing it.
  3. Object hypotheses: Physics-grounded radar queries and radar–camera depth-peak association queries transform these measurements and aligned semantic features into complementary object candidates, while camera-only, historical, and learnable queries preserve coverage when radar returns are missing.
  4. Optimization: Such heterogeneous hypotheses can correspond to the same GT and compete under strict 1v1 matching. Quality-adaptive Dynamic-K provides auxiliary positives according to geometric/IoU match quality without mandatory query duplication.

The dependency is structural: point-level encoding supplies geometric anchors; radar-anchored sampling supplies visual semantics at those anchors; the resulting representation supports heterogeneous object hypotheses; and the heterogeneity motivates quality-adaptive supervision. This is a unified sparse representation → alignment → query construction → assignment pipeline, not a claim that each building block establishes a new standalone paradigm.

Empirical support. Progressive ablations identify radar queries, fusion/sampling, association queries, and matching gains; controlled Dynamic-K comparisons isolate the matching choice. Matched RaCFormer comparisons show that comparable accuracy can be obtained with substantially less latency and peak memory (56.1 ms / 2.76 GB versus 216.0 ms / 4.55 GB). The point is the framework's accuracy–efficiency trade-off and consistent use of radar priors, rather than attributing all gains to any one component.

Revision plan. We will add approximately one page to the main text of the revised PDF devoted to this architectural motivation, a step-by-step diagram of the representation-to-prediction dependencies, and a compact comparison against Far3D, RaCFusion, H-DETR/Group DETR, and dense-BEV fusion paradigms. This material will be moved out of an appendix-only discussion to make the paper's novelty and positioning explicit in the main manuscript.

Reviewer_SuzSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a 3D detection model for autonomous driving that uses a sparse radar-camera query fusion framework. They design a transformer-based density-aware trans-encoder network to extract features from sparse radar point cloud, and a feature fusion module to fuse them with visual features effectively. In addition, they design a multi-modal query generation mechanism to improve spatial sampling and a hybrid match module to improve the stability of set-to-set prediction. By evaluating on nuScenes dataset, they demonstrate improved accuracy compared to previous methods, and strong cross-dataset generalization to MAN TruckScenes.

Critical strengths
  1. The paper is well-written, with a clear explanation of the motivation and method.
  2. The detection accuracy in nuScenes is competitve, with a notable good performance for moving and more distant objects.
  3. The method demonstrates good model efficiency in terms of latency and memory.
Critical weaknesses
  1. For evaluation on MAN TruckScenes in table 11 and table 12, the author only compares with a LiDAR only approach CenterPoint-V and a camera-only approach Far3D, whereas the proposed method uses both radar and camera, which makes it difficult to say if the better performance is due to the proposed architecture or more input information.
  2. SpaRC is an important related work that does not use dense BEV fusion. Although the author comments on difference with SpaRC in supplementary B2, there is no quantitative results for comparison, especially in terms of efficiency.
Questions for authors
  1. What is the cause of the large decoder latency difference between RCQFusion and RaCFormer in table 10? How does it compare to other methods like SpaRC?
  2. In Eq2, for a masked attention logit, after being multiplied by a binary mask of value 0, softmax operation still gives a nonzero softmax probability. In this case how does the mask prevent attention?
  3. In table 3, the multi-source queries and hybrid matching are added together; is it possible to separately add them to the model and report the result with their individual contribution?
1 comment
AuthorsOpenReview ↗

Ablation Clarifications, Decoder Efficiency, SpaRC Reproduction, and Cross-Dataset Evaluation

R1 To W1. MAN TruckScenes comparison and generalization.

MAN TruckScenes is a new multi-camera, multi-4D-radar dataset with few radar–camera baselines. We use it to test transfer from nuScenes radar to 4D radar; our main ablations remain on nuScenes.

Horizon3D (arXiv, June 30, 2026) recently reported MAN TruckScenes results for BEVFusion (16.9% mAP / 28.8% NDS), CRT-Fusion (18.2% / 30.4%), and SpaRC (22.5% / 35.4%). RCQFusion with 900 queries achieves 23.9% / 36.0%. Increasing queries to 1600 for longer ranges and more classes improves this to 24.7% mAP / 38.6% NDS, exceeding these reported results.

Following Reviewer hWYS05's suggestion to evaluate additional datasets, we also conducted an initial adaptation to View-of-Delft (VoD). Under AP11, RCQFusion achieves 58.47% Entire mAP /74.34% ROI mAP. Its Entire mAP exceeds reported results for RCFusion (49.65%), RCBEVDet (49.99%), and RaCFormer (54.44%), supporting cross-dataset generalization.

R2 To W2. SpaRC reproduction and technical concerns.

We recognize SpaRC as relevant. We followed its arXiv version during submission; its IEEE Xplore record appeared on September 30, 2026. Despite unavailable official code/weights, we attempted reproduction from the paper and emailed the authors without a reply. Our reproduction reached only 26.3% mAP / 43.6% NDS, substantially below its reported results.

This raises questions about Sparse Frustum Fusion. As we understand it, SpaRC uses image feature columns as queries and associated Top-K radar points as keys/values to construct sparse frustum-fused features. Its decoder then interacts with those features and radar features, without direct object-query interaction with the original dense image features. Given the extremely sparse radar returns in nuScenes, we question whether this design sufficiently preserves dense camera semantics. Excessive reliance on sparse radar-associated features may discard visual information and impair detection. We cannot determine whether the gap stems from missing implementation details or the fusion design. We thus excluded our unofficial reproduction from quantitative comparisons; missing details also preclude fair latency comparisons.

R3 To Q1. Efficiency Analysis: RCQFusion vs. RaCFormer Decoder.

RaCFormer uses the current frame plus 7 historical frames for temporal feature interactions. Our AMP FP16 profiling of its six-layer decoder measures 178.77 ms(run differs slightly from Table 10's averaged timing): image sampling 56.99 ms, radar-BEV sampling 53.52 ms, LSS-BEV sampling 33.55 ms, adaptive mixing 27.50 ms, and self-attention 4.39 ms. Feature sampling alone takes 144.06 ms (80.6%); the dominant cost.

RCQFusion instead performs radar-anchored sparse fusion before decoding (9.0 ms for fusion and query preparation, Table 10). Its decoder uses current-frame image features and limited historical query memory, without decoding over 7 historical dense feature maps or repeated BEV sampling. These differences explain its latency and memory advantages. Matched SpaRC timing cannot be established without runnable code or complete timing configurations.

R4 To Q2. Attention Mask in Eq. (2).

The reviewer is correct that multiplying attention logits by a binary mask does not ensure zero attention probabilities after Softmax. This is a notation error. In our implementation, masked logits are set to negative infinity before Softmax. The correct formulation is:

Attn⁡(l)=Softmax⁡(QK⊤dk+B(l))V\operatorname{Attn}^{(l)}=\operatorname{Softmax}\left(\frac{QK^\top}{\sqrt{d_k}}+B^{(l)}\right)V

where Bij(l)=0B_{ij}^{(l)}=0 for valid connections and Bij(l)=−∞B_{ij}^{(l)}=-\infty for masked connections. Thus, masked positions receive zero attention weights. We will correct Eq. (2), without affecting the implementation or experimental results.

R5 To Q3. Separate contributions of multi-source queries and hybrid matching.

The final row of Table 3 is ambiguously labeled. “Multi-source queries” is an umbrella term for radar, radar–camera, and other query sources already introduced in preceding rows; it is not a new module added in the final row. Table 3 uses progressive ablations:

  • Radar queries: +1.0 mAP / +0.5 NDS percentage points;
  • Cross-fusion features: +0.6 / +0.3 points;
  • Sampling enhancement: +0.8 / +0.6 points;
  • Radar–camera 2D queries: +0.8 / +0.3 points;
  • The final row keeps the same query configuration and only replaces 1v1 with Dynamic-K hybrid matching, improving 53.9% mAP / 60.4% NDS to 55.6% / 62.1% (+1.7 / +1.7 points).

Thus query generation and hybrid matching are already evaluated separately. The final row should read “+ Dynamic-K Hybrid Matching.” Table 8 compares 1v1 Hungarian (53.9% / 60.4%), naive 1vN (55.1% / 61.2%), Dynamic-K (55.6% / 62.1%), and N-group Dynamic-K (56.0% / 62.7%), quantifying the contribution of the matching strategy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.