- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The core idea of this paper is to improve the efficiency of LiDAR–camera 3D object detection by concentrating computation on informative LiDAR observations and reducing the cost of cross-modal interaction, rather than proposing a fundamentally new detection paradigm. Specifically, it introduces response-guided sampling, which uses image detection heatmaps and a 3D geometric dilation margin to select a small but informative subset of LiDAR points while preserving nearby geometric context; then it applies kernelized linear cross-attention with an ELU+1 kernel, where image features are first summarized into a compact global representation so that LiDAR queries can retrieve image information without constructing a dense pairwise attention matrix, reducing the interaction cost from (O(hMLd)) to . Finally, during training, a frozen full-input TransFusion teacher with an uncertainty head supervises a lightweight student through uncertainty-aware distillation, giving more weight to difficult examples while removing the teacher at inference. The overall idea is therefore a coordinated system design that combines image-guided input reduction, efficient linear fusion, and training-only knowledge transfer to achieve a favorable accuracy–efficiency trade-off for real-time LiDAR–camera 3D detection.
Critical strengths
This paper presents LMF-3DDet, a lightweight LiDAR–camera 3D detection framework that jointly addresses input selection, cross-modal fusion, and knowledge transfer. Its response-guided sampling effectively reduces the LiDAR token budget while preserving detection accuracy, and the kernelized linear cross-attention lowers the cost of image-to-LiDAR interaction from quadratic to linear scaling in token count, yielding a favorable accuracy–efficiency trade-off. The uncertainty-aware distillation further improves the lightweight student without adding inference cost, and experiments on nuScenes and Waymo demonstrate real-time throughput of around 45 FPS with ablations validating the sampling, fusion, and distillation components. The system design is practically relevant for efficient autonomous-driving perception.
Critical weaknesses
The work is largely incremental, as it combines existing ideas—image-guided sampling, linear attention with an ELU+1 kernel, and uncertainty-aware distillation—without introducing a fundamentally new viewpoint or technical breakthrough. The reported accuracy remains below state-of-the-art multi-camera fusion methods, and the evaluation is limited to a single-camera setting. RGS depends on image responses and calibration, so degraded images or projection errors may discard useful LiDAR observations, and the uncertainty weighting may emphasize unreliable teacher targets. The comparisons with random sampling change both the selection policy and token budget, making it difficult to isolate the effect of the sampling strategy. Speed comparisons are not based on a uniform benchmark, and the paper lacks repeated-seed evaluations, embedded-device latency, energy measurements, and class-specific robustness analysis. Thus, while the engineering integration is useful, the novelty and empirical rigor are limited.
Questions for authors
See Weakness.
3 comments
I think this paper is indeed a solid piece of work, but for ICLR the acceptance bar is indeed quite high. I do not think the core motivation of this paper merits a score of 3; for a lightweight work without particularly interesting points, to me I can only give it a 2.
I think this paper is indeed a solid piece of work, but for ICLR the acceptance bar is indeed quite high. I do not think the core motivation of this paper merits a score of 3; for a lightweight work without particularly interesting points, to me I can only give it a 2.
Clarifying the Contribution and Novelty of LMF-3DDet
Thank you for your candid assessment and for recognizing that this is a solid piece of work. We understand your concern that, despite being technically sound, the paper may not offer the level of conceptual novelty expected at ICLR.
We would like to clarify the scope of our claim. We do not present LMF-3DDet as a new detection paradigm, nor do we claim novelty for linear attention or knowledge distillation in isolation. Our contribution is narrower: we study how to allocate computation across LiDAR input selection, cross-modal interaction, and training-time supervision in a lightweight single-camera detector. We agree that combining established components is not, by itself, sufficient evidence of a meaningful contribution. The key question is whether the proposed design provides a measurable benefit under controlled comparisons.
In the revision, we will make this positioning more explicit and strengthen the evidence with matched-budget sampling controls, a configuration matrix that separates the effects of the individual components, and repeated-seed results for the principal comparisons. We will also clearly distinguish measured results from claims that remain unverified, and limit our efficiency claims to the tested hardware and camera setting.
We appreciate your point that these clarifications may not change your assessment of the paper’s novelty. We nevertheless hope that the more precise scope and controlled evidence will help you judge the contribution on its intended terms. If, after considering these clarifications, you still find that the work does not meet ICLR’s novelty bar, we understand and respect your assessment.
Clarifying the System Contribution and Strengthening Experimental Evidence
We thank the reviewer for recognizing the practical relevance and accuracy–efficiency trade-off of LMF-3DDet. We understand the concerns about incremental novelty and experimental controls. We clarify the contribution and distinguish demonstrated results from limitations below.
Contribution Beyond Component Aggregation. As stated in Sections 1 and 3.3, we do not claim a new detection paradigm or a new linear-attention identity. Our contribution concerns how computation is allocated across input selection, cross-modal interaction, and training supervision in a lightweight detector. These components address different constraints: RGS reduces the number of LiDAR queries; kernelized attention reduces interaction cost at a given query count; and KD improves supervision without retaining the teacher at inference. RGS and linear attention therefore act on complementary sources of computation, while distillation supports the reduced-capacity student. This motivates their joint design, although it does not by itself prove an empirical synergy. The existing experiments provide concrete operating points. Table 3 reports 69.7 mAP at 44.5 FPS with 11.5k tokens, compared with 68.5 mAP at 32.1 FPS for 30k random sampling. Table 4 shows a throughput increase from 32.5 to 44.5 FPS with a 0.2-point mAP decrease. We agree that these results do not fully isolate every component. We will sharpen the contribution statement and add a configuration matrix separating individual and combined changes, with a fixed student and training protocol wherever applicable.
Accuracy, Camera Configuration, and Speed Comparisons. We acknowledge that our reported accuracy is below recent multi-camera fusion methods. Section 4.2 explicitly reports this gap and presents our single-camera model as a lightweight operating point, rather than the highest-accuracy detector. A single-camera configuration does not establish superiority under matched inputs. We will annotate modality, camera count, temporal inputs, and timing conditions in the comparison tables. Published FPS values will remain contextual references, not evidence of a controlled speed ranking. Additional matched-setting comparisons will be distinguished from literature results.
Sampling Controls and Robustness. We agree that Table 3 changes both selection policy and token budget, as already acknowledged in Section 4.3 and the Limitations section. To isolate selection quality, we will add random and farthest-point sampling controls matched to the RGS retained-point count per frame, alongside a full-input baseline and a no-dilation variant. Raw point counts and encoded token counts will be distinguished. The paper also acknowledges that dilation cannot guarantee foreground coverage under degraded images or calibration errors. We will quantify object-level point retention by distance and available visibility annotations, reporting objects with no original LiDAR points separately. Coverage alone will not be treated as detection robustness; class- and distance-specific detection results are also needed. Controlled image degradation and calibration perturbations would further test this limitation, and we will distinguish measured evidence from unresolved robustness questions.
Uncertainty Weighting and Statistical Reliability. Section 3.5 explicitly describes the uncertainty weight as hard-example emphasis, not teacher-confidence filtering. Its range, from 0.5 toward approximately 0.9933, limits relative weighting to less than twofold, but does not guarantee that uncertain targets are reliable. We will clarify this trade-off and use same-architecture no-KD, uniform-KD, and uncertainty-KD controls, including a uniform control matched to average effective KD strength. Table 5 contains configuration differences, so its 2.2-point improvement cannot be attributed to KD alone. We also agree that single-run differences do not establish statistical reliability. Repeated-seed results are particularly important for interpreting the 0.2-point attention difference and the 0.6-point distillation difference. We will report means and variability for the principal controlled comparisons rather than imply that these differences are established beyond training variation.
Deployment Scope and Presentation. Our current measurements establish RTX 4090 throughput, not embedded-device latency or energy efficiency. Fewer tokens and lower interaction complexity do not, by themselves, demonstrate lower energy consumption. We will limit deployment claims accordingly and identify embedded and energy evaluation as remaining work unless measured results are available.
To improve readability, we will add a compact configuration table and more explicitly connect each design choice to its supporting experiment and limitation. Essential clarifications will appear in the manuscript, with supporting configurations, measurement procedures, and available supplementary results provided in an anonymous repository.