- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes OMT-Distill, an object-level knowledge distillation framework for heterogeneous 3D object detection. The paper considers the setting where a multimodal teacher transfers knowledge to a LiDAR-only student with a different detector architecture, and argues that decoded 3D object predictions provide a common interface for knowledge transfer. The proposed Object-Measure Transport (OMT) consists of two main components: teacher-location supervision, which supervises student responses at teacher object locations, including locations outside the student’s native top-k candidates, and an unbalanced optimal transport formulation that establishes soft many-to-many correspondences between teacher and student predictions based on their confidence, spatial distance, box geometry, and class compatibility. OMT is further combined with feature, response, and candidate-level distillation to form OMT-Distill. Experiments on nuScenes across several heterogeneous teacher–student pairs show improvements over the considered distillation baselines. In the main BEVDilation-to-CenterPoint-Voxel setting, OMT-Distill achieves 61.07 mAP and 67.92 NDS, improving over the HRCA baseline by 1.01 mAP and 0.39 NDS. The results support the paper’s claim that decoded object-level supervision can provide an effective interface for distillation between heterogeneous 3D detectors.
Critical strengths
-
Well-motivated object-level distillation formulation. The paper identifies the difficulty of distilling knowledge between heterogeneous 3D detectors with different encoders, decoders, and prediction heads, and uses decoded 3D objects as a shared semantic and geometric interface for knowledge transfer. This provides a clear motivation for the proposed framework.
-
Teacher-location supervision addresses candidate coverage limitations. A key strength is that the method supervises student responses at teacher object locations, including locations outside the student’s native top-k candidates. This allows teacher knowledge to influence regions that would otherwise be excluded by the student’s own candidate selection.
-
Flexible matching with encouraging empirical results. The unbalanced optimal transport formulation enables soft many-to-many associations between teacher and student predictions while accommodating differences in prediction coverage and confidence. Experiments across multiple heterogeneous teacher–student pairs show consistent improvements over the considered baselines, with a 1.01 mAP and 0.39 NDS gain over HRCA in the main setting.
Critical weaknesses
-
The necessity of optimal transport is not sufficiently established. Although the paper attributes the flexibility of object-level matching to unbalanced optimal transport, it does not compare OMT against simpler matching strategies, such as nearest-neighbor, Hungarian, IoU/radius-based matching, or soft distance-based matching. It is therefore unclear whether the performance gains are specifically due to the OT formulation or more generally due to object-level matching and additional supervision.
-
The contributions of individual OMT components are not fully disentangled. OMT combines teacher-location carrier supervision, ordering supervision, transport-based quality and geometry losses, and exclusion loss. The current ablations do not sufficiently isolate these components, making it difficult to determine which components are essential for the reported improvement. A more comprehensive component-wise or factorial ablation would strengthen the empirical justification of the method.
-
The role of teacher-score calibration is not sufficiently investigated. The method converts teacher scores into class-relative empirical percentiles and then squares them before using them as quality targets and transport masses. This is a non-trivial design choice that may substantially affect the results. It would be useful to compare this formulation with raw teacher scores and other calibration strategies to determine how much of the improvement comes from score calibration rather than the proposed transport mechanism.
-
The exclusion loss introduces a potentially important false-negative risk. The proposed transport-based exclusion mechanism suppresses native student responses that receive weak support from the teacher. However, teacher predictions are inevitably incomplete, so valid student detections that are missed by the teacher may also be suppressed. Although the paper acknowledges this issue, the experiments do not quantitatively characterize how often valid predictions are removed or how performance changes when the exclusion loss is removed or weakened.
-
The statistical robustness of the main results is limited. The formal experiments are based on a single training seed, while the reported bootstrap analysis only measures variation across validation scenes at fixed model weights and does not capture training randomness. Since the main improvement over HRCA is relatively modest (1.01 mAP and 0.39 NDS), experiments with multiple training seeds would provide stronger evidence that the gains are reproducible.
-
The scope of the generalization claim is not fully supported by the experiments. The paper presents OMT as a framework for heterogeneous 3D detection and emphasizes its architecture flexibility, but the experiments are conducted on nuScenes within a relatively specific LiDAR-based student and multimodal-teacher setting. More diverse datasets, modalities, or substantially different detector families would be needed to substantiate a broader claim of architecture-agnostic or general heterogeneous distillation.
Questions for authors
- Could the authors clarify how the teacher-relative quality score is computed in practice when the number of teacher predictions for a given class is small? In particular, how robust is the empirical percentile estimation in such cases?
- Could the authors clarify the motivation for detaching the optimal transport plan before computing the student gradients? It would be helpful to explain whether this choice is primarily for optimization stability or whether allowing gradients through the transport plan was empirically found to be less effective.
Thanks for authors' response
The authors’ rebuttal addressed some of concerns. I maintain my original score.