- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces BEV^{2}3D a method that updates a sparse BEV representation based on event evidence for LiDAR blind-spot perception, followed by re-detection. By employing FEL and SGTF, BEV^{2}3D achieves significant performance gains on the DSEC-3DOD and Ev-Waymo datasets.
Critical strengths
(1) To address the 50–100ms perception blind spot inherent in 10–20Hz LiDAR systems, this paper proposes preserving and fusing event evidence within a shared Bird’s-Eye-View (BEV) space, rather than compressing such evidence into a motion value for each detected bounding box.
(2) On the DSEC-3DOD dataset, BEV^{2}3D achieves an improvement of 8.55 in Vehicle AP and 4.56 in Pedestrian AP over Ev-3DOD. On Ev-Waymo, it shows an improvement of approximately 9–11 AP across three categories—all while eliminating the reliance on RGB inputs required by Ev-3DOD.
Critical weaknesses
(1) The causal attribution for the claim that "BEV updates outperform RoI-based motion estimation" has not been rigorously isolated. The comparison between BEV^{2}3D and Ev-3DOD involves simultaneous changes to multiple variables. The former employs a frozen VoxelNeXt LiDAR detector and a center-based BEV head, whereas the latter utilizes LoGoNet’s LiDAR+RGB features combined with RoI-specific 4-DoF motion updates. Authors need to provide the comparison with the same detector.
(2) Table 1 and Table 2 pool the active timestamp at t=0 together with nine intermediate query timestamps, whereas BEV^{2}3D directly utilizes the output of a frozen reference detector at t=0. Consequently, the resulting primary AP metrics reflect both the performance of the reference detector and the performance of updates during blind intervals. This may either amplify or diminish the actual differences occurring specifically within the blind intervals. The paper should report separate AP/APH results for three distinct categories: active-only, blind-only (\tau > 0), and the pooled set.
(3) Appendix B merely specifies that the reference detector is a "frozen VoxelNeXt," but it fails to state the origin of the checkpoint, the training set used, or whether it strictly adheres to the train/validation split of the Ev-Waymo dataset. Given that Ev-Waymo is derived from the Waymo Open Dataset, appropriate measures must be taken when using any external or officially provided Waymo checkpoints to prevent potential data leakage (e.g., overlaps in sequences, scenes, or annotations).
(4) One advantage of BEV^{2}3D is that its detection head is no longer limited to updating existing bounding boxes from keyframes. Theoretically, it can rediscover objects from the full BEV representation. However, SGTF still relies on keyframe bounding boxes to construct a support mask, and the fusion process remains heavily dependent on cached LiDAR BEV features.
(5) The paper notes that LiDAR depth values obtained at t=0 become invalid by time \tau due to target motion, and it employs an event-conditioned correction to adjust the depth distribution. However, the study fails to evaluate whether the corrected depth distribution is, in fact, closer to the target’s current position. Furthermore, it lacks an analysis of the failure boundaries of FEL in relation to factors such as target velocity, lateral displacement, the query timestamp, or event density. More critically, in autonomous driving scenarios, significant ego-motion occurs within a 100ms interval. However, the paper’s formulas utilize camera calibration to project event rays into a Bird’s-Eye View (BEV) representation, it is not explicitly stated whether the cached LiDAR BEV, depth priors, support masks, and intermediate-timestamp annotations are transformed into a unified coordinate system using the vehicle’s pose.
(6) This paper adopts VoxelNeXt as the 3D detector but there are many more advanced methods (e.g., LION [1], HEDNet [2], UniLION [3]). Authors should provide experiments with more advanced methods.
[1] LION: Linear Group RNN for 3D Object Detection in Point Clouds
[2] HEDNet: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds
[3] UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs
Questions for authors
[1] The authors should use stronger baselines to validate the effectiveness of the proposed method.
12 comments
Response to Reviewer AFtk — Part 2/2
Active-only and blind-only evaluation (W2)
We now report active-only (), blind-only (), and pooled AP/APH separately for the standard-reference added in revised Tables 2–3. Both methods are evaluated using the same Stage-2 annotations (LEVEL_1 AP/APH, %).
| Evaluation scope | Method | DSEC Vehicle | DSEC Pedestrian | Ev-Waymo Vehicle | Ev-Waymo Pedestrian | Ev-Waymo Cyclist |
|---|---|---|---|---|---|---|
| Active-only | , standard | 58.50 / 57.93 | 43.62 / 30.33 | 73.70 / 73.26 | 71.69 / 64.70 | 36.32 / 35.42 |
| Active-only | Ev-3DOD | 57.34 / 56.67 | 42.77 / 23.41 | 68.46 / 68.06 | 68.58 / 60.32 | 35.29 / 34.50 |
| Blind-only | , standard | 44.56 / 44.02 | 17.09 / 12.26 | 73.22 / 72.67 | 66.18 / 59.93 | 41.72 / 40.50 |
| Blind-only | Ev-3DOD | 41.31 / 40.83 | 18.08 / 10.36 | 59.40 / 59.06 | 55.87 / 49.54 | 30.49 / 29.82 |
| Pooled | , standard | 46.03 / 45.49 | 19.85 / 14.12 | 72.39 / 71.87 | 66.58 / 60.26 | 38.95 / 37.83 |
| Pooled | Ev-3DOD | 42.11 / 41.59 | 20.49 / 11.16 | 59.87 / 59.52 | 56.15 / 49.71 | 30.20 / 29.52 |
The active-only scores differ from Table 1 because Ev-3DOD's released Stage-1 keyframe annotations and Stage-2 interpolated annotations have different annotation coverage.
On Ev-Waymo, the active-only AP advantages of approximately 5.2/3.1/1.0 points increase to 13.8/10.3/11.2 points during the blind interval for Vehicle/Pedestrian/Cyclist. On DSEC, the Vehicle advantage increases from approximately 1.2 to 3.2 AP; Pedestrian blind-only AP is 0.99 points lower, while APH is 1.90 points higher. Thus, the larger blind-interval gains hold across all three Ev-Waymo classes and DSEC Vehicle.
Each scope is evaluated jointly, rather than by averaging timestamp AP. We re-evaluate Ev-3DOD under this protocol; revised Tables 2–3 retain its published benchmark results.
Depth correction and coordinate conventions (W5)
FEL's soft prior allows current event features to reweight depth hypotheses. Original Table 4 (revised Table 6) evaluates its effect on detection: compared with hard assignment, the soft prior gains 0.49/1.08 AP on DSEC and 0.63/1.60/0.50 AP on Ev-Waymo. These results support the benefit of the soft depth prior for detection. We acknowledge that detection metrics do not directly measure the corrected depth error, but the datasets we use do not provide depth ground truth at blind-time queries, preventing direct evaluation. Moreover, adjacent queries are only 10 ms apart: small position changes may be difficult to resolve at the available image resolution and depth discretization, yet may still affect the spatial alignment of event features and the final detection results. We therefore use the detection ablations to evaluate FEL's contribution.
Coordinate convention and ego-motion. We use the per-query 3D annotations released by Ev-3DOD. Predicted boxes follow the same LiDAR coordinate convention as these annotations, which we use directly for supervision and evaluation. The model uses the keyframe cache and events to predict boxes at the query time, without using ego poses or applying explicit cross-time pose transformations during inference. FEL uses the camera and LiDAR calibration parameters for projection. The cached BEV, keyframe depth prior, and support boxes are not transformed across time using ego poses.
Stronger reference detectors (W6 / Q1)
Integrating LION, HEDNet, or UniLION requires adapting feature representations and fusion interfaces, so these backbones are not included in this revision. We evaluate reference capacity through standard/large VoxelNeXt. On DSEC, Stage-1 Vehicle/Pedestrian AP rises from 58.67/41.68 to 64.58/49.95, and our pooled AP from 46.03/19.85 to 50.20/25.24 (revised Tables 1–2). Together with W1's standard-reference and no-event comparisons, this shows that stronger reference features and event-guided updates both contribute.
Thanks for the authors' response. I still have some concerns.
(1) The model uses the keyframe cache and events to predict boxes at the query time, without using ego poses or applying explicit cross-time pose transformations during inference. It will introduce coordinate misalignment during training and inference.
(2) What is the performance of VoxelNeXt-Large when the proposed method is not used? Authors should provide these results to demonstrate the effectiveness of proposed methods.
Response to Reviewer AFtk — Part 1/2
Thank you for your careful review and suggestions on experimental comparisons and evaluation details.
Reference quality and event contribution (W1)
Reference quality and event contribution. Ev-3DOD uses LoGoNet's multi-scale features and RoI pooling; our model updates and re-detects in sparse BEV. Exchanging references requires adapting these interfaces and retraining, so a checkpoint swap cannot isolate the update formulation. We provide three complementary comparisons.
Revised Table 1 distinguishes the original large reference from standard VoxelNeXt:
Entries are LEVEL_1 AP/APH (%) on original keyframe annotations, separately from the pooled protocol below.
Revised Tables 2–3 evaluate our standard-reference system:
These are LEVEL_1 AP/APH pooled over the active frame and nine queries; Ev-3DOD values are its published results. Standard VoxelNeXt matches LoGoNet's DSEC Vehicle reference AP (58.67 vs. 58.68), yet our system gains 4.38 AP. DSEC Pedestrian AP is 0.83 lower and APH 2.24 higher. On Ev-Waymo, the standard references differ by −1.12/+1.20 Vehicle/Pedestrian AP, while our system gains 12.09/9.18 AP.
Revised Table 7 adds a trained no-event baseline with the same frozen reference cache, explicit query time, head architecture and intermediate-time supervision:
These results support an event contribution with the reference cache fixed. The spatial-access ablation also shows 4.77/2.21 Vehicle/Pedestrian AP from full BEV access, with the head fixed (original Table 3; revised Table 5).
Reference training and data splits (W3)
We train the Stage-1 VoxelNeXt reference detectors on the training split of each benchmark, following the same train/validation splits as Ev-3DOD (Appendix A). We do not use external Waymo-pretrained weights. The original system uses VoxelNeXt-large, and the revision additionally evaluates standard VoxelNeXt. These reference detectors are frozen throughout asynchronous training and inference. Revised Table 1 reports their standalone keyframe performance.
Detection beyond keyframe supports (W4)
SGTF assigns extra fusion weights within keyframe supports; it does not restrict output locations. Outside supports, the event coefficient remains 1, and event-only coordinates remain available (Sec. 3.3.3). Restricting access to expanded supports loses 4.77/2.21 Vehicle/Pedestrian AP (revised Table 5).
Revised Table 7 compares annotated and predicted supports during training:
Both use frozen-reference predictions at inference. Performance is retained when predicted supports are also used during training.