acceptodds
Under review as a conference paper at ICLR 2027

BEV3D: BEV-Based Event-Guided 3D Detection for LiDAR Blind Intervals

Abstract

Modern 3D object detectors for autonomous driving often operate at the rate of their slowest sensor, leaving 50–100,ms LiDAR blind intervals without updated detections. To update detections within these intervals, Ev-3DOD introduced event-based blind-time perception through an RoI-based formulation that updates individual detections. We instead introduce BEV3D, a framework named after the two EV's in B**EV** and **EV**ent, which reformulates blind-interval perception as event-guided BEV updating. BEV3D couples a frozen LiDAR reference detector with an event-driven branch that updates a sparse bird's-eye-view (BEV) representation and re-estimates detections throughout each LiDAR blind interval. Within this event-driven branch, Frustum Event Lifting (FEL) lifts event features with an inference-time LiDAR depth prior, while Support-Gated Temporal Fusion (SGTF) combines fresh event evidence with cached LiDAR features. On DSEC-3DOD and Ev-Waymo, BEV3D outperforms Ev-3DOD in every evaluated category by up to 8.6 and 11.3 AP, respectively, while relying only on LiDAR and events, without the RGB modality used by Ev-3DOD. Our code is included in the supplementary material and will be publicly released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D object detection, event cameras, LiDAR, bird's-eye view, autonomous driving

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AFtkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BEV^{2}3D a method that updates a sparse BEV representation based on event evidence for LiDAR blind-spot perception, followed by re-detection. By employing FEL and SGTF, BEV^{2}3D achieves significant performance gains on the DSEC-3DOD and Ev-Waymo datasets.

Critical strengths

(1) To address the 50–100ms perception blind spot inherent in 10–20Hz LiDAR systems, this paper proposes preserving and fusing event evidence within a shared Bird’s-Eye-View (BEV) space, rather than compressing such evidence into a motion value for each detected bounding box.

(2) On the DSEC-3DOD dataset, BEV^{2}3D achieves an improvement of 8.55 in Vehicle AP and 4.56 in Pedestrian AP over Ev-3DOD. On Ev-Waymo, it shows an improvement of approximately 9–11 AP across three categories—all while eliminating the reliance on RGB inputs required by Ev-3DOD.

Critical weaknesses

(1) The causal attribution for the claim that "BEV updates outperform RoI-based motion estimation" has not been rigorously isolated. The comparison between BEV^{2}3D and Ev-3DOD involves simultaneous changes to multiple variables. The former employs a frozen VoxelNeXt LiDAR detector and a center-based BEV head, whereas the latter utilizes LoGoNet’s LiDAR+RGB features combined with RoI-specific 4-DoF motion updates. Authors need to provide the comparison with the same detector.

(2) Table 1 and Table 2 pool the active timestamp at t=0 together with nine intermediate query timestamps, whereas BEV^{2}3D directly utilizes the output of a frozen reference detector at t=0. Consequently, the resulting primary AP metrics reflect both the performance of the reference detector and the performance of updates during blind intervals. This may either amplify or diminish the actual differences occurring specifically within the blind intervals. The paper should report separate AP/APH results for three distinct categories: active-only, blind-only (\tau > 0), and the pooled set.

(3) Appendix B merely specifies that the reference detector is a "frozen VoxelNeXt," but it fails to state the origin of the checkpoint, the training set used, or whether it strictly adheres to the train/validation split of the Ev-Waymo dataset. Given that Ev-Waymo is derived from the Waymo Open Dataset, appropriate measures must be taken when using any external or officially provided Waymo checkpoints to prevent potential data leakage (e.g., overlaps in sequences, scenes, or annotations).

(4) One advantage of BEV^{2}3D is that its detection head is no longer limited to updating existing bounding boxes from keyframes. Theoretically, it can rediscover objects from the full BEV representation. However, SGTF still relies on keyframe bounding boxes to construct a support mask, and the fusion process remains heavily dependent on cached LiDAR BEV features.

(5) The paper notes that LiDAR depth values ​​obtained at t=0 become invalid by time \tau due to target motion, and it employs an event-conditioned correction to adjust the depth distribution. However, the study fails to evaluate whether the corrected depth distribution is, in fact, closer to the target’s current position. Furthermore, it lacks an analysis of the failure boundaries of FEL in relation to factors such as target velocity, lateral displacement, the query timestamp, or event density. More critically, in autonomous driving scenarios, significant ego-motion occurs within a 100ms interval. However, the paper’s formulas utilize camera calibration to project event rays into a Bird’s-Eye View (BEV) representation, it is not explicitly stated whether the cached LiDAR BEV, depth priors, support masks, and intermediate-timestamp annotations are transformed into a unified coordinate system using the vehicle’s pose.

(6) This paper adopts VoxelNeXt as the 3D detector but there are many more advanced methods (e.g., LION [1], HEDNet [2], UniLION [3]). Authors should provide experiments with more advanced methods.

[1] LION: Linear Group RNN for 3D Object Detection in Point Clouds

[2] HEDNet: A Hierarchical Encoder-Decoder Network for 3D Object Detection in Point Clouds

[3] UniLION: Towards Unified Autonomous Driving Model with Linear Group RNNs

Questions for authors

[1] The authors should use stronger baselines to validate the effectiveness of the proposed method.

12 comments
AuthorsOpenReview ↗

Response to Reviewer AFtk — Part 1/2

Thank you for your careful review and suggestions on experimental comparisons and evaluation details.

Reference quality and event contribution (W1)

Reference quality and event contribution. Ev-3DOD uses LoGoNet's multi-scale features and RoI pooling; our model updates and re-detects in sparse BEV. Exchanging references requires adapting these interfaces and retraining, so a checkpoint swap cannot isolate the update formulation. We provide three complementary comparisons.

Revised Table 1 distinguishes the original large reference from standard VoxelNeXt:

Reference detectorDSEC VehicleDSEC PedestrianEv-Waymo VehicleEv-Waymo PedestrianEv-Waymo Cyclist
LoGoNet58.68 / 57.9546.51 / 30.9068.40 / 67.9073.04 / 62.7547.49 / 46.17
VoxelNeXt58.67 / 58.1041.68 / 28.8067.28 / 66.8274.24 / 65.4551.23 / 49.74
VoxelNeXt-large64.58 / 63.8749.95 / 35.2068.26 / 67.8273.80 / 65.1953.44 / 52.06

Entries are LEVEL_1 AP/APH (%) on original keyframe annotations, separately from the pooled protocol below.

Revised Tables 2–3 evaluate our standard-reference system:

Dataset / systemVehicle AP/APHPedestrian AP/APHCyclist AP/APH
DSEC: Ev-3DOD41.65 / 41.2020.68 / 11.88—
DSEC: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard46.03 / 45.4919.85 / 14.12—
Ev-Waymo: Ev-3DOD60.30 / 59.9557.40 / 50.7831.08 / 30.38
Ev-Waymo: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard72.39 / 71.8766.58 / 60.2638.95 / 37.83

These are LEVEL_1 AP/APH pooled over the active frame and nine queries; Ev-3DOD values are its published results. Standard VoxelNeXt matches LoGoNet's DSEC Vehicle reference AP (58.67 vs. 58.68), yet our system gains 4.38 AP. DSEC Pedestrian AP is 0.83 lower and APH 2.24 higher. On Ev-Waymo, the standard references differ by −1.12/+1.20-1.12/+1.20 Vehicle/Pedestrian AP, while our system gains 12.09/9.18 AP.

Revised Table 7 adds a trained no-event baseline with the same frozen reference cache, explicit query time, head architecture and intermediate-time supervision:

DSEC configurationVehicle AP/APHPedestrian AP/APH
No events + query-time conditioning40.12 / 39.6612.59 / 8.84
BEV23D\mathrm{BEV}^{2}\mathrm{3D}-large, full50.20 / 49.4625.24 / 18.15
Improvement+10.08 / +9.80+12.65 / +9.31

These results support an event contribution with the reference cache fixed. The spatial-access ablation also shows 4.77/2.21 Vehicle/Pedestrian AP from full BEV access, with the head fixed (original Table 3; revised Table 5).

Reference training and data splits (W3)

We train the Stage-1 VoxelNeXt reference detectors on the training split of each benchmark, following the same train/validation splits as Ev-3DOD (Appendix A). We do not use external Waymo-pretrained weights. The original system uses VoxelNeXt-large, and the revision additionally evaluates standard VoxelNeXt. These reference detectors are frozen throughout asynchronous training and inference. Revised Table 1 reports their standalone keyframe performance.

Detection beyond keyframe supports (W4)

SGTF assigns extra fusion weights within keyframe supports; it does not restrict output locations. Outside supports, the event coefficient remains 1, and event-only coordinates remain available (Sec. 3.3.3). Restricting access to expanded supports loses 4.77/2.21 Vehicle/Pedestrian AP (revised Table 5).

Revised Table 7 compares annotated and predicted supports during training:

Training-time supportsVehicle AP/APHPedestrian AP/APH
Annotated keyframe boxes (original full model)50.20 / 49.4625.24 / 18.15
Predicted keyframe boxes49.10 / 48.5226.14 / 18.69

Both use frozen-reference predictions at inference. Performance is retained when predicted supports are also used during training.

AuthorsOpenReview ↗

Response to Reviewer AFtk — Part 2/2

Active-only and blind-only evaluation (W2)

We now report active-only (τ=0\tau=0), blind-only (τ>0\tau>0), and pooled AP/APH separately for the standard-reference BEV23D\mathrm{BEV}^{2}\mathrm{3D} added in revised Tables 2–3. Both methods are evaluated using the same Stage-2 annotations (LEVEL_1 AP/APH, %).

Evaluation scopeMethodDSEC VehicleDSEC PedestrianEv-Waymo VehicleEv-Waymo PedestrianEv-Waymo Cyclist
Active-onlyBEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard58.50 / 57.9343.62 / 30.3373.70 / 73.2671.69 / 64.7036.32 / 35.42
Active-onlyEv-3DOD57.34 / 56.6742.77 / 23.4168.46 / 68.0668.58 / 60.3235.29 / 34.50
Blind-onlyBEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard44.56 / 44.0217.09 / 12.2673.22 / 72.6766.18 / 59.9341.72 / 40.50
Blind-onlyEv-3DOD41.31 / 40.8318.08 / 10.3659.40 / 59.0655.87 / 49.5430.49 / 29.82
PooledBEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard46.03 / 45.4919.85 / 14.1272.39 / 71.8766.58 / 60.2638.95 / 37.83
PooledEv-3DOD42.11 / 41.5920.49 / 11.1659.87 / 59.5256.15 / 49.7130.20 / 29.52

The active-only scores differ from Table 1 because Ev-3DOD's released Stage-1 keyframe annotations and Stage-2 interpolated annotations have different annotation coverage.

On Ev-Waymo, the active-only AP advantages of approximately 5.2/3.1/1.0 points increase to 13.8/10.3/11.2 points during the blind interval for Vehicle/Pedestrian/Cyclist. On DSEC, the Vehicle advantage increases from approximately 1.2 to 3.2 AP; Pedestrian blind-only AP is 0.99 points lower, while APH is 1.90 points higher. Thus, the larger blind-interval gains hold across all three Ev-Waymo classes and DSEC Vehicle.

Each scope is evaluated jointly, rather than by averaging timestamp AP. We re-evaluate Ev-3DOD under this protocol; revised Tables 2–3 retain its published benchmark results.

Depth correction and coordinate conventions (W5)

FEL's soft prior allows current event features to reweight depth hypotheses. Original Table 4 (revised Table 6) evaluates its effect on detection: compared with hard assignment, the soft prior gains 0.49/1.08 AP on DSEC and 0.63/1.60/0.50 AP on Ev-Waymo. These results support the benefit of the soft depth prior for detection. We acknowledge that detection metrics do not directly measure the corrected depth error, but the datasets we use do not provide depth ground truth at blind-time queries, preventing direct evaluation. Moreover, adjacent queries are only 10 ms apart: small position changes may be difficult to resolve at the available image resolution and depth discretization, yet may still affect the spatial alignment of event features and the final detection results. We therefore use the detection ablations to evaluate FEL's contribution.

Coordinate convention and ego-motion. We use the per-query 3D annotations released by Ev-3DOD. Predicted boxes follow the same LiDAR coordinate convention as these annotations, which we use directly for supervision and evaluation. The model uses the keyframe cache and events to predict boxes at the query time, without using ego poses or applying explicit cross-time pose transformations during inference. FEL uses the camera and LiDAR calibration parameters for projection. The cached BEV, keyframe depth prior, and support boxes are not transformed across time using ego poses.

Stronger reference detectors (W6 / Q1)

Integrating LION, HEDNet, or UniLION requires adapting feature representations and fusion interfaces, so these backbones are not included in this revision. We evaluate reference capacity through standard/large VoxelNeXt. On DSEC, Stage-1 Vehicle/Pedestrian AP rises from 58.67/41.68 to 64.58/49.95, and our pooled AP from 46.03/19.85 to 50.20/25.24 (revised Tables 1–2). Together with W1's standard-reference and no-event comparisons, this shows that stronger reference features and event-guided updates both contribute.

Thanks for the authors' response. I still have some concerns.

(1) The model uses the keyframe cache and events to predict boxes at the query time, without using ego poses or applying explicit cross-time pose transformations during inference. It will introduce coordinate misalignment during training and inference.

(2) What is the performance of VoxelNeXt-Large when the proposed method is not used? Authors should provide these results to demonstrate the effectiveness of proposed methods.

Reviewer_e2DjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BEV²3D, an event-guided 3D detection framework for LiDAR blind intervals. By introducing Frustum Event Lifting (FEL) and Support-Gated Temporal Fusion (SGTF), the method updates cached LiDAR BEV features using asynchronous event streams. The approach achieves substantial improvements over Ev-3DOD on DSEC-3DOD and Ev-Waymo.

Critical strengths
  1. The BEV-based reformulation of blind-time detection is well-motivated and provides a meaningful alternative to proposal-based motion estimation.

  2. The method demonstrates strong and consistent improvements across two benchmarks, supported by component ablations and temporal analysis.

  3. The framework avoids RGB inputs at inference and achieves competitive computational efficiency.

Critical weaknesses
  1. FEL largely builds upon existing depth-guided BEV lifting techniques, while SGTF employs a relatively simple support-based fusion strategy. The technical novelty beyond integrating established components into the proposed framework could be better justified.

  2. The main comparison with Ev-3DOD involves different reference detectors (VoxelNeXt vs. LoGoNet) and sensor configurations. Although their reference performance is comparable, these differences make it difficult to isolate the benefits of the proposed BEV formulation.

  3. The current ablations validate FEL and SGTF but do not establish how much improvement comes from newly observed event information versus re-detection using cached LiDAR BEV features. Additionally, the reported 11.8 ms latency does not fully satisfy the targeted 100 Hz update rate.

Questions for authors
  1. Can the authors provide a baseline that uses the same BEV detection architecture but excludes event inputs, to quantify the actual contribution of event-guided updates?

  2. Can the authors further clarify the performance gains attributable specifically to BEV-based reformulation, independently of differences in detector architecture and capacity?

2 comments
AuthorsOpenReview ↗

Response to Reviewer e2Dj — Part 1/1

Thank you for recognizing the motivation and results, and for your suggestions on clarifying our contributions.

Contribution of the BEV formulation (W1)

Our contribution is the formulation of blind-time perception as event-guided BEV updating followed by scene-level re-detection (Secs. 1 and 3). FEL makes cached LiDAR depth a soft prior that current event features can revise; SGTF changes the emphasis on keyframe supports over time while retaining evidence outside them. The spatial-access ablation in original Table 3 (revised Table 5) supports full-BEV detection, while new Table 7 evaluates the event-guided branch against a trained no-event predictor.

Reference differences and gain attribution (W2 / Q2)

Ev-3DOD couples LoGoNet's multi-scale features with RoI pooling and per-box motion prediction; our model updates sparse BEV and re-detects. Sharing a reference requires feature/interface adaptation and retraining. We therefore examine reference capacity, events and spatial access separately.

Revised Table 1 distinguishes the original large reference from standard VoxelNeXt:

Reference detectorDSEC VehicleDSEC PedestrianEv-Waymo VehicleEv-Waymo PedestrianEv-Waymo Cyclist
LoGoNet58.68 / 57.9546.51 / 30.9068.40 / 67.9073.04 / 62.7547.49 / 46.17
VoxelNeXt58.67 / 58.1041.68 / 28.8067.28 / 66.8274.24 / 65.4551.23 / 49.74
VoxelNeXt-large64.58 / 63.8749.95 / 35.2068.26 / 67.8273.80 / 65.1953.44 / 52.06

Entries are LEVEL_1 AP/APH (%) on original keyframe annotations, separately from pooled evaluation.

Revised Tables 2–3 add our standard-reference system:

Dataset / systemVehicle AP/APHPedestrian AP/APHCyclist AP/APH
DSEC: Ev-3DOD41.65 / 41.2020.68 / 11.88—
DSEC: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard46.03 / 45.4919.85 / 14.12—
Ev-Waymo: Ev-3DOD60.30 / 59.9557.40 / 50.7831.08 / 30.38
Ev-Waymo: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard72.39 / 71.8766.58 / 60.2638.95 / 37.83

These are LEVEL_1 AP/APH pooled over the active frame and nine queries; Ev-3DOD values are its published results. DSEC Vehicle reference AP is nearly identical (58.67 vs. 58.68), yet our standard system gains 4.38 AP. Pedestrian AP is 0.83 lower and APH 2.24 higher. Ev-Waymo standard references differ by −1.12/+1.20-1.12/+1.20 Vehicle/Pedestrian AP, while our system gains 12.09/9.18 AP. Ev-3DOD retains its RGB branch.

Full spatial access adds 4.77/2.21 DSEC Vehicle/Pedestrian AP over expanded-support access with the BEV head fixed (original Table 3; revised Table 5). The no-event comparison below tests event contribution with the reference cache fixed.

Event contribution and runtime (W3 / Q1)

Revised Table 7 adds a trained no-event baseline with the same frozen VoxelNeXt-large cache, head architecture and intermediate-time supervision, plus explicit query time:

DSEC configurationVehicle AP/APHPedestrian AP/APH
No events + query-time conditioning40.12 / 39.6612.59 / 8.84
BEV23D\mathrm{BEV}^{2}\mathrm{3D}-large, full50.20 / 49.4625.24 / 18.15
Improvement+10.08 / +9.80+12.65 / +9.31

Entries use the same DSEC pooled LEVEL_1 AP/APH (%) protocol. The full model gains 10.08/12.65 Vehicle/Pedestrian AP.

Revised Table 4 compares per-query component costs on the same RTX 4090 D at batch size 1:

ComponentOurs-large (ms)Ev-3DOD (ms)
Event encoder3.8154.209
FEL4.066—
SGTF + detection head + decoding/NMS3.843—
Point/voxel feature preparation—4.474
RoI pooling and fusion—4.857
Motion and score prediction—0.794
Box decoding—0.337
Output postprocessing—0.729
Component total11.72315.401
Updates/s from component total85.364.9

Timing starts with inputs and reference features on the GPU, excluding preprocessing, event voxelization, transfer and reference refresh. The 100 Hz grid specifies query timestamps. The current implementation achieves approximately 85 updates/s; although below 100 Hz, kernel fusion and reduced kernel-launch overhead could further improve the update rate.

Dear reviewer, thank you for your time. We hope our responses have addressed your concerns and would appreciate it if you could let us know whether any questions remain.

Reviewer_QttSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

BEV²3D addresses 3D detection during the blind interval between two LiDAR sweeps, using an event camera. A frozen LiDAR detector (VoxelNeXt) caches its BEV features and boxes at each keyframe; a trainable branch lifts the events accumulated since the keyframe into a sparse event BEV, using the projected keyframe LiDAR depth as a soft prior that the events can correct (FEL), and fuses it with the cached features through a gain that is higher inside expanded keyframe boxes and decays over the interval (SGTF). A center-based head then re-detects objects from the fused BEV at every query time, instead of propagating each keyframe box as Ev-3DOD does. Only the event branch is trained, against the 100 Hz blind-time annotations released by Ev-3DOD, and evaluation follows the Ev-3DOD protocol, which pools the keyframe and the nine blind-time queries before computing AP. On DSEC-3DOD (real events) and Ev-Waymo (synthetic events) the paper reports consistent gains over Ev-3DOD in every class (Tables 1–2), without an RGB branch and at similar per-update latency (Table 7).

Critical strengths
  1. The problem is defined precisely and the motivation is clear. The blind-interval setting is stated as a causal constraint: at query time τ the detector may use only the latest LiDAR keyframe and the events accumulated since it (Sec. 3.1), motivated by the 50–100 ms gap between sweeps at 10–20 Hz LiDAR, in which a vehicle at 30 m/s moves up to 3 m (Sec. 1). The alternative to the existing per-box propagation is equally clear: keep a spatial BEV and re-detect at every query (Fig. 1). This matters for my assessment because the paper poses a well-defined, practically relevant problem and a concrete, testable alternative formulation; the remaining questions are about evidence, not about what is being claimed.

  2. The reported gains are consistent under an established protocol, without an RGB branch. Under the Ev-3DOD protocol, BEV²3D is ahead of Ev-3DOD in every class on both benchmarks, by 4.6–11.3 AP (Tables 1–2), and at every blind-time query (DSEC overall AP, Fig. 5; each Ev-Waymo class, Fig. 6), without the RGB branch Ev-3DOD uses and at similar per-update latency (Table 7); on Ev-Waymo the margin survives every SGTF endpoint tested (Table 8). This matters because the size and consistency of the effect are what keep my recommendation at weak rather than clear rejection: if the attribution and variance questions below are answered, these results would support acceptance.

Critical weaknesses
  1. The paper does not show that its gains come from event-guided BEV updating.

    • The main comparison changes several things at once: reference detector (VoxelNeXt vs LoGoNet), detection head and, possibly, event backbone and training recipe (Secs. 3.2–3.4; p. 7 l. 372–377). The argument that the two reference detectors are comparable rests on pooled "online" rows that mix t = 0 with nine stale queries (Table 2; Sec. 4.1.2) and cites Ev-Waymo only; on DSEC the VoxelNeXt row is 1.92 Vehicle AP above LoGoNet and 0.61 Pedestrian AP below it (Table 1). These rows are taken from Ev-3DOD (Sec. 4.1.2), while the supplement trains a wider VoxelNeXt ("voxelnext-large", Stage 1 in the README) as BEV²3D's reference detector, whose own AP is not reported.
    • No experiment removes the events: every variant in Tables 3–5 and 8 keeps E_τ, so fresh event evidence cannot be separated from a BEV head re-trained on cached LiDAR features with dense 100 Hz labels. A LiDAR-only control would separate them: train the same asynchronous detector, head, loss and labels with the event branch removed and the query time τ given as an explicit input, in two forms — (i) static: re-detect from the cached F₀ alone; (ii) motion: from the same cached F₀, predict a τ-conditioned BEV displacement field, warp F₀ to τ, and re-detect (a flow-and-warp step similar to AsyncBEV's [AsyncBEV, ICLR2026]). Both controls see exactly what BEV²3D sees minus the events, so the gap between (ii) and BEV²3D indicates what blind-time events add beyond the motion already predictable from the cached LiDAR features (on DSEC the cached cloud is accumulated over sweeps, App. A, so it may already contain motion cues), and the gap between (ii) and (i) indicates how much of that comes from explicit extrapolation. A constant-velocity propagation of D₀, with velocities from matching the previous keyframe's cached boxes D₋ᵀ to D₀, is a cheaper, training-free reference point.
    • Table 3 is the closest control, but it masks LiDAR and event context together and leaves 6–7% of intermediate ground-truth centers outside the retained grid (coverage 92.91% / 94.05%), so it shows that full fused-BEV access helps but does not isolate event evidence outside the supports.
    • The per-query curves do not show a gain that grows with event accumulation: on Ev-Waymo the advantage is already ≈12 AP (Vehicle), ≈11 AP (Pedestrian) and ≈19 AP (Cyclist) at the first query (τ = 10 ms), and it does not grow over the interval (Fig. 6; values read from the plot). On DSEC, the mean center distance on objects matched by both methods at IoU ≥ 0.1 differs by only 0.005–0.045 m (Fig. 5); this does not decompose the AP gain, which uses stricter IoU thresholds. Both patterns are also consistent with a stronger detector or head at t ≈ 0.
    • Consequence: the stated contribution is the reformulation (Sec. 1), but the evidence is also consistent with a simpler explanation — a stronger detector or head at t ≈ 0 on cached LiDAR features. Tables 4–5 show that the FEL and SGTF settings matter within BEV²3D, but not how much of the gain over Ev-3DOD comes from blind-time events. The LiDAR-only controls above and the t = 0 AP of both reference detectors would address this.
  2. The only real-event result rests on one seed and is unusually sensitive to one hand-set gain.

    • All BEV²3D results are reported for a single seed (666, Table 6) with no variance estimate, and the paper does not say on which data α_start/α_end, r and σ were chosen (App. A lists only the train/validation splits).
    • On DSEC, changing only the SGTF endpoint moves Vehicle AP from 38.51 to 50.20 (Table 8), more than the 8.55-AP headline gain (Table 1); the same sweep moves Ev-Waymo Vehicle AP by 1.60. Endpoints 0.0–0.4 all give 49.22–50.20 Vehicle AP, and 38.51 is the non-decaying setting (α_end = α_start = 1.0).
    • A constant in-support gain with the same mean as the schedule scores 4.71 AP below uniform fusion with no extra in-support gain (38.96 vs 43.67, Table 5), which the paper does not explain; without SGTF the margin over Ev-3DOD is +2.02 Vehicle and −1.00 Pedestrian AP (Table 5 vs Table 1).
    • Consequence: the run-to-run variation of the DSEC margin is unknown, and without the selection protocol the SGTF effect may partly reflect tuning on the reported split rather than the mechanism described in Sec. 3.3.3. The selection protocol and three seeds for the full model and uniform fusion would address this.
Questions for authors

Questions whose answers could change my assessment:

  1. Attribution of the gain (W1). Because no reported variant removes the events and the two systems differ in several components, could you report (a) the AP of the keyframe detections alone (t = 0, no blind-time queries), on both datasets, of LoGoNet and of the Stage-1 VoxelNeXt checkpoint actually used by BEV²3D, and (b) the two LiDAR-only controls described under W1 — the static one and the motion one with a τ-conditioned displacement field predicted from the cached F₀ — under the same protocol? If (b) is not feasible in the discussion period, the constant-velocity propagation of D₀ would already be informative.

  2. Variance and hyperparameter selection (W2). Given the single seed (Table 6) and the sensitivity in Tables 5 and 8, could you report mean ± std over at least three seeds for the full model and the uniform-fusion variant (Table 5) on DSEC, and state how α_start/α_end, r and σ were chosen and on which split? It would also help to know whether uniform fusion plus an explicit query-time input to the head recovers the effect of the decaying schedule, since that would show whether SGTF acts mainly as a time cue.

  3. Where the gain appears. Since localization on commonly detected objects differs by only a few centimetres (Fig. 5) and the advantage is already full-size at the first query (Fig. 6), could you, using existing predictions, split blind-time AP and recall by whether each ground-truth object was detected by the respective reference detector at t = 0, and report false positives per frame for both methods?

Clarification requests needed to verify the reported numbers and claims:

  1. Causal inputs. Appendix A says the DSEC reference input is "accumulated LiDAR sweeps augmented with disparity-derived 3D points", while Eq. 1 and Sec. 3.3.2 use the active-frame cloud P₀. Which sweeps and disparity products are used, over what time window, and what is the latest raw-observation timestamp each depends on? Is the same cloud used for both the reference detector and the FEL depth prior, as the shared symbol P₀ suggests? For Ev-Waymo, does the Vid2E synthesis use the frame at t = T (interpolation between consecutive frames), so that blind-interval events depend on the next keyframe? These determine how the claims "causal" (p. 7 l. 369–371) and "without RGB" (abstract) should be read; they apply to Ev-3DOD as well, so they bear mainly on these two claims rather than on the AP comparison itself.

  2. Frames, timestamps and the "overall" metric. In which coordinate frame are the τ targets and predictions expressed, and is ego-pose used anywhere at inference? Please confirm that query index k in Figs. 5–6 corresponds to τ = 10k ms, as in Figs. 7–8. How is "overall" AP in Fig. 5 computed (class mean?), and why are DSEC per-class per-timestamp curves not shown?

  3. Latency scope. For Table 7, which steps are inside the 11.8 ms (event voxelization, depth projection, the amortized reference detector, cache refresh), and was Ev-3DOD re-timed on the same hardware?

6 comments
AuthorsOpenReview ↗

Response to Reviewer QttS — Part 1/3

Thank you for your detailed review and suggestions on identifying the sources of improvement and assessing stability.

Reference capacity and standalone accuracy (W1.1 / Q1(a))

Revised Table 1 reports standalone keyframe AP/APH for LoGoNet and both VoxelNeXt references, distinguishing the large reference used in our original results:

Reference detectorDSEC VehicleDSEC PedestrianEv-Waymo VehicleEv-Waymo PedestrianEv-Waymo Cyclist
LoGoNet58.68 / 57.9546.51 / 30.9068.40 / 67.9073.04 / 62.7547.49 / 46.17
VoxelNeXt58.67 / 58.1041.68 / 28.8067.28 / 66.8274.24 / 65.4551.23 / 49.74
VoxelNeXt-large64.58 / 63.8749.95 / 35.2068.26 / 67.8273.80 / 65.1953.44 / 52.06

These LEVEL_1 AP/APH (%) use original keyframe annotations, separately from the interpolated-annotation protocol for temporal and pooled results.

Revised Tables 2–3 test whether gains require the large reference:

Dataset / systemVehicle AP/APHPedestrian AP/APHCyclist AP/APH
DSEC: Ev-3DOD41.65 / 41.2020.68 / 11.88—
DSEC: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard46.03 / 45.4919.85 / 14.12—
Ev-Waymo: Ev-3DOD60.30 / 59.9557.40 / 50.7831.08 / 30.38
Ev-Waymo: BEV23D\mathrm{BEV}^{2}\mathrm{3D}, standard72.39 / 71.8766.58 / 60.2638.95 / 37.83

These are pooled LEVEL_1 AP/APH over the active frame and nine queries; Ev-3DOD values are its published results. With nearly identical DSEC Vehicle reference AP (58.67 vs. 58.68), our standard-reference system gains 4.38 AP. DSEC Pedestrian AP is 0.83 lower and APH 2.24 higher. On Ev-Waymo, standard references differ by −1.12/+1.20-1.12/+1.20 Vehicle/Pedestrian AP, while our system gains 12.09/9.18 AP.

Ev-3DOD uses multi-scale point/voxel features and RoI pooling, whereas our model uses sparse BEV features. Exchanging references requires interface adaptation and retraining. The standalone evaluations and standard-reference results address reference quality; the no-event experiment below tests event contribution.

Event contribution beyond cached features (W1.2 / W1.5 / Q1(b))

Revised Table 7 adds a trained no-event, query-time-conditioned baseline on DSEC. It retains the full model's frozen VoxelNeXt-large reference and cached features, including the accumulated-cloud input. A residual MLP conditions the cache on query time; the MLP and head use the same intermediate-time labels, detection loss and training budget, without events.

ConfigurationVehicle AP/APHPedestrian AP/APH
No events + query-time conditioning40.12 / 39.6612.59 / 8.84
BEV23D\mathrm{BEV}^{2}\mathrm{3D}, full50.20 / 49.4625.24 / 18.15
Improvement+10.08 / +9.80+12.65 / +9.31

Entries are LEVEL_1 AP/APH (%) pooled over the active frame and nine queries. The 10.08/12.65 Vehicle/Pedestrian AP gains support the event-guided branch beyond cached features, explicit time and dense supervision.

The suggested AsyncBEV-style flow-and-warp control requires adding and training a displacement predictor and feature-warping module. It introduces modeling choices beyond removing events, so this revision prioritizes the cache-plus-time baseline; the flow-and-warp model has not been implemented.

What the spatial-access ablation measures (W1.3)

Original Table 3 (revised Table 5) evaluates full fused-BEV access, with both modalities subject to the spatial restriction. The trained no-event comparison in revised Table 7 separately tests the event-guided branch, as detailed in W1.2 above.

AuthorsOpenReview ↗

Response to Reviewer QttS — Part 2/3

Seed stability and parameter selection (W2.1 / W2.4 / Q2)

Appendix F, Table 10 reports three seeds for the full model and uniform fusion on DSEC. Full achieves higher AP/APH for both classes in every same-seed comparison, with mean Vehicle/Pedestrian AP gains of 6.65/6.39 points:

SeedConfigurationVehicle AP/APHPedestrian AP/APH
666Full50.20 / 49.4625.24 / 18.15
666Uniform43.67 / 43.1619.68 / 14.30
777Full48.72 / 48.0024.81 / 17.67
777Uniform43.25 / 42.7117.60 / 13.02
888Full48.72 / 48.0325.00 / 18.08
888Uniform40.75 / 40.3018.62 / 13.41
Mean ±\pm stdFull49.21±0.85/48.50±0.8349.21 \pm 0.85 / 48.50 \pm 0.8325.02±0.22/17.97±0.2625.02 \pm 0.22 / 17.97 \pm 0.26
Mean ±\pm stdUniform42.56±1.58/42.06±1.5442.56 \pm 1.58 / 42.06 \pm 1.5418.63±1.04/13.58±0.6618.63 \pm 1.04 / 13.58 \pm 0.66

Summary rows give mean ±\pm sample standard deviation.

Parameter selection. We fix αstart=1\alpha_{\mathrm{start}}=1, set r=1.25r=1.25 empirically to accommodate motion and nearby context, and set σ\sigma to one depth-bin width per dataset. The shared αend=0.2\alpha_{\mathrm{end}}=0.2 is selected by the endpoint sweep using validation AP on both datasets (Appendix E, Table 9).

SGTF uses query time to adjust fusion weights in keyframe support regions. Explicit time input to the head requires time encoding and feature-fusion design, introducing additional design variables; that variant is not included in this revision. Revised Table 7 compares uniform fusion, constant support gain and decaying gain to evaluate time-varying fusion weights.

Sensitivity and the role of gain decay (W2.2 / W2.3)

The large DSEC sensitivity comes from maintaining high support gain throughout the interval: αend=αstart=1\alpha_{\mathrm{end}}=\alpha_{\mathrm{start}}=1 yields 38.51 Vehicle AP. In contrast, endpoints 0.0/0.2/0.4 yield 50.12/50.20/49.22 AP, all above Ev-3DOD (Appendix E, Table 9). The shared endpoint 0.2 also gives the highest AP in all three Ev-Waymo classes in this sweep.

Equal mean gain does not imply equivalent fusion because event features change with time. Uniform fusion has event coefficient 1; constant support gain uses 1.6 inside keyframe supports, while SGTF decreases it from 2.0 to 1.2. Vehicle AP is 43.67, 38.96 and 50.20, respectively (revised Table 7). This supports reducing emphasis on aging supports, rather than simply increasing gain magnitude.

Temporal gains, localization and detection (W1.4)

Temporal trend. Fig. 6 compares two event-based systems, so their AP gap need not grow monotonically as both accumulate events. Their Ev-Waymo standalone Vehicle/Pedestrian reference AP is comparable (LoGoNet 68.40/73.04, VoxelNeXt-large 68.26/73.80; revised Table 1), while our system leads from the first query through the interval. W1.2 directly tests event contribution on DSEC.

Localization and detection. Fig. 5 includes only objects matched by both methods at IoU≥0.1\mathrm{IoU} \geq 0.1, excluding misses and unmatched predictions. Appendix D, Fig. 7 uses detection IoU thresholds of 0.7/0.5/0.5 for Vehicle/Pedestrian/Cyclist. Across all three Ev-Waymo classes, our method has higher precision over most shared recall values and fewer false positives at comparable recall.

For example, maximum recall among evaluated thresholds achieving at least 70% precision is:

Ev-Waymo classEv-3DOD recall (%)BEV23D\mathrm{BEV}^{2}\mathrm{3D}-large recall (%)
Vehicle54.2875.37
Pedestrian55.8165.77
Cyclist10.5543.21

Higher recall at this precision requirement shows gains beyond localization on common matches.

AuthorsOpenReview ↗

Response to Reviewer QttS — Part 3/3

Grouped detection gains and false positives (Q3)

We group blind-time targets by whether their respective reference detector detected them at t=0t=0, using score ≥0.3\geq 0.3 and Vehicle/Pedestrian/Cyclist IoU thresholds of 0.7/0.5/0.5. The table gives diagnostic AP101 / recall at score ≥0.3\geq 0.3 (%), and GT instance count NN:

Dataset / classMethodDetected at t=0t = 0: AP101 / recall (NN)Missed at t=0t = 0: AP101 / recall (NN)
DSEC / VehicleOurs-large62.74 / 82.85 (12,365)0.23 / 9.16 (5,209)
DSEC / VehicleEv-3DOD59.66 / 80.14 (10,949)0.47 / 9.71 (6,625)
DSEC / PedestrianOurs-large25.36 / 63.08 (1,953)0.70 / 17.86 (1,008)
DSEC / PedestrianEv-3DOD27.27 / 57.06 (1,458)0.54 / 8.98 (1,503)
Ev-Waymo / VehicleOurs-large84.26 / 92.93 (111,451)0.53 / 11.99 (20,327)
Ev-Waymo / VehicleEv-3DOD69.04 / 91.28 (111,670)0.05 / 9.54 (20,108)
Ev-Waymo / PedestrianOurs-large75.93 / 87.09 (103,502)0.36 / 13.92 (18,528)
Ev-Waymo / PedestrianEv-3DOD67.91 / 88.32 (96,095)0.56 / 19.43 (25,935)
Ev-Waymo / CyclistOurs-large75.23 / 89.45 (3,308)0.64 / 6.95 (2,445)
Ev-Waymo / CyclistEv-3DOD56.44 / 91.33 (3,023)0.02 / 2.34 (2,730)

AP101 uses fixed LEVEL_1 groups and 101 score thresholds. Matches to out-of-group targets are ignored; unmatched background predictions count as false positives. It is separate from official Waymo AP, and group APs are not additive.

Our method has higher AP101 in four of five detected groups; relative performance on missed targets varies by class. Groups are defined by each reference detector, so their target sets differ. Appendix D, Fig. 7 shows higher precision over most shared recall values and fewer false positives at comparable recall for all three Ev-Waymo classes. Fig. 5's center error on common matches excludes misses and false positives, so similar center distances are consistent with overall detection gains.

Dataset construction and detector inputs (Q4)

We use the datasets and preprocessing released by Ev-3DOD. DSEC uses recorded real-world events, while Ev-Waymo is a synthetic-event dataset whose images are used for offline event synthesis and are not fed into the detector. “Without RGB” means that the detector receives no RGB images and has no RGB feature branch. On DSEC, the reference detector and the FEL depth prior use the same point cloud.

Coordinates, query times and per-class results (Q5)

Query index kk corresponds to τ=10k ms\tau = 10k\,\mathrm{ms}, k=1,…,9k = 1,\ldots,9. “Overall” in Fig. 5 is the unweighted mean of Vehicle and Pedestrian LEVEL_1 AP at each query. DSEC per-class values are:

Query time (ms)Ev-3DOD VehicleOurs VehicleEv-3DOD PedestrianOurs Pedestrian
1048.3061.5038.3837.11
2053.1659.4037.3139.63
3053.6356.4231.4835.08
4051.7053.1723.5827.17
5047.2549.2415.6918.88
6041.4445.8312.2213.36
7033.2540.888.029.91
8027.3336.504.646.01
9020.5131.413.114.32

Coordinate convention and ego-motion. We use the per-query 3D annotations released by Ev-3DOD. Predicted boxes follow the same LiDAR coordinate convention as these annotations, which we use directly for supervision and evaluation. The model uses the keyframe cache and events to predict boxes at the query time, without using ego poses or applying explicit cross-time pose transformations during inference.

Latency scope and component costs (Q6)

The original 11.8 ms measures an event-time update after inputs and cached reference features are available on the GPU. It includes event encoding, FEL depth-prior projection/construction, fusion, detection, decoding and NMS. Event voxelization, input preprocessing/transfer, the reference detector, and cache refresh are excluded; reference cost is not amortized into this number.

Revised Table 4 compares per-query component costs on the same RTX 4090 D at batch size 1:

ComponentOurs-large (ms)Ev-3DOD (ms)
Event encoder3.8154.209
FEL4.066—
SGTF + detection head + decoding/NMS3.843—
Point/voxel feature preparation—4.474
RoI pooling and fusion—4.857
Motion and score prediction—0.794
Box decoding—0.337
Output postprocessing—0.729
Component total11.72315.401
Updates/s from component total85.364.9

These totals sum component means. The current implementation achieves approximately 85 updates/s. While this is below 100 Hz, future implementation optimizations, such as kernel fusion and reducing kernel-launch overhead, could further improve the update rate.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for the Area Chair

The reviewers recognized the motivation for event-guided BEV updating and scene-level re-detection, the consistent gains on DSEC-3DOD and Ev-Waymo, and the absence of an RGB input branch. Their main concerns centered on attributing these gains, establishing stability across runs, and clarifying evaluation and implementation details. We address these concerns with the following additions:

  • Separating event contribution from reference capacity. We added standalone reference-detector evaluations and standard/large VoxelNeXt comparisons (revised Tables 1–3). Gains also occur with the standard reference. A trained no-event baseline retains the same frozen reference cache, explicit query time, and intermediate-time supervision; the full model improves DSEC Vehicle/Pedestrian AP by 10.08/12.65 points (Table 7), directly supporting the contribution of the event-guided branch.

  • Checking stability and explaining parameter selection. We report three seeds for both the full model and uniform fusion, with mean ±\pm sample standard deviation (Appendix F, Table 10). The full model achieves higher AP/APH for both classes in every same-seed comparison. We also explain the shared hyperparameter settings and their selection, including the validation-based endpoint sweep.

  • Showing where the gains occur and clarifying the protocol. We provide separate active-only, blind-only, and pooled results, together with grouped detection analysis. Added Ev-Waymo PR and false-positive curves show higher precision over most of the shared recall range and fewer false positives at comparable recall (Appendix D, Fig. 7). We clarify reference training, dataset inputs, coordinate conventions, and timing scope, and provide component latency comparisons on the same RTX 4090 D (Table 4).

Thank you for your time and consideration of our responses and revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.