acceptodds
Withdrawn from ICLR 2027

EventGeoPose: Pose-Referenced Event Encoding and Event-Geometry Consistency for Spiking 6D Pose Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Event cameras have shown considerable potential for 6D object pose estimation under fast motion and challenging illumination. However, current event-based approaches still rely heavily on dense ANN computation, fixed-duration temporal discretization, and conventional pose supervision, which respectively limit energy-efficient event processing, representation of highly nonuniform event dynamics, and explicit exploitation of event-derived geometric evidence. To address these issues, we present EventGeoPose, a directly trained SNN framework for 6D object pose estimation from event streams. We construct the spiking backbone with I-LIF neurons to improve the numerical fidelity of continuous regression while retaining spike-driven inference. We propose Pose-Referenced Adaptive Event Encoding, which combines activity-adaptive temporal discretization with duration-normalized local event-rate encoding to mitigate motion-state mixing and uneven observation density. We further introduce Event-Geometry Consistency Regularization, which couples predicted 3D object geometry with observed event structures through differentiable image-space reprojection, providing observation-aware geometric supervision for pose prediction. Additionally, we construct two complementary event-based datasets, EVA-6D-Syn and EVA-6D-Real, covering diverse motion, illumination, and real-sensor conditions. Extensive experiments demonstrate consistent improvements over representative methods while maintaining low theoretical inference energy, highlighting the potential of event-driven SNNs for robust and energy-efficient 6D pose estimation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
6D pose estimation, event camera, spiking neural network, event representation, event-geometry consistency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jXpKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-target detection and 6D pose estimation based on event cameras, and proposes a spiking neural network framework EventGeoPose for direct training. The model uses existing I-LIF neurons to improve the numerical expression of continuous attitude regression through integer value training, and expands integer activation into binary spike during inference. In addition, the author constructed two datasets, EVA-6D-Syn and EVA-6D-Real. The experiment covers the above dataset and publicly available YCB-Ev SD, reporting pose accuracy, component ablation, time step sensitivity, and theoretical inference energy consumption. The complete model exceeded the listed external baselines on all three datasets, but the same structured ANN variant had higher accuracy on synthetic data.

Critical strengths

1. Performance improvement covers both publicly available synthetic benchmarks and real data. The complete model achieved 71.2%, 34.7%, and 56.8% on EVA-6D-Syn, EVA-6D-Real, and YCB-Ev SD, respectively, surpassing the strongest listed external baselines by 11.8, 12.5, and 20.7 percentage points. The use of Ours-TS with the same Time Surface input also has advantages, indicating that the results are not entirely derived from differences in input representation.

2. Component ablation provides clear internal evidence. In Table 4, the basic SNN is 62.5%, and after adding PRAEE and EGCR separately, it reaches 65.7% and 66.1%, respectively. When used in combination, it reaches 71.2%. Table 5 further shows that after removing the event rate channel, it decreased to 67.6%, indicating the importance of supporting duration information for adaptive binning. The author also reported that ANN variants have higher accuracy, and increasing the time step does not always improve the results, which helps to understand the trade-offs of the method.

Critical weaknesses

W1. Table 6 reports that the theoretical energy consumption of I-LIF is 135.5 mJ and ANN is 866.7 mJ, but the main text and appendix do not clearly provide the operation count, single operation energy consumption, activation statistics, and calculation boundaries. Especially according to equation (6), integer activation needs to be expanded into multiple binary pulses, and the number of synaptic accumulations should reflect the activation amplitude, rather than relying solely on non-zero activation ratios. Due to the accuracy of ANN being 76.6%, which is higher than the 71.2% of I-LIF, the energy efficiency evidence directly determines the main value of using SNN.

W2. All major results are based on ADD(-S), which follows the setting of YCB-Ev SD. However, a single loose threshold is difficult to determine whether the improvement continues to meet stricter pose requirements. Appendix B.3 allocates the first 60% of each real sequence for training and the last 40% for testing. This reduces random interleaving while still sharing the collection environment, objects, and motion sequences. Therefore, 34.7% of the results support evaluation within the same collection sequence, which does not fully demonstrate generalization to independent recording sequences.

Questions for authors

Q1. Please provide the energy consumption formulas, MAC/AC operation quantities, single operation energy consumption assumptions, and activation statistics for ANN, LIF, and I-LIF. Does I-LIF count the number of binary spike after unfolding based on integer activation amplitude and include them in all time steps? What calculations are included in event encoding, neuron update and detection, and pose head?

Q2. Do the event windows on both sides of the training and testing boundary overlap, and are time intervals set? Can existing recording data be used to supplement independent sequences for evaluation and distinguish between normal and low light conditions?

Q3. The event distance field use the entire scene event or restrict the target instance? How to avoid the projection edge being attracted to event structures that do not belong to the object when a large number of events are generated by multiple targets, background edges, or dynamic lighting?

Q4. The energy consumption calculation is based on theoretical inference energy, but the experiment was not tested on the edge computing platform.

Reviewer_N2ZJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EventGeoPose, a directly trained Spiking Neural Network (SNN) framework for multi-object 6D pose estimation from asynchronous event streams. The authors identify three limitations in current event-based 6D pose estimation: high energy consumption from dense ANN backbones, motion-state mixing from fixed-duration temporal binning, and a lack of observation-level geometric supervision linking 3D CAD models to 2D event contours. To address these, EventGeoPose combines three main components:

  1. I-LIF Spiking Backbone: Built upon a YOLO-based detection and decoupled pose regression architecture using Integer-valued Leaky Integrate-and-Fire (I-LIF) neurons (D=4D=4), which use integer activations during training via a straight-through estimator (STE) to reduce quantization error in continuous 6D pose regression, and decompose into DD binary spikes at inference.
  2. Pose-Referenced Adaptive Event Encoding (PRAEE): Partitions NN events within an observation window [ts,tr][t_s, t_r] into T=4T=4 equal-count bins anchored to the pose reference timestamp trt_r (assigning the remainder rr to the earliest bin), and constructs a 3-channel tensor per bin consisting of positive event counts Ct+C_t^+, negative event counts Ct−C_t^-, and a log-normalized local 3×33 \times 3 event-rate map QtQ_t.
  3. Event-Geometry Consistency Regularization (EGCR): A training-only 2D image-plane loss that projects J=256J=256 pre-sampled visible 3D CAD model edge points under the predicted pose (R^,t^)(\hat{R}, \hat{t}) onto the 2D image plane and penalizes their distance to a recency-weighted, dilated 2D event distance field DE(u)D_E(u). In addition, the authors introduce two datasets—EVA-6D-Syn (10,000 synthetic sequences of 21 YCB-Video objects simulated via BlenderProc and v2e) and EVA-6D-Real (13,128 frames across 6 objects and 12 sequences captured with a DVXplorer Micro and RealSense D435i with motion-capture pose propagation)—and evaluate on the public YCB-Ev SD benchmark. Experiments report consistent improvements in ADD(-S) at 0.5d0.5d over adapted frame-based baselines while achieving an estimated 6.4×6.4\times reduction in theoretical energy compared to an isomorphic ANN.
Critical strengths
  1. Well-Motivated Problem Formulation and Practical Inference Pipeline: Jointly performing multi-object detection and direct 6D pose regression with an SNN directly addresses the latency and power bottleneck of running heavy ANN pose estimators on high-rate event streams. Designing EGCR as a training-only 2D projection regularizer is also a practical choice that improves geometric alignment (+3.6% on its own, +5.5% on top of PRAEE) with zero inference overhead.
  2. Effective Pose-Referenced Event Representation (PRAEE): Anchoring equal-count event bins to the pose reference time trt_r (placing the remainder in the earliest bin) and augmenting polarity count maps with a duration-normalized event-rate channel QtQ_t is intuitive and well-ablated. Table 5 clearly demonstrates that equal-event binning without the duration-normalized rate channel drops performance from 71.2% to 67.6% (worse than uniform temporal binning at 69.3%), validating that variable bin durations Δt\Delta_t must be explicitly encoded when using count-based adaptive slicing.
  3. Valuable Real and Synthetic Event-Based 6D Pose Datasets: Event-based 6D object pose datasets remain scarce. Contributing both a diverse synthetic benchmark (EVA-6D-Syn with independent camera, object, and coupled motion plus dynamic lighting) and a synchronized real-world RGB-D + Event benchmark (EVA-6D-Real with 240 Hz motion capture ground truth under normal and low-light conditions) is a meaningful contribution to the community.
Critical weaknesses
  1. Overly Loose Evaluation Metric Threshold (0.5d0.5d instead of Standard 0.1d0.1d) Masks Fine-Grained Pose Errors: Throughout all experiments (Tables 1–9), pose accuracy is evaluated exclusively using ADD(-S) at 0.5d0.5d (50% of the object diameter), whereas the universal standard in 6D object pose estimation (e.g., YCB-Video, BOP, GDRNPP, FoundationPose) is 0.1d0.1d (10% of the object diameter) or the Area Under the Curve (AUC) up to 10 cm. For standard YCB objects with diameters of 15–25 cm, a 0.5d0.5d threshold tolerates 3D pose errors of 7.5 to 12.5 cm! At such a loose threshold, a prediction with completely wrong 3D rotation RR can still be counted as "correct" as long as its 3D translation tt is within roughly half an object diameter. Because SNNs are specifically challenged by numerical quantization in continuous regression (the core motivation for using I-LIF), evaluating only at 0.5d0.5d fails to test whether the SNN actually achieves precise rotational and translational fidelity required for robotic manipulation. Standard ADD(-S) at 0.1d0.1d, AUC, and decoupled rotation/translation errors (e.g., <5∘<5^\circ, <5 cm<5\text{ cm}) are essential.

  2. Baseline Selection and Modality Mismatch in State-of-the-Art Comparisons:

    • Absence of Native Event-Based 6D Pose Baselines: Although Section 2 cites several recent event-based 6D pose estimators and datasets (e.g., Liu et al., 2024; Hay et al., 2025b; Kang et al., 2026; Wang et al., 2026; and the baselines reported in YCB-Ev SD by Rojtberg and Kuhn, 2025), none of these native event-based methods are compared in Tables 1–3.
    • Unfair Modality Adaptation for Zero-Shot / Template-Matching RGB Baselines: Four of the six baselines—SAM6D, BoxDreamer, FoundationPose, and PoseGAM—are zero-shot / unseen-object methods that rely on pre-trained RGB foundation features (e.g., DINOv2) or render textured RGB CAD templates to match against input images. Feeding a single-frame event Time Surface (TS) into methods that compare against rendered RGB CAD appearances creates a severe domain mismatch, explaining why FoundationPose (41.4% syn / 9.9% real) and BoxDreamer (6.2% syn / 0.7% real) collapse.
    • Input Representation Disparity for Supervised Baselines: Supervised baselines (YOLO6D-Pose, GDRNPP) are trained only on single-frame Time Surfaces (TS) rather than the multi-step PRAEE representation (T×3×H×WT \times 3 \times H \times W), and importantly, the authors' own ANN baseline (Table 6) achieves 76.6%, which is 5.4 percentage points higher than EventGeoPose (71.2%). Reporting GDRNPP or YOLO6D-Pose trained with PRAEE inputs alongside the ANN version of EventGeoPose in the main tables would provide a much fairer picture of SNN vs. ANN performance.
  3. Data Leakage Risk in the Within-Sequence Split of EVA-6D-Real: According to Appendix B.2 and B.3, EVA-6D-Real consists of 12 recordings (6 objects ×\times 2 lighting conditions) captured with static cameras, and each sequence is split chronologically by taking the first 60% of frames for training and the remaining 40% for testing. Splitting within the same continuous recording with a static camera means the training and test splits share the exact same background, lighting setup, camera extrinsics, and motion generation setup. Furthermore, the paper does not clarify how the objects undergo continuous 6D motion in front of the static cameras in EVA-6D-Real (e.g., moved by a human hand, robot arm, suspension string, or turntable)—if a human hand or robot arm moves the object, it generates dense foreground events rigidly coupled to the object's trajectory. Evaluating across unseen sequences (e.g., training on sequences 1..k and testing on held-out sequences) is necessary to rule out trajectory and background overfitting.

  4. Global vs. Instance-Level Formulation Flaws in PRAEE and EGCR:

    • Global Event Budgeting in PRAEE: PRAEE partitions NN events across the entire 640×480640 \times 480 frame into equal-count bins. In multi-object scenes (like YCB-Ev SD and EVA-6D-Syn), event generation is dominated by the largest, closest, or fastest-moving object (or background motion). Consequently, a smaller or slower object in the same scene does not receive an adaptive temporal partition tailored to its own motion state.
    • Unmasked Global 2D Distance Field in EGCR: In Section 3.4, the 2D event evidence map AE(x,y)A_E(x,y) and 2D distance field DE(u)D_E(u) are computed globally over all events in the image plane without masking by ground-truth instance segmentation or bounding boxes during training. In cluttered multi-object scenes with occlusion, projected 3D CAD edge points u^j\hat{u}_j of object mm can fall onto strong 2D event contours belonging to an occluding or adjacent object (yielding DE(u^j)≈0D_E(\hat{u}_j) \approx 0), corrupting the pose gradient.
  5. Ambiguity in SNN Temporal Aggregation for Pose-Referenced (trt_r) Regression: In SpikeYOLO (Luo et al., 2024), I-LIF is applied to static images repeated over TT steps, and outputs are averaged across all TT steps. Here, however, the T=4T=4 inputs represent sequentially moving object states over [ts,tr][t_s, t_r], while the target pose is defined strictly at the final reference timestamp trt_r. The paper never specifies how the TT temporal steps are aggregated before or within the detection and pose heads. If features/outputs are averaged across all TT steps as in SpikeYOLO, earlier pose states (t<Tt < T) are directly mixed with the reference pose state at trt_r, contradicting the paper's core motivation of avoiding motion-state mixing. Conversely, if only the final timestep k=Tk = T is used for readout, the membrane update in Eq. 3 (uk=λ(uk−1−sg(sk−1))+xku_k = \lambda(u_{k-1} - \text{sg}(s_{k-1})) + x_k with λ=0.25\lambda = 0.25) retains only the post-rounding residual (uk−1−sk−1)∈[−0.5,0.5](u_{k-1} - s_{k-1}) \in [-0.5, 0.5] scaled by 0.250.25, meaning the carried-over membrane potential from step k−1k-1 is bounded by ±0.125\pm 0.125 and earlier temporal information decays rapidly.

  6. Missing Details on Theoretical Energy Calculation and EGCR Hyperparameters:

    • Energy Model: The paper reports exact theoretical energy numbers in mJ (e.g., 135.5 mJ vs. 866.7 mJ) without providing the energy equation, hardware assumptions (e.g., 45nm CMOS energy per MAC vs. AC), layer-wise spike firing rates, or whether non-spiking operations (PRAEE preprocessing, log/division operations, first-layer encoding, and Gram-Schmidt orthogonalization) are included. Notably, in Table 6, standard binary LIF consumes more energy (153.1 mJ) than 4-level I-LIF (135.5 mJ), despite I-LIF decomposing activations into D=4D=4 virtual binary spike steps (Eq. 6); this requires explanation via firing-rate statistics.
    • Unspecified EGCR Parameters: The threshold used to binarize AE(x,y)A_E(x,y) into ΩE\Omega_E, the morphological dilation kernel size, the truncation distance dmax⁡d_{\max} in Eq. 15, and the loss weights in Eq. 29 are omitted from both the main text and the appendix.
Questions for authors
  1. Performance under Standard 0.1d0.1d Threshold and Separate R,tR, t Metrics: Could you report ADD(-S) at the standard 0.1d0.1d threshold, ADD(-S) AUC, and separate rotation/translation errors (e.g., mean/median rotation error in degrees and translation error in cm, or <5∘5 cm<5^\circ 5\text{ cm} accuracy) on EVA-6D-Syn, EVA-6D-Real, and YCB-Ev SD? How large is the accuracy gap between the ANN variant and I-LIF variant at 0.1d0.1d vs. 0.5d0.5d?

  2. Instance-Masked vs. Global 2D Distance Field in EGCR: Since ground-truth instance masks are available during training (Section 4.1), why is the 2D event distance field DE(u)D_E(u) in EGCR computed globally across the whole image plane rather than per-instance using the ground-truth object mask (or dilated bounding box)? Does instance-masking DE(u)D_E(u) further improve accuracy on occluded multi-object scenes?

  3. SNN Temporal Readout vs. Membrane Decay (λ=0.25\lambda = 0.25): How are the T=4T=4 temporal steps aggregated in the detection and pose heads to predict the pose at the final reference time trt_r (e.g., temporal averaging across all TT steps vs. taking only the final step TT vs. learned temporal weighting)? Given that Eq. 3 resets the membrane potential via uk−1−sk−1∈[−0.5,0.5]u_{k-1} - s_{k-1} \in [-0.5, 0.5] and scales it by λ=0.25\lambda = 0.25 (limiting the carried-over potential to ±0.125\pm 0.125), how much information from earlier bins (t=1,2t=1, 2) actually survives to timestep TT?

  4. Energy Calculation and Firing Rates (LIF vs. I-LIF): Please provide the exact mathematical formula and per-operation energy constants used to compute the theoretical inference energy (mJ) in Tables 4–6, along with the average spike firing rates of the LIF and I-LIF models. Why does I-LIF (D=4D=4 virtual steps) consume less energy (135.5 mJ) than binary LIF (153.1 mJ)?

  5. Fair Baseline Comparisons and Missing Hyperparameters: How do YOLO6D-Pose and GDRNPP perform when trained with your 3T3T-channel PRAEE representation instead of a single Time Surface? Also, please specify the threshold and morphological dilation kernel size for ΩE\Omega_E, the truncation distance dmax⁡d_{\max} in Eq. 15, and the loss weights in Eq. 29.

Reviewer_jSBjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents EventGeoPose, a spiking neural network framework for event-based 6D object pose estimation. The method combines adaptive event encoding, an I-LIF-based spiking backbone, and event-geometry consistency regularization. The authors also introduce two event-based pose datasets and demonstrate improvements over several existing methods.

Critical strengths
  • The paper addresses an important problem: efficient and robust 6D object pose estimation from event streams under challenging motion and illumination conditions.

  • The method achieves promising improvements across three datasets. The introduction of synthetic and real-world event-based pose datasets is also a useful contribution.

Critical weaknesses
  • Limited methodological novelty relative to existing event-based representations and geometric alignment methods. The proposed framework primarily integrates established techniques. The I-LIF neuron is adopted from SpikeYOLO [1], while event-count-based temporal partitioning has been explored in prior event-based vision research [2]. More importantly, EGCR shares its core geometric idea with existing event-based 3D–2D alignment approaches, particularly EDFT [3], which also aligns projected 3D model points with an event distance field. Although EventGeoPose incorporates this constraint into network training rather than iterative pose optimization, the methodological novelty beyond this adaptation remains insufficiently established.

[1] Integer-Valued Training and Spike-Driven Inference Spiking Neural Network for High-Performance and Energy-Efficient Object Detection, ECCV 2024.

[2] Events-to-Video: Bringing Modern Computer Vision to Event Cameras, CVPR 2019.

[3] Event-based 6-DoF Object Tracking With Distance Field Reaching 130 Hz, RA-L, 2026.

  • Insufficient experimental comparisons and limited real-world generalization. The experiments mainly compare against frame-based 6D pose estimation methods adapted to event inputs, without evaluating event-based approaches such as AEE [4]. Related spiking event-based pose estimation methods [5] are also not experimentally discussed, although they target spacecraft rather than general objects. Furthermore, EVA-6D-Real adopts a chronological 60/40 split within each sequence, meaning that training and testing share the same objects and motion sequences. This protocol provides limited evidence of generalization to independent real-world trajectories.

[4] Learning 6D Object Pose Estimation With Event Cameras Using Synthetic Data and Domain Randomization, IEEE RA-L 2026.

[5] Spiking Monocular Event Based 6D Pose Estimation for Space Application, arXiv 2025.

Questions for authors
  • Could the authors clarify the key architectural and methodological differences between EventGeoPose and SpikeYOLO [1]? Beyond adapting the existing I-LIF-based spiking framework to 6D pose estimation, what are the substantive technical innovations?

  • Could the authors clarify the key methodological differences between PRAEE/EGCR and existing event representations and geometric alignment approaches, particularly EDFT [3]? Please refer to Weakness 1 for details.

  • Can the authors provide comparisons against event-based 6D pose estimation methods, such as AEE [4], under compatible evaluation protocols?

  • Can the authors evaluate EVA-6D-Real using independent held-out sequences or unseen objects, rather than splitting each sequence chronologically, to better demonstrate real-world generalization?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.