- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors introduce a novel framework for point tracking in using monocular event cameras. Specifically, their major contributions are the use of pre-trigger interval evidence to filter between potential candidate point tracks, learning to lift tracks to 3D with the use of initial depth estimation and local group motion and introducing a new benchmark based on the BlinkVision dataset called EventTraj3D, which includes RGB frames, 2D and 3D metric tracks as well as being balanced across categories. They show improvements in both 2D and 3D tracking against a number of event based point trackers.
Critical strengths
The following are considered strengths of the paper:
- Evaluation performance supports the efficacy of their method. The authors show state-of-the-art performance on their introduced benchmark for both 2D and 3D tracking. They also show good results on the existing E-3DTrack benchmark, coming first on 5/7 metrics and second the remaining 2 by a small margin.
- The model variant ablations further support the novel contribution of including pre-trigger evidence for filtering tracks.
- The shuffled history is a good addition to the ablations to ensure that the temporal history is a relevant contribution.
Critical weaknesses
The following are considered to be weaknesses of the paper:
- The paper is extremely difficult to understand. The abstract and introduction reference lots of field-specific phrases without context (e.g. candidate-evaluation gap, pre-trigger brightness histories, symmetric thresholds, longitudinal motion). The first figure attempts to motivate the problem but it is not properly labelled or referenced well in the text so simply adds to the confusion. The related works section does not attempt to resolve any of these ambiguities. Lastly, the method section is incredibly difficult to follow. It lacks any motivation for each module, so the reader is simply thrust into equations without any understanding of why they are needed, figures 2 and 3 are meant to aid in the understanding but instead cause more confusion due to a lack of explanation of symbols and link back to the text and even the additional method explanations in the supplementary do not fix the confusion.
- Although the new EventTraj3D benchmark does have its merits (e.g. large number of training and evaluation sequences, 2D and 3D tracks alongside event data, ...), the details of how this dataset is created and the contribution over the original BlinkVision is not clear from the text. This needs to be expanded to fully understand the contribution of the dataset.
- There are not enough details on what the "Direct XYZ" variant is in the ablation table. The text says that it "retain encoders, GRU and input features" but this is not clear. What is aim of including this in the ablation? This should be clear.
- The candidate matcher is missing an explanation in the paper. There does not seem to be any details about the architecture, how many candidates are predicted and how these are chosen. The same issue follows for the "Event Backbone" referenced in figure 2 which is not mentioned anywhere in the text or explained in architecture terms.
- Although the "Initial-depth sensitivity" sub-section is a welcome ablation, the writing lets it down so it is unclear what the authors are trying to conclude from the experiment. Furthermore, the comparison to the "Direct head" is also not clear. What type of model is this? What architecture does it have? Why is the comparison against it important?
- Again, although the "Hit-survival analysis" has good intentions and the shuffled history is a good experiment, it is just very poorly explained and therefore the affect of including it is then lost on the reader. A lot of the details are in the supplementary so when reading the main paper we have no context for what we are looking at. The figure in panel a seems to have a box and black point, either of which could be the query but both are together are confusing so makes the reader worry the figure may be AI generated. Lastly, it only uses a set of hand-picked examples and not the whole test set so seems limited.
Questions for authors
- What architecture is used for the candidate matcher?
- What architecture is used for the event backbone? And how does this compare to other models in the evaluation?
- What contributions have been made on top of BlinkVision to construct EventTraj3D? And what are the length of the clips in number of frames?
- Can we have more details on the "Direct XYZ" variant in the ablation table?
- Can we have more details on the "direct head" model from the ablation and why this is used as a comparison?
- How does the method perform on large point displacements? Are these captured by the evaluation benchmarks? If not, is there a good reason for avoiding this?
6 comments
W3 / Q4. What are Direct XYZ and the direct head designed to test?
Direct XYZ and “direct head” denote the same variant. Line 421 introduces “Direct XYZ,” and line 423 immediately refers to it as “The direct head.” App. C.3 (lines 928–931) specifies the same encoder and GRU architectures, input features, initialization, and training protocol, with output units and bounds adapted for direct coordinates.
| Component | Architecture / comparison |
|---|---|
| Four input encoders | 1156 / 775 / 14 / 8 dims → 384 dims each |
| Fusion and temporal model | Reliability-weighted support pooling; Linear(1536,384) → GRUCell(384,384) |
| Direct XYZ decoding | Linear(384,3); direct XYZ in a shared reference scale |
| QART decoding | Linear(384,3) → β·tanh → initial anchor + initial depth × relative offset |
The variants are trained separately and differ in output representation: Direct XYZ regresses coordinates at a common scale, whereas QART normalizes displacement by query depth and restores metric position from a fixed anchor (Sec. 3.3, Eq. 7). The comparison therefore tests how 3D position is represented rather than additional network capacity.
| Output | 2D EPE (px) | 3D EPE (mm) | 3D FDE (mm) |
|---|---|---|---|
| Direct XYZ | 3.886 | 1167.57 | 1644.48 |
| Full QART | 3.213 | 400.69 | 821.74 |
Direct XYZ has comparatively close 2D error but substantially higher 3D EPE/FDE, supporting normalized anchor-relative decoding. The restricted GRU still has higher error with the same relative output, supporting the additional value of candidate and neighborhood information. The revised PDF will use “Direct XYZ (direct head)” consistently and present the architecture and output forms together in App. C.3.
W4 / Q1. Candidate Matcher architecture and proposal selection
The matcher combines current events, initial templates, and temporal context to propose endpoints and base matching scores (Sec. 3.2, lines 216–224). Its structure is summarized below.
| Component | Architecture |
|---|---|
| Query and memory | 1162→768→384 MLP; GRUCell(768,384) |
| Cross-attention | Q: Linear(384,384); K/V: Linear(389,384) |
| Base scores | Feature similarity + endpoint appearance + candidate priors/normalization |
| 2D readout | 769→384→192→2 MLP; posterior mean + residual |
Candidates surround the previous estimate: 207 per query and 47 per support (25 local, 13 disk, and 9 recovery). Cross-attention provides base scores; Hit–Survival and local geometry further assess candidates before probability-weighted 2D readout (Eqs. 2–6), allowing temporal and motion evidence to distinguish similar appearances. The revised PDF will place proposal rules and architecture details beside the matcher description.
W4 / Q2. Event Backbone architecture and inputs
The backbone is the event encoder in Sec. 3.1 (lines 197–203). Five temporal bins and two polarities form a 10-channel volume; C(a,b,s) denotes a 3×3 convolution, eight-group GroupNorm and SiLU, and R(c) a two-convolution residual block. The architecture follows.
| Component | Architecture |
|---|---|
| Event backbone | C(10,96,1) → R(96) → C(96,192,2) → R(192) → C(192,384,2) → R(384) → R(384) → GN(8) → SiLU |
| Event feature map | B×384×(H/4)×(W/4) |
| Reference template | 31×31 grayscale patch → C(1,192,1) → C(192,192,2) → R(192) → C(192,384,2) → mean pooling → 384-D |
| Initialization encoder | Same architecture as event backbone; independent weights |
The backbone supplies matching features, reference patches anchor appearance, and per-pixel trigger order supports candidate-history tests. DeepEvT combines local matching with cross-track attention; ETAP uses a spatiotemporal Transformer. Our convolutional encoder and candidate cross-attention feed history and local-geometry assessment (Secs. 2.1, 3.1–3.2). The revised PDF will consolidate architecture and algorithm details.
Messikommer et al. Data-Driven Feature Tracking for Event Cameras. CVPR 2023, 5642–5651. Paper
Hamann et al. ETAP: Event-based Tracking of Any Point. CVPR 2025, 27186–27196. Paper
W5 / Q5. What does the direct-head depth-sensitivity test show?
“Direct head” is Table 4’s Direct XYZ: line 421 introduces the variant and line 423 immediately uses this name; App. C.3 specifies the architectural controls. Table 4 compares unperturbed accuracy. App. F.2 instead fixes trained weights, applies identical initial-depth perturbations with three paired seeds, and subtracts each model’s unperturbed error. At 10% and 30% noise, added 3D EPE is 272.72/377.77 mm and 904.26/1207.96 mm, respectively (Ours/Direct XYZ; Table F1), supporting lower initialization sensitivity.
With anchor-relative decoding and GRU capacity held fixed, full QART lowers 3D EPE from 488.84 to 400.69 mm relative to the restricted-input GRU, separately supporting the value of candidate and neighborhood information for 3D accuracy. The paired perturbation results show lower initialization sensitivity for the combined pipeline, not independence from the initial metric anchor (Eq. 8; App. F.1–F.2). The revised PDF will clarify these distinct findings and use the name 'Direct XYZ (direct head)' consistently.
W6. Why these Fig. 5 controls, and what do the markers mean?
History analysis and tracking gains. Fixing weights, candidates, and base scores isolates temporal association within the same match (lines 453–461; App. D). Cases (i–ii) move probability toward better locations without changing the mode; case (iii) corrects the selection, reducing mode error from 14.519 to 1.950 px (Table D1). Each case has 64 valid shuffles preserving locations, polarities, event counts, and the multiset of interval durations. True history gives higher P₂ and lower posterior-weighted error than endpoint-only scoring and the shuffled mean in all three cases (Apps. D.1–D.2). P₂ is probability mass within 2 px of ground truth.
On the full test set, removing Survival scoring raises 2D EPE from 3.213 to 3.830 px and also increases 3D errors. Removing the interval objective likewise reduces accuracy. Candidate history thus supports more accurate final trajectories beyond the local cases. The revised PDF will distinguish the purposes of fixed-candidate controls and retrained ablations and connect their settings to the reported results.
Figure markers. The box and dot represent the same query at different times and therefore share the Query identity label. The hollow box marks the initial location; the dark dot marks the previous estimate and the common start of paths to current candidates A and B; the star marks the current ground truth. The revised PDF will explain each marker in the Fig. 5(a) caption and add explicit temporal indices.
Q6. Can SPAR track large accumulated displacements?
Large cumulative displacement primarily challenges long-term point identity and drift control rather than single-step matching. The high temporal resolution of event cameras enables short observation intervals and incremental tracking, as also demonstrated by EKLT (IJCV 2020) and DeepEvT (CVPR 2023). However, small matching errors can accumulate over time, potentially causing trajectory drift or incorrect reassociation with similar neighboring points.
SPAR addresses these challenges through candidate-conditioned pre-trigger evidence and reliability-weighted local geometry, which help preserve point identity and suppress accumulated matching errors (Sec. 3.2, Eqs. 2–6). On EventTraj3D, 70.09% of test trajectories exhibit cumulative displacements of at least 25 px, indicating that substantial accumulated motion is common in this benchmark and directly relevant to continuous event-based tracking. SPAR achieves a 2D EPE of 3.71 px on this subset, only 0.50 px higher than its 3.213 px EPE on the full test set. This limited degradation indicates that SPAR maintains reliable point correspondences under substantial accumulated motion, consistent with its design for mitigating identity switches and cumulative tracking drift.
Gehrig et al. EKLT: Asynchronous Photometric Feature Tracking Using Events and Frames. IJCV 128, 601–618 (2020). DOI: 10.1007/s11263-019-01209-w
Messikommer et al. Data-Driven Feature Tracking for Event Cameras. CVPR 2023, 5642–5651. Paper
W1. Why are Hit–Survival, local geometry, and QART needed?
The candidate-evaluation gap arises when similar observed-event support masks incompatible pre-trigger histories (Introduction, lines 45–75; Fig. 1). Symmetric thresholds are an idealized event-model assumption (App. A.1), and longitudinal motion denotes motion along the viewing ray (App. F.1). Hit–Survival evaluates candidate histories, local geometry reweights motion-consistent supports, and QART retains correspondence reliability and local motion for 3D inference (Secs. 3.2–3.3).
The notation in Fig. 2 is defined in App. A.3 (lines 765–776). In the revised PDF, we will define these terms at first use, introduce each module's purpose before its equations, and clarify the symbols, captions, and main-text links for Figs. 1–3.
W2 / Q3. What does EventTraj3D contribute beyond BlinkVision?
We appreciate the reviewer’s recognition of the dataset’s value. Resources combining events, persistent point identities, and metric 3D trajectories remain limited. EventTraj3D builds on BlinkVision simulation to provide monocular tracking data across indoor/outdoor scenes and rigid, articulated, and non-rigid motion, complementing real stereo E-3DTrack (App. C.1, Table C2).
Construction contributions. We tried propagating 2D points by optical flow and sampling depth, but observed unstable correspondences during propagation and near occlusions and depth boundaries. We therefore bind queries to native surface points, bridge absolute correspondences across reference frames, and check overlapping positions and depths to preserve physical identity. Queries must satisfy local event coverage, silence-duration, and competing-location discriminability checks. The primary query determines clip duration; supports span the clip, while local points retain validity masks. Same-instance overlap is limited, and splits are disjoint by physical instance.
Clip length and scale. Table C1 reports 1,427 clips, 6,494 trajectories, and a 999/214/214 split. Complete clips span 18–96 frames. The revised PDF will include two construction algorithms: native-track linking, and event-quality screening with clip selection, together with the selection thresholds.
Li et al. BlinkVision: A Benchmark for Optical Flow, Scene Flow and Point Tracking Estimation using RGB Frames and Events. ECCV 2024. Paper
Li et al. 3D Feature Tracking via Event Camera. CVPR 2024, 18974–18983. Paper