acceptodds
Under review as a conference paper at ICLR 2027

Events Are Not Unit Tokens: Learning Threshold-Consistent Representations for Event Cameras

Abstract

Event cameras encode a brightness change as an asynchronous event once the log-intensity variation reaches a contrast threshold C. This sensing rule gives event streams their high temporal resolution, but it also means that an event is not a threshold-independent unit token: the same visual change can produce many events at a low threshold and only a few events at a high threshold. Most event representations aggregate counts, timestamps, polarities, or recurrent tokens without preserving the physical contrast carried by each event, so a change in sensor threshold can be mixed with a change in visual content. To address this issue, we propose Pixel Contrast Flow (PCF), a threshold-consistent event representation that uses physical contrast mass as the state-update variable. PCF combines contrast-mass encoding, a semigroup contrast flow, and pixel-local temporal integration before spatial mixing. The resulting update is unchanged when the same contrast is split into finer event quanta, while pixel locality removes sensitivity to threshold-induced reordering across different pixels. On the paired Caltech101-CT benchmark, PCF maintains a nominal accuracy of 49.59% while achieving the best accuracy at 0.25C_0, 0.5C_0, and 4C_0, the highest worst-case OOD accuracy of 19.06%, and the lowest paired feature drift with a RelL2 of 0.215. Detailed robustness tests further show that PCF obtains the best Global Mean, Global Worst, polarity-asymmetry accuracy, and competitive pixel-mismatch performance. These results support a simple principle: event representations should follow the physical quantity measured by the sensor rather than the arbitrary number of emitted tokens.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Event Cameras, Representation Learning, Threshold Consistency, OOD Generalization, Sensor Robustness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZVEVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Pixel Contrast Flow, an event representation designed to reduce sensitivity to changes in an event camera's contrast threshold. The method converts the number of events in each pixel and temporal bin into a contrast-mass quantity, applies a learned exponential flow to pixel-local states, and performs spatial mixing only after local temporal integration. The authors argue that the resulting update is invariant to splitting a fixed contrast into finer event quanta and less sensitive to changes in cross-pixel event ordering. They evaluate the method with algebraic diagnostics, a synthetic threshold sweep, a paired Caltech101-based event benchmark, robustness tests for global and polarity-specific threshold changes and pixel mismatch, and a threshold-estimation sensitivity experiment. The reported results suggest lower representation drift and improved worst-case threshold robustness, although the evaluation remains confined to synthetic classification settings and the strongest baseline is not consistently surpassed on average accuracy.

Critical strengths
  1. The observation that event multiplicity depends on the sensor contrast threshold is well motivated, and replacing raw event counts with a contrast-mass quantity is a reasonable design direction. The exponential flow has a clear compositional property for contrast increments, and the pixel-local construction gives a principled explanation for why cross-pixel update order can be made irrelevant before spatial mixing.
  2. Regenerating streams from the same image, motion trajectory, and class under different thresholds is a sensible way to separate threshold robustness from changes in visual content. The paper also includes mechanism-level diagnostics and ablations that attempt to distinguish contrast normalization from the proposed local flow. These are valuable starting points for studying threshold variation, even though they do not by themselves establish performance on real sensors.
Critical weaknesses
  1. All main results are obtained from synthetic streams generated from static Caltech101 images under fixed camera trajectories. The E1 and E2 experiments use additional simplified simulations, while E3–E5 remain variations of the same synthetic classification setting. The paper does not evaluate a real event-camera dataset, a sensor-recorded threshold change, or an independent event simulator with clearly documented sensor nonidealities.
  2. It also reports no downstream task beyond classification and does not compare against common event representations such as voxel grids, time surfaces, HATS, EST, asynchronous convolutions, or modern event backbones. The baselines are mainly custom GRU variants, and the paper does not report sufficient parameter, hidden-state, FLOP, or training details to establish capacity-matched comparisons with PCF’s multiple state channels, temporal summaries, and spatial residual network.
Questions for authors
  1. Can the authors evaluate PCF on at least one real event-camera dataset or sensor-recorded threshold variation, using common event-representation baselines and capacity-matched models?
  2. In the global-shift, polarity-asymmetry, and pixel-mismatch experiments, does PCF receive the true effective threshold for every pixel and polarity? How would this threshold field be obtained on a real sensor, and how does performance change when it is unknown or estimated from calibration data rather than supplied by the simulator?
Reviewer_9xjdOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors present Pixel Contrast Flow, a way of representing event camera data to represent the underlying contrast changes in order to improveme robustness to contrast threshold changes. PCF weights events by their contrast threshold and processes them using pixel-local exponentially decayed state updates. The authors evaluate this through paired event streams generated from identical visual trajectories at different thresholds as well as under polarity asymmetry, pixel-level threshold mismatch and calibration errors. PCF achieves improvements for representation drift and classification accuracy. However, I have some thoughts about the novelty.

Critical strengths
  • The paper addresses a problem that is of practical importance to research in the event sensor field.
  • I like the paired-threshold evaluation method. Would be nice to have that with some real recordings too though (could get “close enough” by recording a preset video recording or something).
  • The results demonstrate a substantial reduction in representation drift across contrast thresholds.
  • The robustness evaluation is comprehensive, including global threshold changes, polarity asymmetry, pixel-level threshold mismatch, and sensitivity to calibration errors.
Critical weaknesses
  • So events in of themselves do not represent a physical contrast change since they are dependent on the contrast threshold. However, there have been lots of efforts to calibrate events, e.g “Event Camera Calibration of Per-pixel Biased Contrast Threshold” which this paper cites, or in “Reducing the sim-to-real gap for event cameras” or in “Back to Event Basics”. As far as I can tell this paper is different to that, because it could learn a nonlinear feature transformation that operates directly on contrast increments. However if the same flow operator is used for all contrast increments, then I think this just becomes equivalent to event sensor calibration. So basically, this paper would need to show it is substantially different to these approaches IMO. I would like to see an ablation where contrast threshold is calibrated, the resulting calibrated events are fed into the same spatial backbone as PCF and then compare results to PCF. OK, so that was a bit of a mouthful. TL;DR: What information does your learned contrast flow retain that a calibrated contrast integrator followed by a learned encoder cannot?
  • The exponential decay on the contrast flow seems similar to the filtering in “Continuous-time intensity estimation using event cameras”. I think that should be discussed in the literature review.
  • I am a little confused by how the semigroup property applies. If I understood correctly, then the events are aggregated into temporal bins before applying the flow. So if you consider several events with the same binned contrast mask, then the output will be identical regardless of whether the update is exponential, quadratic or some arbitrary function in fact…. In other words, to me it seems that the invariance follows from aggregating the event contrast before the network is applied to it, not from the semigroup property. Anyways, this does not obviate the paper, but it obviates the bulk of the mathematics that the authors present.
  • The comparison with Mass-GRU and ContrastFlow have several confounding variables/design decisions that are not ablated well. e.g recurrent update, state locality. It is currently hard to tell what contribution comes from the mathematical contribution and which comes just from the architecture changes.
  • Re-reading some of the relevant literature, I am really a bit worried about the novelty here. Wang et al. (2019) [Event Camera Calibration of Per-pixel Biased Contrast Threshold] already address per-pixel contrast-threshold calibration, while Scheerlinck et al. (2018) [Continuous-time Intensity Estimation Using Event Cameras] employ contrast-weighted event integration + pixel-local temporal filtering and exponential decay. Furthermore, Stoffregen et al. (2020) [Reducing the Sim-to-Real Gap for Event Cameras] investigate CT variations and their impact on CNN generalization. Although PCF introduces a particular learned contrast-dependent update, it is unclear what fundamentally new insight or capability it provides beyond these established approaches. The authors need to better position their contribution relative to this prior work, provide comparisons and acknowledge that work.
  • No evaluation at all on real data, not even qualitative. Hard to validate the natural distribution of CTs in a real sensor this way.
Questions for authors
  • Global-flow interleaving error and Pixel-local interleaving error are given to a lot of decimal places! Is that necessary?
Reviewer_ZAJ2OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an event representation named contrast flow, targeting representation robustness across different threshold tuning settings by designing a learnable state-evolution mechanism that strictly obeys the semigroup property. This paper also simulates a benchmark with paired GT contrast values to measure robustness for a classification task. Experiments including classification accuracy, feature drift, and generalizability on unseen thresholds prove the threshold-consistency of the proposed representation.

Critical strengths
  1. This paper proposes a quite novel and important point on the threshold-consistency requirement of event representations.
  2. The proposed learnable state-evolving mechanism strictly exploits the semigroup property as a realization of threshold-consistency, which also commutes with spatial operations for ease of learnable feature extraction.
  3. This paper conducts detailed experiments on the simulated paired dataset to verify the claimed threshold independence of the proposed representation
Critical weaknesses
  1. Experiments are conducted only on the proposed Caltech101-CT benchmark and no other compared methods.
Questions for authors
  1. In view of the state-update mechanism, what is the difference between constructing a contrast flow and an intensity sequence? It seems like once the contrast value is known before and used to normalize the intensity sequence, the threshold consistency can also be preserved.
  2. The state-update mechanism is intentionally defined on discrete temporal bins to achieve threshold consistency. But since scene dynamics vary heterogeneously in both space and time, is it possible to extend the semigroup property to a continuous field? I'm rather curious about the equivalent concept in this case.
  3. The current problem setting is deterministic, but real events are essentially stochastic in both space, time, and event count. Is the proposed framework applicable to such a case, and if so, what minor modification is required?
  4. Little information is provided about the event simulation process. It's better to clarify both the diversity of simulated trajectories and the chosen event simulator (ESIM, v2e, DVS-Voltmeter, or self-defined ones). I would suggest that the authors carefully examine the implementation of DVS-Voltmeter if haven't, as it is closely related to the continuous-time and stochastic nature of events.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.