Fidelity-aware Event Representation Learning

Event cameras excel in dynamic sensing. Harnessing this potential hinges on constructing faithful, compact, and statistically stable representations from irregular events, a fundamental yet nontrivial challenge. Existing event representations face a persistent dilemma: information fidelity is traded for statistical tractability through progressively stronger structural constraints. To reconcile this tension, we propose a fidelity-aware event representation framework that distills asynchronous events into physics-grounded latents while preserving temporally structured event information. Raw events are first structured into conflict-free volumes using the coarsest temporal discretization observed to avoid intra-bin event collisions. We then employ a forward-physics transformation that integrates sparse differential signals into relative log-intensity trajectories. This transformation is exactly invertible by temporal differencing, reorganizing the information retained after discretization into a more coherent temporal form without further information loss, thereby bridging information fidelity and statistical tractability. A variational autoencoder subsequently maps these proxy trajectories into compact and distributionally regular latents that serve as model-ready representations. Extensive experiments demonstrate faithful and stable event latents that transfer across diverse domains and visual objectives, spanning semantic understanding, motion perception, and photometric reconstruction. These results support physics-guided latent learning as an effective approach to event representation. Code will be publicly available.