Long-Range Latent Modeling for Quanta Video Reconstruction and Transmission

Quanta image sensors (QIS) capture dynamic scenes through high-rate binary photon measurements, offering fine temporal resolution but extremely sparse photon evidence in each exposure. High-quality video reconstruction therefore requires jointly exploiting photon evidence over extended observation intervals. Existing QIS reconstruction methods typically use bounded temporal contexts to generate frames at predefined timestamps, fixing the reconstruction as a discrete image sequence before compression and transmission. We propose a temporal latent representation framework for long QIS sequences that models scene evolution throughout the observation interval as queryable latent trajectories. A hierarchical long-range encoder jointly aggregates sparse photon evidence and produces multi-scale temporal latent anchors, providing discrete support for latent trajectories at complementary resolutions. Given an arbitrary temporal coordinate within the encoded interval, a temporal query module constructs the corresponding latent state through interpolation between neighboring anchors and learned residual refinement, and a shared decoder maps this state to a video frame. Scene evolution is thus retained in a reusable latent representation, allowing receivers to select output timestamps after encoding and transmission without revisiting the raw measurements or rerunning the encoder. The same multi-scale latent state supports compact transmission through cross-scale predictive compression, unifying long-range reconstruction, state compression, and on-demand temporal querying within a shared representation. Experiments across five photon levels show that our method outperforms the evaluated QIS reconstruction baselines and effectively exploits photon evidence from longer observation intervals to improve reconstruction quality. For a 4096-frame binary QIS sequence, the compressed latent state occupies only 4.628 MiB, compared with 64.000 MiB for the bit-packed raw measurements, while retaining arbitrary temporal query capability after transmission.