- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Event Generative Compression (EGC), a cross-modal event compression framework in which an event-camera sender transmits a highly compressed discrete latent message and a receiver uses an adapted pretrained video diffusion model to reconstruct RGB video. The method combines event-latent alignment, finite scalar quantization with a 3D checkerboard entropy model, residual flow matching, and generated temporal history, and is evaluated for reconstruction, segmentation, optical flow, and depth on DSEC and BS-ERGB.
Critical strengths
The paper poses a useful alternative to conventional event coding: preserve the information needed for visual reconstruction and downstream tasks rather than requiring event-stream fidelity. This is a meaningful formulation for settings where the receiver consumes visual content rather than events
The system design is coherent. Figure 2 clearly separates the event-only sender, exact entropy-coded symbol transmission, and the receiver-side generative prior. In particular, the architecture makes the operational assumption clear: no RGB frame or transmitted reference frame is available to the sender at inference. Figure 3 also provides a helpful account of what is optimized and frozen at each of the three training stages.
The temporal-history mechanism is supported by both visuals and metrics. visible window-boundary color discontinuity without history that is mitigated by generated history, while Table reports substantial decreases in the seam measure on both DSEC and BS-ERGB. This is a well-chosen ablation because temporal continuity is a central practical challenge once no reference frames are transmitted.
The empirical rate regime is striking.Table 3 verifies that the learned checkerboard context model actually reduces the measured container rate relative to zstd while preserving exactly the same symbols. This is much stronger than reporting only estimated entropy or nominal FSQ capacity.
Critical weaknesses
The baseline suite is incomplete for the paper's central claim about event compression. The comparison emphasizes reconstruction methods combined with random retention, temporal downsampling, or GLC, but does not evaluate against a dedicated event-stream compression method such as flow-based visual stream compression. Rezaee et al. is cited as lossy event compression work but is not included experimentally. This is important because EGC’s stated advantage is not only good event-to-video reconstruction, but substantially better preservation of useful visual/task information at a given event-transmission budget. A direct event-compression baseline should be encoded at matched rates and evaluated through the same reconstruction and downstream pipelines. Otherwise, it remains unclear how much of the gain derives from a stronger receiver prior versus the proposed learned compressed message.
Stage 3 trains with zero or perturbed ground-truth history, whereas inference conditions on the model's own increasingly imperfect output. This can produce long-horizon drift, identity swaps, or temporal inconsistency not captured by two boundary examples or a seam statistic. The paper should report metrics over substantially longer rollouts, error as a function of window index, and recovery behavior after a reset or corrupted history. This is particularly important because the method is presented for continuous remote perception.
This may be acceptable for an offline or server-side receiver, but it is a substantial limitation for low-latency remote perception. The paper should report receiver memory, end-to-end latency including coding/transport/decoding, throughput at the stated resolution, and at least a discussion of whether a less expensive generative receiver changes the rate-quality conclusions.
Questions for authors
Please provide the complete Stage-1, Stage-2, and Stage-3 objectives in the main paper, including all coefficients, the FSQ parameterization, the flow-matching schedule the time sampling distribution, and the exact treatment of history during training.
Can the authors compare to a dedicated event-stream compression baseline at matched rates, using the identical reconstruction receiver and downstream task protocols? This would directly test the paper’s compression claim.
How much do reconstruction and task metrics vary across diffusion sampling seeds? history handled across adjacent windows ?, and what happens after long rollouts, a receiver restart, or a corrupted/generated-history mismatch?
2 comments
Thank you for your feedback and reassessment
Thank you for taking another look at the paper and for revising your assessment. We have included the additional results and training details in the revision, and made their locations easier to find in our response. We appreciate the time you have spent discussing the work with us.
Long rollouts, history recovery, receiver cost, and additional comparisons
Thank you for your assessment and suggestions. We have clarified the rollout length and added history-intervention results and receiver timing measurements. The revised manuscript includes the additional reconstruction and downstream comparisons in Figures 8 and 9.
1. Long rollouts and history feedback. The 121-frame length refers to one generation window. Our DSEC full-stream evaluation includes a continuous rollout of 25 windows, producing 2,827 frames over 113.08 seconds. History is updated from the model's own outputs throughout this rollout. Section A.3 now states the sequence length explicitly.
To examine error propagation, we evaluate a one-time history perturbation and a history reset at the same selected window of this DSEC sequence. After the intervention, normal generation resumes and the affected outputs continue to supply history to later windows. The comparisons use the same event inputs and paired sampling settings. We report reconstruction quality over time and the difference from the unperturbed rollout to distinguish the immediate effect from its persistence in later windows. Section D.3 and Figures D1–D2 now report these results. Normal-rollout quality fluctuates without sustained deterioration over this sequence. At window 9, Gaussian noise with standard deviation 0.5 is added to the most recent history position, or all eight history positions are reset to zero. With noise, LPIPS distance to the paired normal rollout decreases from 0.00990 at window 9 to 0.00231 at window 11, then rises to 0.00851 at window 25. Reset causes a larger difference, decreasing from 0.14943 at window 9 to 0.03860 at window 25; the reset branch does not fully return to the normal trajectory.
2. Receiver cost and first output. Section F.3 adds a measurement from events in host memory through actual CCM bitstream construction to reconstructed RGB in host memory. On one GPU at 640 × 464 resolution with 25 sampling steps, an independent 121-frame window takes 4.015 s at the sender and 17.880 s at the receiver, totaling 21.895 s of processing. Collecting the first event window takes 4.84 s. Assuming a net transmission rate of 10 Mbit/s, its 14,537-byte bitstream takes approximately 0.012 s to transmit. The first reconstructed window is therefore available after approximately 26.75 s in this sequential setting. Processing time is measured; transmission time is estimated from the stated bandwidth. Model loading and warm-up are excluded.
The receiver throughput is 6.77 fps for this independent window and 6.22 fps for the eight-window measurement. The separately measured receiver peak allocated GPU memory reported in Section F.2 is 20.133 GiB. The current receiver requires buffering and does not keep pace with a continuous 25 fps input on this GPU. The intended use is bandwidth-limited transmission where receiver-side computation and delayed reconstruction are acceptable. Moving video generation to the receiver reduces computation at the event-only sender (Table 1a). A faster or smaller receiver is an important next step; its rate–quality trade-off needs a separate evaluation. Section F.3 also reports throughput and first-output measurements for other diffusion-based reconstruction methods, with their measurement settings specified.
3. Additional compression comparisons. We have completed reconstruction and downstream comparisons for JPEG2000, G-PCC, x265/HEVC, and DCVC-FM. These cover event-domain compression followed by reconstruction and reconstruction followed by video compression. Main-text Figure 8 reports grayscale LPIPS against measured bitrate, and Figure 9 reports the three downstream tasks with shared frozen task networks for the RGB routes. Supplementary Section C.7 gives codec settings and the corresponding Q comparison. The fixed-receiver comparison in main-text Table 2b and Supplementary Table C3 continues to isolate the transmitted representation using the same generative receiver.
4. Training details and sampling seeds. We have added the straight-through treatment of quantization, the flow-matching time-sampling distribution, the 25-step Euler schedule with shift 5, and the velocity-loss definition to main-text Sections 3.2–3.4. Supplementary Section A.2 gives the complete objectives and coefficients. History is zeroed with probability 0.5 during training; otherwise Gaussian noise with standard deviation 0.1 is added to ground-truth history. Inference uses generated history. Supplementary Figure D2 also reports an additional seed through window 14, showing a similar initial decrease in the effect of history noise.