- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes H-DDCM, a training-free generative image and video compression framework built upon Turbo-DDCM. The main idea is to replace dense Gaussian codebooks with implicit randomized Hadamard dictionaries, allowing efficient residual projection and reconstruction through the Fast Walsh-Hadamard Transform. The authors also propose a table-free lexicographic coding scheme to reduce the computational and memory costs of encoding selected codebook indices.
For video compression, the method is extended to a pretrained autoregressive video generation model, with a re-encoded-tail KV-cache refresh mechanism to maintain temporal context across video segments. Experiments on image and video datasets show that H-DDCM achieves reconstruction quality comparable to full-codebook Turbo-DDCM while substantially reducing memory requirements. The video implementation also achieves higher throughput than the evaluated training-free methods using the same generative backbone.
Critical strengths
I think the strongest contribution of this paper is that it addresses a real scalability problem in training-free generative compression. The memory cost of dense Gaussian codebooks becomes impractical with modern high-dimensional generative models. Replacing them with an implicit Hadamard dictionary is a simple but effective solution. In particular, the FLUX experiments show that H-DDCM can achieve reconstruction quality close to full-codebook Turbo-DDCM without materializing an extremely large dictionary. This is a meaningful improvement for the DDCM framework.
I also appreciate the attempt to extend the method to streaming video compression. The re-encoded-tail KV-cache refresh provides a practical way to preserve temporal context across segments without transmitting additional bits. The corresponding ablations are useful and show improvements over directly resetting the cache or allowing it to grow indefinitely.
Critical weaknesses
My first concern is the gap between the substantial reduction in theoretical complexity and the actual end-to-end efficiency. Replacing dense codebook operations with the Hadamard transform is clearly beneficial, but the improvement in overall runtime is much less impressive. On SD 2.1, both H-DDCM and full-codebook Turbo-DDCM require approximately 1.3 seconds for a 512×512 image, so there is almost no practical speed improvement. On FLUX, the round-trip time decreases from approximately 66 seconds to 44 seconds, which is meaningful, but still extremely slow for compressing and reconstructing a single 1024×1024 image. The situation is similar for video: the reported throughput is only around 10–16 FPS at 832×480 resolution on an NVIDIA H200 GPU. This is far from the real-time performance that modern video compression systems increasingly target.
I understand that training-free compression operates under different assumptions from trained codecs, and I do not expect H-DDCM to outperform all existing compression systems. However, recent works have made substantial progress toward practical high-speed compression, including CoD-Lite (Jia et al., ICML 2026) for generative image compression, GVC-RT (Dang et al., ACM MM 2026) for generative video compression, and strong neural video codecs such as DCVC-FM (Li et al., CVPR 2024) and DCVC-UF (Li et al., CVPR 2026). DCVC-UF is already included in the paper, but its runtime is not directly compared. I would like to see a clearer discussion of how H-DDCM fits into this rapidly developing landscape. Direct speed numbers from different hardware are not necessarily comparable, but the overall rate-distortion-perception versus computational cost trade-off is important for judging the practical significance of this work. My second and more important concern is the remaining potential of the DDCM framework itself. H-DDCM largely removes the explicit codebook memory bottleneck and achieves quality close to full-codebook Turbo-DDCM, but it does not substantially improve reconstruction quality over that baseline. Meanwhile, the overall computation is still dominated by the generative sampling process, and the method remains behind several trained compression approaches in distortion and practical speed. This makes me wonder whether the current results are approaching a practical performance ceiling of noise-codebook-based compression with frozen generative models. If the main remaining bottleneck is the repeated evaluation of a large generator, further optimizing the dictionary may provide diminishing returns. The video results also do not show a consistent advantage across all quality metrics, making the long-term competitiveness of this approach less clear.
I am not suggesting that the DDCM paradigm has necessarily reached its limit. There may still be opportunities in better noise selection, trajectory optimization, bitrate allocation, or reducing generative model evaluations. However, the paper does not provide much insight into these possibilities. Since the work mainly improves the implementation efficiency of an existing compression framework, I think understanding whether that framework still has substantial room for improvement is particularly important to assessing the significance of this contribution.
Questions for authors
- Practical efficiency and comparison with recent codecs
The proposed Hadamard dictionary substantially reduces codebook computation and memory, but why does this lead to almost no end-to-end speed improvement on SD 2.1? Could the authors provide a breakdown of the time spent on generative model inference, residual matching, codebook operations, and bitstream coding?
Also, how should we understand the practical efficiency of H-DDCM relative to recent fast compression methods such as CoD-Lite (Jia et al., ICML 2026), GVC-RT (Dang et al., ACM MM 2026), and strong DCVC models such as DCVC-FM and DCVC-UF? I am not necessarily requesting direct comparisons across incompatible hardware or training settings, but a clearer analysis of the quality–speed trade-off would be helpful.
- Fairness of the DCVC-UF comparison
Could the authors clarify the exact DCVC-UF configuration used in the experiments, including the model variant, intra period, number of coded frames, and color-space conversion? Since DCVC-UF is already included as a strong trained baseline, reporting or discussing its encoding and decoding speed under comparable conditions would help readers better understand the practical trade-offs. It would also be useful to explain how the different operating assumptions and compression objectives affect the interpretation of the perceptual and distortion results.
- Remaining potential of the DDCM paradigm
This is my most important question, and the answer would significantly affect my assessment.
H-DDCM has largely addressed the codebook scalability problem, but the overall compression performance remains limited by the expensive generative process. Do the authors believe that noise-codebook-based compression with frozen generative models is approaching its practical rate-distortion-perception and efficiency limits?
If not, what are the main remaining bottlenecks, and what concrete improvements could lead to substantial further gains beyond replacing or accelerating the codebook? For example, could optimizing the full sampling trajectory or reducing the number of generative model evaluations significantly improve the current results?
I would particularly appreciate an answer grounded in the behavior observed in this paper, rather than only general suggestions for future work. A convincing explanation of the remaining potential of this approach would make me considerably more positive about the paper.
I am currently leaning toward a rating of 6/10. However, since the review system uses a four-point scale, this would correspond to approximately 2.4/4, so I am selecting 2 for now. My assessment is still somewhat borderline, and I am open to raising my rating if the authors can convincingly address my main concerns during the rebuttal.
7 comments
Comparison with recent codecs: trained methods, such as DCVC-FM and DCVC-UF, are commonly distortion-oriented. Most trained codecs emphasize distortion-oriented optimization, although not all do: GVC-RT, for example, explicitly targets perceptual quality, however it suffers from visible artifacts, which can be observed in Fig. 4 and in our anonymous website. Training-free methods such as H-DDCM offer a complementary quality-speed trade-off by reusing a pretrained generative prior, which provides high perceptual quality, without codec-specific training.
We have evaluated CoD-Lite for comparison with H-DDCM. The following table reports the obtained metrics (PSNR, FID and round-trip time) for 2 illustrative BPP values, approximately 0.015 and 0.035, on the Kodak24 dataset. The results demonstrate that H-DDCM achieves comparable performance, without any architecture fine-tuning or training. We have added CoD-Lite comparison to the paper (see Figs. 1, S1 and S2).
| Method | BPP | PSNR ↑ | FID ↓ | Round-trip time (s) ↓ |
|---|---|---|---|---|
| CoD-Lite | 0.0157 | 20.75 | 50.57 | 0.035 |
| H-DDCM | 0.0148 | 20.53 | 49.82 | 1.141 |
| CoD-Lite | 0.0314 | 21.98 | 43.64 | 0.035 |
| H-DDCM | 0.0347 | 22.66 | 40.23 | 1.422 |
GVC-RT is concurrent work (first released on arXiv in August 2026), which we additionally evaluated on all 5 video datasets. DCVC-UF was already evaluated in our submission. We added GVC-RT to the quantitative and qualitative comparisons (Figs. 3, 4, and S8-S14). The qualitative comparison in Fig. 4 and the videos on our anonymous website reveal grid-like artifacts in GVC-RT reconstructions. To complement these visual observations, we additionally measure source-motion-guided warping error following [1-3] - optical flow is estimated from the source video, used to warp adjacent reconstructed frames, and their discrepancy is measured after rejecting pixels that fail forward-backward flow-consistency checks. This measures consistency with source-estimated motion. At approximately 0.0056 BPP on MCL-JCV, GVC-RT achieves higher PSNR and better DOVER/FVMD than H-DDCM but exhibits worse spatio-temporal jitter artifacts, as evident by the fact that H-DDCM has lower source-flow warping error ( versus ). DCVC-UF has lower warping error (), but at the cost of far worse perceptual quality compared to H-DDCM, as can be observed in the qualitative figures. GVC-RT also offers higher throughput compared to our method (more than 100 FPS). These results illustrate distinct quality-speed trade-offs rather than uniform superiority of one method.
| Method | BPP | PSNR ↑ | DOVER ↑ | FVMD ↓ | Warping Error (×10⁻³) ↓ |
|---|---|---|---|---|---|
| GVC-RT | 0.0063 | 23.47 | 0.608 | 244.79 | 1.96 |
| DCVC-UF | 0.0056 | 26.44 | 0.595 | 515.74 | 0.83 |
| H-DDCM | 0.0056 | 20.94 | 0.604 | 419.67 | 1.6 |
Indeed there is a cost in adapting diffusion generative models for downstream tasks due to their iterative sampling process. However, they provide a valuable prior on how the data is distributed which is exactly the point that allows us to achieve high perceptual quality. Many trained video codecs prioritize distortion, although there are some perception-oriented trained codecs such as GVC-RT. Refer to our anonymous website for comparisons between video compression methods, which show that indeed training-free methods, especially H-DDCM, can achieve high-perceptual quality results in this regime.
In contrast to task-specific trained codecs, H-DDCM can be adapted to different distributions or modalities when a compatible pretrained generator is available, without codec-specific retraining. Trained codecs typically require redesign and retraining for such changes. Additionally, trained methods could be trained on a specific dataset distribution, and adapting them to other distributions requires retraining, whereas a compatible pretrained generative model can provide the prior needed for training-free adaptation.
We have included a dedicated paragraph regarding the discussion between training-free and trained methods in the revised manuscript (see Sec. 6).
DCVC-UF configuration (Q2):
We use Microsoft's official pretrained DCVC-UF HT-S model with its corresponding intra-frame checkpoint. The configuration uses one initial I-frame (intra_period=-1), followed by 8-frame inter chunks, skip_thres=0.15, and feature-memory reset interval 128. Intra/inter QPs are equal and selected per sequence to approach the target physical bitrates using actual entropy-coded stream sizes. These settings follow the official example configuration. Inputs use 8-bit YUV420 via FFmpeg, and all methods are evaluated with the common RGB-based metric pipeline.
DCVC-UF targets rate-distortion efficiency and high throughput, whereas H-DDCM uses a generative prior to achieve high perceptual quality at very low rates. We emphasize that H-DDCM is a training-free method, which inherently follows the generative backbones bottleneck, while other trained methods such as DCVC-UF achieve higher throughput, and target low distortion at the cost of perceptual quality. DCVC-UF achieves encoding and decoding throughput of approximately a thousand FPS.
Remaining potential of the DDCM framework (W2/Q3):
We view H-DDCM as removing one bottleneck - the computation ceiling timing is the model calls, and H-DDCM almost reaches it. We believe that in terms of rate-distortion-perception for noise-codebook compression the DDCM framework can still be improved. The H-DDCM framework opens room for methods that would previously have been impractical at large latent dimensions. In particular, H-DDCM tracks the quality of complete-codebook Turbo-DDCM on the image compression task, and outperforms it on the video compression task, while avoiding its dense dictionary operations. This suggests that the choice of dictionary representations can be further improved to achieve better rate-distortion-perception tradeoff under the evaluated settings. The H-DDCM framework does not establish an intrinsic ceiling for the broader DDCM framework.
-
Faster reconstruction trajectories and fewer generator evaluations - the remaining runtime is largely governed by pretrained generator inference, particularly at smaller latent sizes. A direct path is to use fewer sampling steps or faster few-step generators, as our autoregressive video experiments already demonstrate. Another promising approach is to exploit inversion-guided noise selection (e.g., ODE inversion of the source) to make fewer step trajectories more informative; efficient inversion and sampling methods, including FireFlow [4], provide concrete algorithmic tools to explore this in compatible flow backbones.
-
Better use of the freed compute and memory for rate-distortion-perception optimization - our current selection is greedy and local to each step. With implicit -memory dictionaries and fast transforms, the encoder can afford more sophisticated search, e.g. optimizing noise choices jointly across a sampling trajectory, adapting bit budgets across timesteps and video segments, and potentially online source-conditioned optimization. These strategies target the quality ceiling of the current selection procedure under the H-DDCM framework, rather than the already-reduced dictionary cost. They may improve quality at fixed bitrate or reduce the bits needed for a target quality. These are concrete algorithmic opportunities enabled by removing the dictionary bottleneck, and are interesting future directions.
The same scalability also broadens the range of usable pretrained priors. Larger latent dimensions, new modalities, and pixel-space generative models such as JiT [5, 6] would make dense-codebook compression particularly costly, whereas H-DDCM's implicit dictionary makes adaptation more feasible without training a new codec. Thus, the novelty is not merely a constant-factor speedup - it removes a scaling obstacle and creates computational headroom for trajectory optimization, smarter rate allocation, and faster generator/inversion integration. We view these as concrete opportunities enabled by H-DDCM, while distinguishing them from the improvements already demonstrated here.
[1] Lai, Wei-Sheng, et al. "Learning blind video temporal consistency." (ECCV’18)
[2] Ceylan, Duygu, Chun-Hao P. Huang, and Niloy J. Mitra. "Pix2video: Video editing using image diffusion." (ICCV’23)
[3] Geyer, Michal, et al. "Tokenflow: Consistent diffusion features for consistent video editing." (ICLR’24)
[4] Deng, Yingying, et al. "Fireflow: Fast inversion of rectified flow for image semantic editing." (ICML’25)
[5] Li, Tianhong, and Kaiming He. "Back to basics: Let denoising generative models denoise." (CVPR’26)
[6] Vallaeys, Théophane, Jakob Verbeek, and Matthieu Cord. "Pixel-space video generation with U-JiT." (arXiv 11/26)
Practical efficiency and comparison with recent codecs (W1/Q1):
In the tables below we report a per-step runtime breakdown of model inference, codebook operations and bitstream coding, for H-DDCM and Turbo-DDCM, on SD2.1 and FLUX at a fixed bitrate ($\sim$0.05 BPP, averaging over 5 images each). We emphasize that both methods use our bitstream implementation, so Turbo-DDCM uses our efficient implementation without its combinatorial tables bitstream implementation.
SD2.1 (Kodak24, 512x512, d = 16384):
FLUX (DIV2K, 1024x1024, d = 262144)
The gain is small on SD2.1 compared to FLUX, as the round trip is dominated by the diffusion model: on SD2.1 the network calls of the decoder alone take $\sim$0.8s of the $\sim$1.4s. The dense codebook of Turbo-DDCM costs O(dk) per step, against O(dlogd) for our fast Walsh-Hadamard transform. At the small SD2.1 latent (d=214) the dense codebook is only $\sim$0.5GB, and one step costs $\sim$3ms against a 27ms UNet call - the codebook accounts for just 0.09s of Turbo-DDCM's 1.44s round trip, so removing it can save at most a few percent. Therefore, when the latent dimension is small, H-DDCM's runtime advantage over Turbo-DDCM is small, and its benefit is mainly memory (dense buffer of 0.54GB vs. less than 1MB) and scalability. The dense cost grows quadratically with d: from SD2.1 to FLUX, the Turbo-DDCM codebook step grows ∼280× (3ms → 838ms), while the model call grows only ∼34× (27ms → 921ms) and ours stays at $\sim$2ms. On FLUX the codebook is then a half of a Turbo-DDCM compress step and a third of its round trip (22s of 66s), whereas H-DDCM implicit codebook operations are less than 1% of the model call, resulting in 0.06s, which explains 66s for Turbo-DDCM, and 44s for H-DDCM. The advantage keeps growing with the latent size (higher resolutions, video).
We have added in App. C.1 Fig. S3 which demonstrates the runtime breakdown efficiency comparison between H-DDCM and Turbo-DDCM codebook.