- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LOONA, a controllable encoding framework for zero-shot diffusion image compression. It models per-step noise selection as a linear surrogate of the next-step objective, derives the optimal scoring coefficient via Stein's identity, and approximates it with one backward pass, so that any differentiable objective can guide codebook atom selection without changing the bitstream format or the decoder. The same encoder supports pixel fidelity, full-resolution region-of-interest, semantic preservation, and perception-distortion traversal, achieving substantial BD-rate reductions over Turbo-DDCM.
Critical strengths
Stein's identity rigorously establishes that the optimal linear scoring coefficient equals the noise-averaged gradient, and one backward pass approximates it with a number of network evaluations independent of codebook size and atom count. This elevates a previously heuristic correlation-based score into a principled selection criterion with minimal changes, and the reasoning is clean.
Critical weaknesses
- The approximation chain from the theoretical optimal score to the practical gradient lacks quantitative validation. The argument in Appendix C relies on replacing the average of a product by the product of averages and treating the step Jacobian as a scaled identity, both exact only when the loss is nearly affine or single-step curvature is negligible. LPIPS, DINOv2 features, and Re-IQA are strongly nonlinear objectives, yet the paper never measures the resulting approximation error or states when the approximation breaks down.
- The method depends on several hyperparameters tuned on the DIV2K training set, including the gradient-start threshold, the number of gradient steps, the loss weights, and the choice of the steering quality metric, and the sensitivity of these choices across datasets and objectives is not clear.
Questions for authors
- Can you quantify the approximation error between the deployed gradient and the theoretical optimal score for LPIPS, DINOv2, and Re-IQA?
- Under what conditions does replacing the average of a product by the product of averages fail, and does the error vary across bitrates and diffusion steps?
3 comments
Thank you for the detailed responses. These responses addressed my concerns and I believe this is a worthy submission.
However, when evaluating whether this paper has greater significance, I would like to ask the authors again. Is the strategy of this paper likely to be extended to other diffusion-based image compression paradigms (such as PerCo or post-processing paradigms), or is it only applicable within the DDCM-based methods?
Thank you for the follow-up and for the positive assessment. We believe that our strategy opens a path to making other diffusion-based codecs controllable.
A direct transfer to trained codecs such as PerCo [1], or to post-processing codecs, is not trivial. Prop. 1 and the tower property (Eq. 12) assume Gaussian noise atoms and a denoiser that is the posterior-mean (Tweedie) estimator, whereas a learned codebook is not Gaussian and a jointly trained U-Net may deviate from that estimator.
The strategy itself is not entirely tied to only DDCM: rank the encoder's discrete choices by the optimal linear score of a chosen objective, obtained from one gradient on the denoised estimate. We believe it can be leveraged to design post-processing diffusion-based codecs with per-image control of the objective, a promising direction beyond this paper.
[1] Careil, Marlene, et al. "Towards image compression with perfect realism at ultra-low bitrates." International conference on learning representations. Vol. 2024. 2024.
We thank the reviewer for the constructive comments. Both approximation steps of Appendix C are now measured (new appendix section E).
Notation. Next state xt−1=μ+σtz (μ: step mean, z: injected noise). x^0∣s: denoised estimate at step s, Jacobian Js. c(z): loss gradient ∇L at the next estimate x^0∣t−1(μ+σtz); cˉ: ∇L at the average next estimate. E: average over z.
W1: the approximation chain, measured
Appendix C goes from the ideal coefficient u⋆ (Prop. 1) to the deployed gradient gt (Eq. 13) through the product of averages g~:
u⋆=σtE[Jt−1⊤c(z)]
g~=σtE[Jt−1]⊤cˉ
We estimated u⋆ and g~ by Monte Carlo (32 to 256 samples of z) at every coded step of the deployed Kodak24 encodes. Mean cosines over the gradient steps and images; cells are 0.0128 / 0.0301 bpp; (a) and (b) are plain cosines, biased low.
Q1: LPIPS, DINOv2, Re-IQA. LOONA uses these terms only inside the deployed objectives above. There the cosine is 0.89 to 0.96, and the deployed target (Eq. 14) obtains 92 to 96% of the next-step loss reduction obtained with u⋆. Alone, which LOONA never uses, the cosine is 0.68 to 0.81, and −gt still obtains 71 to 82% of that reduction, against -1 to 16% for Turbo-DDCM's residual.
Q2: when does the product of averages fail? The error of (a) is exactly
u⋆−g~=σtCov(Jt−1⊤,c)+σtE[Jt−1]⊤(E[c]−cˉ)
Both terms vanish for affine L (constant c) and grow with the curvature of L times the spread of the next estimate over candidates: (a) is 0.98 to 0.99 for Lpixel and 0.91 to 0.97 with the DINOv2 or Re-IQA term.
End to end. Re-encoding with a Monte Carlo u⋆ in place of gt (Kodak24, 3 seeds, same bits). Change against deployed over the three objectives and both rates (LPIPS: lower is better):
One backward pass loses at most 0.14 dB, 0.009 MS-SSIM and 0.008 LPIPS (for Lpixel: 0.10 / 0.07 dB, against a gain of 0.58 / 0.37 dB over Turbo-DDCM). Neither a one-sample estimate of u⋆ nor the Taylor estimator (Table 8) is better than gt. Section E contains more exp.
W2: hyperparameter sensitivity
Tuning is not what produces the gain. In new experiments we replaced the tuned Table 1 by one untuned rule for all rates. PSNR BD-rate against Turbo-DDCM, 3 seeds (Kodak24 / DIV2K-val-100): -18.5% / -17.1% for Table 1, -20.1% / -18.4% for Ng = half of the coded steps, -16.0% / -14.8% for one start step (Tg=40) at every rate.
DIV2K-val-100 was never used for tuning (development: DIV2K-train crops, six Kodak images).