acceptodds
Under review as a conference paper at ICLR 2027

WideR: Rate-adaptive Balancing for Wide-range Rate-Distortion-Perception Control in Neural Image Compression

Abstract

Neural image compression has advanced both in reducing distortion and improving perceptual quality, yet improving one can compromise the other at a fixed bitrate. Distortion-perception (D-P) controllable codecs address this trade-off by adjusting the balance between them within a single codec. Existing methods commonly train the codec for rate-distortion performance and then freeze the encoder while adapting the decoder to other preferences. Such adaptation changes how the encoded information is used, but cannot change what is encoded. Our study reveals that distortion-oriented representations incur a perception penalty that grows toward lower rates, whereas perception-oriented representations incur a distortion penalty that grows toward higher rates, even after decoder adaptation. To alleviate these cross-endpoint penalties, we propose rate-adaptive D-P objective balancing, which optimizes the codec with both reconstruction objectives and progressively shifts their relative weighting from perception to distortion as rate increases. We instantiate our method in WideR, a rate-distortion-perception controllable codec with a pretrained one-step diffusion prior, variable-rate coding, and a decoder-side D-P control mechanism. Across diverse datasets, WideR demonstrates broader D-P control than EGIC, the strongest baseline among the compared methods, achieving BD-rate gains of up to -60.0% in FID for perception-oriented reconstruction and -5.6% in PSNR for distortion-oriented reconstruction. It also achieves BD-rate gains of up to -20.3% in rate-PSNR at matched FID and -70.6% in rate-FID at matched PSNR. Code will be released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
neural image compression, diffusion-based image compression, rate-distortion-perception control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PVAbOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WideR, a neural image compression framework that supports a wide bitrate range and distortion-perception trade-off control. Through rate-adaptive loss weights, the authors emphasize perceptual quality at low bitrates and distortion optimization at high bitrates. Combined with a one-step diffusion prior and decoder-side LoRA interpolation, this enables reconstruction preference control using the same bitstream.

Critical strengths
  1. A single model supports bitrate control over a wide range of approximately 0.02-1.0 bpp, along with decoder-side distortion-perception adjustment, which has some practical value.
  2. The experiments are relatively extensive, including endpoint performance, comparisons at matched PSNR/FID, and ablation studies.
Critical weaknesses
  1. The core weight schedule remains relatively empirical, lacking sensitivity analyses of the boundary weights and scheduling forms. The overall approach is very simple.
  2. The fixed-weight-ratio ablation only uses a ratio of 4 without separate tuning. The validation of the advantages of adaptive scheduling is too limited. This is the core method.
  3. The conclusion about representation limitations relies on an experimental setting where the pretrained backbone is frozen and only modules such as LoRA are trained. This is insufficient, and I need to see more justification, especially since full fine-tuning is not performed.
  4. The model is large, with 864M total parameters, significantly more than EGIC's 33M. The decoding latency with the diffusion branch enabled is 179 ms, also higher than EGIC's 108 ms. This cost is too high.
  5. More comparisons should be provided on LPIPS and DISTS.
Questions for authors
  1. Do the advantages of adaptive scheduling remain stable when changing the boundary weights, adopting other scheduling forms, or thoroughly tuning the fixed weight ratio? I think these experiments can be added quickly.
  2. Do the observed cross-endpoint performance gaps still exist after increasing the LoRA rank or fully fine-tuning the decoder?
  3. Could you provide peak GPU memory usage and decoding latency for high-resolution images to help us evaluate the actual deployment overhead?
  4. Is the FID BD-rate as high as 60%? Is there any performance loss in certain cases?
  5. How about LPIPS and DISTS RD performance?
5 comments
AuthorsOpenReview ↗

Response to Reviewer PVAb [1/2]

We greatly appreciate your insightful and detailed feedback. Your comments have been carefully discussed and fully taken into account, and we improve our work accordingly.

Addressing Weaknesses

1. Sensitivity Analyses and Simplicity

Thank you for these important points. We add sensitivity analyses of the boundary weights and the interpolation function (we now use this term instead of "schedule" to avoid confusion) in the new Tables 3–4. The core of our method is the monotonic increase of the distortion-to-perception weight ratio with rate (Eq. 3), and the geometric interpolation of Eq. 4 is one natural instance. To show that its advantage is not sensitive to the choice of the other hyperparameters, both analyses keep this property and vary only the other choices. For the boundary weights, we fix λPlo\lambda_P^{lo} and λDhi\lambda_D^{hi}, which make the perception term dominant at the lowest rate and the distortion term at the highest rate and let a single model span a wide rate range. We then vary λDlo\lambda_D^{lo} or λPhi\lambda_P^{hi}, the weights of the opposite loss at each end, over several values (Table 3).

CLIC 2020vs. paper's settingvs. EGIC
λDlo\lambda_D^{lo}λDhi\lambda_D^{hi}λPlo\lambda_P^{lo}λPhi\lambda_P^{hi}PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting2⁻⁹2¹¹2⁻⁴2²0.0%0.0%−3.1%−60.0%
2⁻⁸2¹¹2⁻⁴2²+0.7%−0.7%−1.7%−60.6%
2⁻¹⁰2¹¹2⁻⁴2²+1.4%−2.7%−1.7%−61.8%
2⁻⁹2¹¹2⁻⁴2³+4.0%−7.5%+1.3%−63.4%
2⁻⁹2¹¹2⁻⁴2¹−0.7%+5.3%−1.8%−54.3%

Changing the boundary weights alters the overall D–P balance and shifts the operating points, i.e., one endpoint improves while the other degrades. However, these shifts are small compared with the advantage of WideR over EGIC. Thus, as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights, and every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

To see how the results change with the shape of the interpolation function, we replace qq in Eq. 4 with qβq^β, which makes the log of the weight ratio linear (β = 1), convex (β = 2), or concave (β = 0.5) in qq (Table 4).

CLIC 2020vs. paper's settingvs. EGIC
InterpolationPSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting (β = 1)0.0%0.0%−3.1%−60.0%
β = 0.5+6.1%+8.9%−3.2%−62.5%
β = 2+0.8%−9.6%+1.4%−57.6%

Changing β also shifts the operating points and thus changes the BD-rates against the paper's setting. However, the BD-rates against EGIC show a different trend, because β shifts the operating points mainly at the lowest rates of WideR, which EGIC does not cover. Over the full rate range, β = 1 shows the most balanced performance between the two endpoints (Figure 11b in Appendix D). Every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

These two analyses show that as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights or the interpolation function. We believe the simplicity of our approach is a strength, as it adds no parameters or inference cost yet substantially reduces the cross-endpoint penalties (Figure 4).

2. Fixed-Weight-Ratio Ablation

The purpose of this experiment is to show that jointly optimizing the D and P objectives alone is not sufficient, and that changing their balance with rate is necessary. The ratio of 4 is not a tuned value but the midpoint (q = 0.5) of our rate-adaptive ratio, i.e., the geometric mean of λD(q)/λP(q)\lambda_D(q)/\lambda_P(q) under the uniform sampling of qq (Appendix C). Thus, the fixed ratio and the rate-adaptive ratio have the same average balance and differ only in whether the ratio changes with rate. Even in this matched comparison, rate-adaptive balancing (ours) achieves a −18.1% PSNR BD-rate at matched FID and a −84.3% FID BD-rate at matched PSNR (Section 5.4). Nevertheless, we are training two more fixed ratios, 2−1.52^{-1.5} and 25.52^{5.5}, which equal the rate-adaptive ratio at q = 0.25 and 0.75. Their training is not expected to end within the rebuttal period, so we will add the results to Section 5.4 and Appendix C in the final version. Given the large gaps of the fixed ratio of 4, we strongly expect the same conclusion.

3. Representation Limitations with a Frozen Backbone

We answer this point in Question 2.

4. Model Size and Decoding Latency

We answer this point in Question 3.

5. Comparisons on LPIPS and DISTS

We answer this point in Question 5.

AuthorsOpenReview ↗

Response to Reviewer PVAb [2/2]

Response to Questions

1. Stability of the Advantages of Rate-Adaptive Balancing

[Discussed in Weaknesses 1–2] As long as the monotonic increase of the weight ratio is kept, the advantage of WideR is not sensitive to the boundary weights or the interpolation function (Tables 3–4). In addition, the fixed ratio of 4, the midpoint of our rate-adaptive ratio, is clearly outperformed by rate-adaptive balancing (Section 5.4).

2. Fully Fine-Tuning the Decoder

The backbone is frozen identically in all experiments of DoR, PoR, and WideR to preserve the prior knowledge of the perceptually pretrained VAE decoder. This is a controlled experiment, in which the three models differ only in their objectives. To further examine the representation capacity in a more extreme setting, as you suggested, we add an experiment that fully fine-tunes the VAE decoder during the training of DoR and PoR (the new Table 13, Appendix M). On Kodak24, full fine-tuning does not reduce the perception penalty of DoR (+2.3%) and reduces the distortion penalty of PoR by only 14.8%, whereas rate-adaptive balancing (ours) reduces them by 68.0% and 90.3% with the decoder frozen. Therefore, the cross-endpoint penalties do not come from the frozen backbone but from the training objective.

Kodak24Perception penalty (ΔFID) ↓Distortion penalty (ΔPSNR, dB) ↓
DoR9.64–
DoR (w/ VAE decoder full fine-tuning)9.86 (+2.3%)–
PoR–1.74
PoR (w/ VAE decoder full fine-tuning)–1.48 (−14.8%)
Rate-adaptive balancing3.08 (−68.0%)0.17 (−90.3%)

3. Peak GPU Memory Usage and Decoding Latency for High-Resolution Images

We agree that deployment feasibility at high resolutions is important. We add the peak GPU memory and encoding and decoding latency at the DIV2K resolution (2040×1356) to the new Appendix N (Table 14) and report them below.

2040×1356 (DIV2K)Latency, enc. / dec. ↓Peak memory, enc. / dec. ↓
EGIC680 / 853 ms4.2 / 8.2 GB
AEIC-ME122 / 4,333 ms5.7 / 9.5 GB
OneDC1,573 / 6,998 ms13.2 / 16.9 GB
WideR322 / 4,641 ms5.9 / 9.6 GB

(All models decode the full image in a single pass without tiling.)

At this resolution, WideR decodes slower than EGIC (4.6 vs. 0.9 s) and requires more peak GPU memory in both the encoder and the decoder. The slower decoding mainly comes from the attention modules of the pretrained diffusion model, whose cost grows quadratically with the image size, and AEIC-ME and OneDC show a similar trend.

Please note that EGIC does not consider ultra-low rates and thus does not use a large generative prior. In contrast, WideR also targets ultra-low rates, where most details are smoothed out or lost in compression, and a large diffusion prior is effective in naturally generating such missing information. Generative codecs dedicated to ultra-low rates also use such a prior, and their complexity is similar to that of WideR (Table 5). At 2040×1356, AEIC-ME and OneDC also take several seconds to decode and need similar or more decoding memory than WideR, as shown above. Thus, we believe that the complexity of WideR is better compared with that of EGIC and an ultra-low-rate generative codec together, rather than with EGIC alone.

In addition, our rate-adaptive balancing itself does not depend on a specific diffusion prior. Thus, we believe that applying it to lightweight diffusion priors such as CoD-Lite (Jia et al., 2026), which are actively studied, is a promising next step. As this study does not cover such an extension, we leave it as a limitation in Section 6.

4. FID BD-Rate of 60% and Performance Loss in Certain Cases

Thank you for the question. Yes, −60.0% is the P-endpoint FID BD-rate of WideR against EGIC on CLIC 2020 (Table 1; −50.3% on Kodak24 and −51.7% on DIV2K). This gain is consistent with the curves and the visual examples in Figures 1, 16, and 17, which show no performance loss against EGIC. WideR also achieves a better D-endpoint PSNR BD-rate than EGIC on all three datasets (Table 1).

5. LPIPS and DISTS RD Performance

Thank you for the suggestion. Table 1 reports the P-endpoint BD-rates in LPIPS and DISTS on all three datasets, and Figure 16 (Appendix I) shows the rate–LPIPS and rate–DISTS curves of all compared codecs. However, our ablations and matched-quality comparisons (Table 2) use only PSNR and FID. We would be glad to add these results in LPIPS and DISTS if you find them helpful.

We hope these responses address your concerns. If any point remains unclear, we would be glad to provide further clarification or results during the discussion period.

Thank you for the additional experiments. The sensitivity analyses and full fine-tuning result address some of my concerns. However, the comparison still uses only a fixed weight ratio of 4, with additional results pending. And different fixed ratios may perform differently across bitrates, so a more thorough comparison is necessary to demonstrate the benefit of adaptive weighting. The high-resolution decoding cost also remains substantial. So I will keep my original rating for now.

Reviewer_gS6JOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WideR, a rate–distortion–perception controllable neural image compression framework that introduces rate-adaptive objective balancing to alleviate the limitations of distortion-oriented and perception-oriented compressed representations.

Critical strengths
  1. The paper identifies an important limitation of existing controllable codecs: optimizing the compressed representation for one reconstruction objective can restrict the attainable performance of the opposite objective, even after decoder adaptation. The analysis of distortion-oriented and perception-oriented representations provides a convincing motivation for jointly optimizing the shared representation.
  2. WideR demonstrates substantial improvements over existing controllable codecs on Kodak24, CLIC 2020, and DIV2K. In particular, the reported BD-rate reductions of up to 60.0% in FID and 5.6% in PSNR against EGIC indicate that the proposed approach can improve both perceptual and distortion-oriented reconstruction within a shared controllable framework.
  3. The paper includes comparisons with RDP-controllable codecs and diffusion-based generative codecs, together with controlled studies on objective-specific representations, fixed versus rate-adaptive balancing, decoder-side control, and computational complexity. These experiments provide useful evidence for the proposed design and help distinguish the contribution of objective balancing from other architectural components.
Critical weaknesses
  1. The central contribution is a rate-dependent adjustment of the relative weights assigned to distortion and perceptual losses, implemented through geometric interpolation between empirically selected boundary values. While the paper provides limited theoretical justification for why the proposed scheduling form should be preferable to alternative adaptive balancing strategies.
  2. Although a fixed-ratio baseline is evaluated, the experiments do not establish how sensitive performance is to alternative schedules, boundary values, or rate distributions. Consequently, the robustness and transferability of the proposed strategy remain uncertain.
  3. The paper uses FID as a primary measure of perceptual quality and computes it from overlapping image patches. Although the authors provide a sensitivity analysis for patch sampling in Appendix G, the reported FID-based BD-rate improvements remain dependent on the evaluation protocol and the available overlapping rate ranges.
  4. WideR requires a large pretrained diffusion-based architecture, with approximately 864M parameters and decoding latency of 179 ms per image when perceptual enhancement is enabled. Although the proposed single-model design reduces the need to maintain multiple rate-specific checkpoints, its computational and memory requirements remain considerably higher than those of lightweight controllable codecs such as EGIC.
Questions for authors
  1. Could the authors further explain why geometric interpolation of distortion and perception loss weights is particularly suitable for rate-adaptive RDP optimization? Is there any theoretical or empirical evidence?
  2. How sensitive is WideR to the four empirically selected boundary weights and the distribution used to sample rate-control parameters during training?
  3. Could the authors provide additional perceptual assessments, such as human preference evaluations or reconstruction-fidelity analyses?
  4. Could the authors provide a more detailed comparison of model storage, peak GPU memory, encoding and decoding latency, and reconstruction quality under matched computational budgets?
3 comments
AuthorsOpenReview ↗

Response to Reviewer gS6J [1/2]

We greatly appreciate your insightful and detailed feedback. Your comments have been carefully discussed and fully taken into account, and we improve our work accordingly.

Addressing Weaknesses

1–2. Rationale of the Boundary Weights and Interpolation Function and Their Sensitivity

Thank you for these important points, which we address together. Our rationale is based on the analysis of the cross-endpoint penalties, which grow toward opposite ends of the rate range (Figures 3–4). Hence, the core of our method is the monotonic increase of the distortion-to-perception weight ratio with rate (Eq. 3), and the geometric interpolation of Eq. 4 is one natural instance. We set λPlo\lambda_P^{lo} and λDhi\lambda_D^{hi} relatively high so that the perception term dominates at the lowest rate and the distortion term at the highest rate, and their scales relative to the rate term let a single model span a wide rate range. We set λDlo\lambda_D^{lo} and λPhi\lambda_P^{hi} relatively low so that the opposite endpoint is also learned at each end.

We add sensitivity analyses of the boundary weights and the interpolation function (we now use this term instead of "schedule" to avoid confusion) in the new Tables 3–4. To show that the advantage of this monotonic increase is not sensitive to the choice of the other hyperparameters, both analyses keep it and vary only the other choices. For the boundary weights, we fix λPlo\lambda_P^{lo} and λDhi\lambda_D^{hi} and vary λDlo\lambda_D^{lo} or λPhi\lambda_P^{hi}, the weights of the opposite loss at each end, over several values (Table 3).

CLIC 2020vs. paper's settingvs. EGIC
λDlo\lambda_D^{lo}λDhi\lambda_D^{hi}λPlo\lambda_P^{lo}λPhi\lambda_P^{hi}PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting2⁻⁹2¹¹2⁻⁴2²0.0%0.0%−3.1%−60.0%
2⁻⁸2¹¹2⁻⁴2²+0.7%−0.7%−1.7%−60.6%
2⁻¹⁰2¹¹2⁻⁴2²+1.4%−2.7%−1.7%−61.8%
2⁻⁹2¹¹2⁻⁴2³+4.0%−7.5%+1.3%−63.4%
2⁻⁹2¹¹2⁻⁴2¹−0.7%+5.3%−1.8%−54.3%

Changing the boundary weights alters the overall D–P balance and shifts the operating points, i.e., one endpoint improves while the other degrades. However, these shifts are small compared with the advantage of WideR over EGIC. Thus, as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights, and every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

To see how the results change with the shape of the interpolation function, we replace qq in Eq. 4 with qβq^β, which makes the log of the weight ratio linear (β = 1), convex (β = 2), or concave (β = 0.5) in qq (Table 4).

CLIC 2020vs. paper's settingvs. EGIC
InterpolationPSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting (β = 1)0.0%0.0%−3.1%−60.0%
β = 0.5+6.1%+8.9%−3.2%−62.5%
β = 2+0.8%−9.6%+1.4%−57.6%

Changing β also shifts the operating points and thus changes the BD-rates against the paper's setting. However, the BD-rates against EGIC show a different trend, because β shifts the operating points mainly at the lowest rates of WideR, which EGIC does not cover. Over the full rate range, β = 1 shows the most balanced performance between the two endpoints (Figure 11b in Appendix D). Every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

These two analyses show that as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights or the interpolation function.

3. Evaluation Protocol and Rate Ranges

To compute FID stably on small datasets, we use the patch-based protocol of HiFiC (Mentzer et al., 2020), which many later generative codecs also follow. We also follow this convention rather than a protocol that favors WideR, and additionally report LPIPS and DISTS (Table 1).

We compute each BD-rate over the available overlapping rate range of any two codecs. A complete comparison could be performed over identical operating ranges, but the compared codecs cover limited ranges and arbitrarily extrapolating their curves is discouraged. Please note that WideR covers a sufficiently wide range, so this constraint mainly comes from the compared codecs.

4. Computational and Memory Requirements

We answer this point in Question 4.

AuthorsOpenReview ↗

Response to Reviewer gS6J [2/2]

Response to Questions

1. Why Geometric Interpolation Is Suitable for Rate-Adaptive RDP Optimization

[Discussed in Weaknesses 1–2] We choose the geometric interpolation of Eq. 4 because learned codecs usually increase λ exponentially to target different rates. The core of our method is the monotonic increase of the weight ratio with rate, and this interpolation is one natural instance of it. In addition, we add a sensitivity analysis on the shape of the interpolation function (Table 4).

2. Sensitivity to the Four Boundary Weights and the Distribution of Rate-Control Parameters

[Discussed in Weaknesses 1–2] As long as the monotonic increase of the weight ratio is kept, the advantage of WideR is not sensitive to the boundary weights or the interpolation function (Tables 3–4). Regarding the rate distribution, we simply sample qq uniformly over 64 levels as in CRDR (Iwai et al., 2024). That is, we use no special sampling distribution.

3. Human Preference Evaluations

We agree that a human preference evaluation would further strengthen our results. Nevertheless, we believe that the current results sufficiently support our main claims. FID, which measures realism, and LPIPS and DISTS, which are designed to match human judgments, consistently favor WideR against EGIC on all three datasets (Table 1). As a user study is not feasible within the rebuttal period, we will conduct it for the final version and analyze its correlation with these metrics.

4. Detailed Comparison of Storage, Peak GPU Memory, and Latency

We provide a more detailed comparison of model storage, peak GPU memory, and encoding and decoding latency on Kodak24 below (the new Tables 14–15 in Appendix N).

CodecComponentParamsLatencyPeak memory
EGICEncoder23 M120 ms0.8 GB
Decoder20 M108 ms1.3 GB
WideREncoder57 M44 ms4.1 GB
Latent decoder55 M40 ms
U-Net789 M81 ms
VAE decoder16 M58 ms
Decoder (total)860 M179 ms4.5 GB

WideR requires more storage and peak GPU memory than EGIC in both the encoder and the decoder. However, WideR encodes faster than EGIC (44 vs. 120 ms), and its latent decoder and VAE decoder together also decode faster than the EGIC decoder (98 vs. 108 ms). The U-Net holds most of the parameters (789 M) and adds 81 ms when the perception branch is enabled, which is the main reason WideR decodes slower than EGIC.

Please note that EGIC does not consider ultra-low rates and thus does not use a large generative prior. In contrast, WideR also targets ultra-low rates, where most details are smoothed out or lost in compression, and a large diffusion prior is effective in naturally generating such missing information. As a result, WideR achieves a P-endpoint FID BD-rate of −50.3% to −60.0% against EGIC (Table 1). Generative codecs dedicated to ultra-low rates also use such a prior, and their complexity is similar to that of WideR, as shown below (Table 5).

D–P controlParamsLatency, enc. / dec.
AEIC-MEP926 M18 / 160 ms
OneDCP1406 M117 / 311 ms
WideRD/P864 M44 / 179 ms

Thus, we believe that the complexity of WideR is better compared with that of EGIC and an ultra-low-rate generative codec together, rather than with EGIC alone.

In addition, we showed that a simple hard routing can omit deeper U-Net blocks at mid-to-high rates (Section 5.5 and Appendix F). At these rates, reducing the U-Net depth barely changes the performance (BD-rates within 1.35%) while reducing the U-Net latency by up to 47.5%. We believe that such rate-adaptive complexity is worth further investigation.

Also, please note that our rate-adaptive balancing itself does not depend on a specific diffusion prior. Thus, we believe that applying it to lightweight diffusion priors such as CoD-Lite (Jia et al., 2026) is a promising next step. As this study does not cover such an extension, we leave it as a limitation in Section 6.

We hope these responses address your concerns. If any point remains unclear, we would be glad to provide further clarification or results during the discussion period.

Thank you for the additional experiments. The sensitivity analyses and fixed-ratio comparisons address some of my concerns. However, I still find the core idea of adjusting loss weights rather incremental, and the fixed-ratio results are preliminary. Also, WideR uses a much larger diffusion backbone than EGIC (860M vs. 20M parameters), so it is hard to tell how much of the improvement comes from the proposed method. The decoding cost also remains high. I will keep my original rating.

Reviewer_4caBOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To achieve wide-range RDP control using a codec built on a pretrained model, the paper empirically identifies rate-dependent cross-endpoint penalties that cannot be eliminated through decoder-only adaptation. Then the paper introduces a simple geometric weighting schedule that changes the relative distortion/perception training loss weights, and develops a multi-stage training procedure. The overall intuition is clear, and the experiments support the effectiveness of the proposed scheme.

Critical strengths
  1. Clear motivation and insight into the RDP traversal problem. I appreciate the experiments on cross-endpoint distortion/perception penalties. The authors experimentally observe that the perception penalty of DoR increases toward lower rates, whereas the distortion penalty of PoR increases toward higher rates, and training a decoder for the opposite objective cannot fully overcome this. Thus, the design of rate-adaptive DP balancing is well-supported.

  2. Extensive experiments and ablation studies. The authors report the RDP performance with different metrics, e.g., BD-rate, R-P/R-D curves at matched D/P, and model complexity. They also train different variants to identify the improvements.

Critical weaknesses
  1. Important baselines are missing, and several statements need to be modified. For the diffusion-based generative codecs, DiffC (Theis et al., 2022; Vonderfecht and Liu, 2025) is another line of methods utilizing pre-trained diffusion (e.g., Stable Diffusion and Flux) to perform compression. Recent work (Wang et al., 2026) also shows that by manipulating the diffusion trajectories, DiffC can traverse RDP without re-training or fine-tuning. Since their scheme also uses one parameter tt in the encoder to control rate and uses parameter ρ\rho in the decoder to control DP, this formulation is conceptually related to WideR's separate rate and D–P control mechanisms. Unlike WideR, this approach does not require optimizing a pretrained diffusion model using a rate-adaptive D–P training objective. I suggest the authors compare with this line of work in terms of RDP performance, model complexity, and computational cost. Meanwhile, the statement that the proposed first RDP-controllable codec that covers a wide RDP range with a single trained model need to be re-evaluated.

  2. The visualization and evaluation of RDP controllability could be improved. The DP traversal plot in Figure 1 is misleading. It seems that the shaded region in Figure 1 is not an actual DP tradeoff curve, since it connects rate-distortion curves and rate-perception curves. Their numerical separation is meaningless given different axis scales. To illustrate the ability of RDP traversal, the authors should draw different R-D and R-P curves with different \alpha, then check how the R-D or R-P sweeps with α\alpha. Meanwhile, I suggest the authors label  alpha\ alpha values around the curves for clearer comparison.

Theis et al., “Lossy Compression with Gaussian Diffusion,” arXiv, 2022

Vonderfecht and Liu, “Lossy Compression with Pretrained Diffusion Models,” ICLR, 2025

Wang et al., “Training-Free Rate-Distortion-Perception Traversal With Diffusion,” ICML, 2026

Questions for authors

Is the training performance sensitive to the hyperparameters λD(q)\lambda_D(q) and λP(q)\lambda_P(q)? In practice, how to choose the weight in the training loss, and how to choose α\alpha for target DP operation point in inference? I’m afraid the calibration may take effort and limit practical deployment.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4caB [1/2]

We greatly appreciate your insightful and detailed feedback. Your comments have been carefully discussed and fully taken into account, and we improve our work accordingly.

Addressing Weaknesses

1. Missing Baseline (Wang et al.)

Thank you for pointing out this important work. Wang et al. (2026) is similar to our work in that it is an RDP-controllable coding framework built on a pretrained diffusion model. However, while our work targets RDP control over a wide range, Wang et al. focuses on establishing that a pretrained diffusion model enables RDP control without any additional training. We add a detailed comparison and analysis below. We reimplement Wang et al. on the official DiffC code (Vonderfecht & Liu, 2025) (the code of Wang et al. is not public).

First, the revised Figure 1 with Wang et al. added shows that Wang et al. saturates at about 0.26 bpp on Kodak24. It raises the rate only by transmitting more steps of the pretrained diffusion process, and even transmitting all steps of the official encoding schedule gives about 0.26 bpp, where its PSNR has already almost converged. Its D–P control range is also very narrow. Even at 0.26 bpp, its D-endpoint PSNR is 26.5 dB, while the D endpoint of WideR reaches about 31.5 dB at the same rate. Since Wang et al. uses a pretrained diffusion model without retraining, its PSNR is inherently bounded by the pretrained autoencoder (26.7 dB on Kodak24 even without compression).

The tables below show a detailed performance comparison on Kodak24.

PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓LPIPS$_\mathrm{P}$ ↓DISTS$_\mathrm{P}$ ↓
vs. Wang et al.−46.6%−18.0%+20.5%−19.0%
PSNR BD-rate ↓FID BD-rate ↓
(matched FID)(matched PSNR)
vs. Wang et al.−40.6%N/A*

(*Not available because WideR's P endpoint already has a higher PSNR than Wang et al.'s D endpoint at every common rate.)

For the P-endpoint performance, Wang et al. is worse than WideR in FID and DISTS but better in LPIPS. Its better LPIPS seems to come from the LPIPS loss used to train the pretrained autoencoder, while WideR uses DISTS instead of LPIPS in its final training stage. In FID, which neither method uses as a loss, WideR performs better.

For computational cost, although Wang et al. runs in FP16 precision as in the official DiffC code, WideR encodes over 250× and decodes over 40× faster with fewer parameters. This is because Wang et al. searches over many candidate samples at each diffusion step for every input image and then decodes through multi-step diffusion.

Parameters ↓Encoding ↓Decoding ↓Peak decoding memory ↓
Wang et al.950 M12,128 ms7,838 ms2.9 GB
WideR864 M44 ms179 ms (α > 0)4.5 GB

Overall, WideR provides a much wider D–P control range than Wang et al., especially at the D endpoint. It also covers a wider rate range and runs far faster in both encoding and decoding. Nevertheless, Wang et al. needs no training of its own and achieves a strong P endpoint at low rates, so we regard it as a valuable baseline. We add Wang et al. to Figure 1 and Table 1 (endpoint performance), the new Figure 5 (D–P traversal), and Table 5 (complexity).

2. Contribution Statement

We agree that the original statement is too strong, since the method of Wang et al. also covers many rates with one pretrained model. Thus, we revise it in Section 1 to

"WideR demonstrates that a single trained RDP-controllable codec can cover both the conventional bitrate range (0.1–1.0 bpp) of the controllable codecs and the ultra-low-rate range (≈0.02 bpp) of the generative realism codecs."

3. Visualization of RDP Controllability in Figure 1

We fully agree that the shaded region in Figure 1 connected rate–distortion and rate–perception curves with different axis scales and was not an actual DP tradeoff curve. Accordingly, we separate Figure 1 into rate–PSNR and rate–FID panels and show both the D and P endpoints in each panel, so that the shaded range of WideR lies on a single metric axis. In addition, as you suggested, we add Figure 5 (Section 5.2), which shows the D–P traversals with α from 0 to 1 in steps of 0.1.

AuthorsOpenReview ↗

Response to Reviewer 4caB [2/2]

Response to Questions

1. Sensitivity of the Training Performance and Choice of the Weights

Thank you for this important question. We add sensitivity analyses of the boundary weights and the interpolation function (we now use this term instead of "schedule" to avoid confusion) in the new Tables 3–4. The core of our method is the monotonic increase of the distortion-to-perception weight ratio with rate (Eq. 3). To show that its advantage is not sensitive to the choice of the other hyperparameters, both analyses keep this property and vary only the other choices. For the boundary weights, we fix λPlo\lambda_P^{lo} and λDhi\lambda_D^{hi}, which make the perception term dominant at the lowest rate and the distortion term at the highest rate and let a single model span a wide rate range. We then vary λDlo\lambda_D^{lo} or λPhi\lambda_P^{hi}, the weights of the opposite loss at each end, over several values (Table 3).

CLIC 2020vs. paper's settingvs. EGIC
λDlo\lambda_D^{lo}λDhi\lambda_D^{hi}λPlo\lambda_P^{lo}λPhi\lambda_P^{hi}PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting2⁻⁹2¹¹2⁻⁴2²0.0%0.0%−3.1%−60.0%
2⁻⁸2¹¹2⁻⁴2²+0.7%−0.7%−1.7%−60.6%
2⁻¹⁰2¹¹2⁻⁴2²+1.4%−2.7%−1.7%−61.8%
2⁻⁹2¹¹2⁻⁴2³+4.0%−7.5%+1.3%−63.4%
2⁻⁹2¹¹2⁻⁴2¹−0.7%+5.3%−1.8%−54.3%

Changing the boundary weights alters the overall D–P balance and shifts the operating points, i.e., one endpoint improves while the other degrades. However, these shifts are small compared with the advantage of WideR over EGIC. Thus, as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights, and every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

For the interpolation function, we use the geometric interpolation of Eq. 4. To see how the results change with the shape of this function, we replace qq in Eq. 4 with qβq^β, which makes the log of the weight ratio linear (β = 1), convex (β = 2), or concave (β = 0.5) in qq (Table 4).

CLIC 2020vs. paper's settingvs. EGIC
InterpolationPSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓PSNR$_\mathrm{D}$ ↓FID$_\mathrm{P}$ ↓
Paper's setting (β = 1)0.0%0.0%−3.1%−60.0%
β = 0.5+6.1%+8.9%−3.2%−62.5%
β = 2+0.8%−9.6%+1.4%−57.6%

Changing β also shifts the operating points and thus changes the BD-rates against the paper's setting. However, the BD-rates against EGIC show a different trend, because β shifts the operating points mainly at the lowest rates of WideR, which EGIC does not cover. Over the full rate range, β = 1 shows the most balanced performance between the two endpoints (Figure 11b in Appendix D). Every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance.

These two analyses show that as long as the monotonic increase is kept, the advantage of WideR is not sensitive to the boundary weights or the interpolation function.

2. Choice of α for a Target DP Operation Point

As a guide for choosing α, Figure 5 shows how the D–P balance of WideR changes with α at three fixed rates. PSNR changes almost linearly with α, whereas FID decreases rapidly even with a small α. Since these trends are similar across rates, we expect that users can choose an α suitable for their deployment and obtain a consistent D–P balance across rates without additional per-rate effort.

We hope these responses address your concerns. If any point remains unclear, we would be glad to provide further clarification or results during the discussion period.

Thank you for the detailed rebuttal. The new experiments comparing with Wang et al. (2026) seem promising: the proposed method achieves a much wider RDP region and better performance. The sensitivity analysis also strengthens the paper. The rebuttal addresses my major concerns, and I intend to increase my score to 3.

We sincerely appreciate your careful reassessment and your decision to raise the score.

We are especially glad that you find the sensitivity analysis strengthens the paper, as it was the concern shared by all three reviewers.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Apology for the Late Response

Dear Reviewers,

We apologize for the delay in posting our responses. Thanks to the insightful feedback from reviewers with deep expertise in this area, we were able to substantially improve our paper within a short period. In the revised paper, we make our key contribution more concrete, add experiments that support it more clearly, and summarize our position on the complexity concerns.

As our late posting leaves only a short time before the discussion period closes, we would greatly appreciate it if you could let us know whether our responses address your concerns. We would also be glad to answer any further questions in the remaining time.

Thank you again for your time and thoughtful reviews.

Best regards,

The Authors

AuthorsOpenReview ↗

Report to the Area Chair (AC) [1/2]

Dear Area Chair,

Thank you for handling our submission. We briefly summarize the strengths acknowledged by the reviewers and the main concerns with our responses.


Strengths Commented by the Reviewers


1. A New Perspective on Controllable Codecs (Reviewers 4caB and gS6J)

It was commented that our analysis of the cross-endpoint penalties gives "clear motivation and insight into the RDP traversal problem" and that "the design of rate-adaptive DP balancing is well-supported" (Reviewer 4caB). It was also commented that the paper "identifies an important limitation of existing controllable codecs" and that the analysis "provides a convincing motivation for jointly optimizing the shared representation" (Reviewer gS6J). Our controlled comparison showed that the training strategies of existing controllable codecs cause cross-endpoint penalties that grow toward opposite ends of the rate range (Section 4). Based on this finding, rate-adaptive balancing reduces both penalties across all rates by changing only the training objective.

2. Clear Gains over Existing Codecs (Reviewer gS6J)

It was commented that WideR "demonstrates substantial improvements over existing controllable codecs on Kodak24, CLIC 2020, and DIV2K," with BD-rate reductions of up to 60.0% in FID and 5.6% in PSNR against EGIC. These gains extend to ultra-low rates. Against generative codecs dedicated to ultra-low rates such as AEIC-ME and OneDC, which train a separate model for each rate, WideR reconstructs visually similar images with higher PSNR and slightly lower perceptual scores (Section 5.3 and Figures 6–7).

3. Practical Value (Reviewer PVAb)

It was commented that a single model supporting about 0.02–1.0 bpp with decoder-side D–P adjustment "has some practical value". In particular, a single WideR model can take on the role of both a controllable codec for conventional rates and a generative codec dedicated to ultra-low rates.

4. Extensive and Solid Experiments (Reviewers 4caB, gS6J, and PVAb)

The breadth of our experiments was acknowledged by all three reviewers. It was commented that we report the RDP performance with various metrics, including BD-rates, R–P/R–D curves at matched D/P, and model complexity, and train different variants to identify the improvements (Reviewer 4caB). It was also commented that our comparisons with controllable and diffusion-based generative codecs and our controlled studies "help distinguish the contribution of objective balancing from other architectural components" (Reviewer gS6J). Finally, it was commented that "the experiments are relatively extensive, including endpoint performance, comparisons at matched PSNR/FID, and ablation studies" (Reviewer PVAb).

AuthorsOpenReview ↗

Report to the Area Chair (AC) [2/2]


Main Concerns and Our Responses


We summarize the main concerns raised by the reviewers. Concerns 1 and 4 were raised by two or more reviewers. Please note that they are not the all of the concerns.

1. Rationale and Sensitivity of the Weighting (Reviewers 4caB, gS6J, and PVAb)

W1. It was pointed out that the weighting is empirical and that its sensitivity to the boundary weights, the interpolation function, and the rate distribution was not established.

Brief summary of Response to W1. We explained that the core of our method is the monotonic increase of the distortion-to-perception weight ratio with rate, which follows from the cross-endpoint penalties growing toward opposite ends of the rate range (Figures 3–4). We added sensitivity analyses of the boundary weights and the interpolation function (Tables 3–4). As long as the ratio increases monotonically, every variant achieves a D-endpoint performance similar to EGIC and a much better P-endpoint performance. For the rate distribution, we simply sample q uniformly as in CRDR.

Reviewer comments.

  • Reviewer 4caB: "The sensitivity analysis also strengthens the paper." (rating raised 2 -> 3)
  • Reviewer PVAb: "The sensitivity analyses … address some of my concerns."
  • Reviewer gS6J: We are still awaiting the reviewer's reply.

2. Comparison with a Single Fixed Ratio (Reviewer PVAb)

W2. It was pointed out that the fixed-ratio ablation uses only a ratio of 4.

Brief summary of Response to W2. This ratio is the midpoint of our rate-adaptive ratio, so the two differ only in whether the ratio changes with rate, and rate-adaptive balancing outperforms it by a large margin (Section 5.4). We further compared three fixed ratios at the quartiles of q using the intermediate weights at 200k steps. Rate-adaptive balancing reduces both cross-endpoint penalties in every rate range, which none of the fixed ratios achieves. The final results will be added to Section 5.4 and Appendix C.

Reviewer comments.

  • Reviewer PVAb: We are still awaiting the reviewer's reply to our follow-up.

3. Frozen Backbone (Reviewer PVAb)

W3. It was pointed out that the cross-endpoint penalties might come from the frozen backbone.

Brief summary of Response to W3. We fully fine-tuned the VAE decoder during the training of DoR and PoR (Table 13). Full fine-tuning does not reduce the perception penalty of DoR (+2.3%) and reduces the distortion penalty of PoR by only 14.8%, whereas rate-adaptive balancing reduces them by 68.0% and 90.3% with the decoder frozen. Thus the penalties come from the training objective rather than the frozen backbone.

Reviewer comments.

  • Reviewer PVAb: "The … full fine-tuning result address some of my concerns."

4. Computational and Memory Cost (Reviewers gS6J and PVAb)

W4. It was pointed out that WideR requires more computation and memory than lightweight controllable codecs such as EGIC, especially at high resolutions.

Brief summary of Response to W4. We reported the parameters, latency, and peak memory of each component at two resolutions (Tables 14–15). This cost comes from the one-step diffusion backbone, and rate-adaptive balancing itself adds no inference cost. EGIC does not consider ultra-low rates and thus does not use a large generative prior. Generative codecs dedicated to ultra-low rates such as AEIC-ME and OneDC use such a prior and show a cost similar to that of WideR (Table 5 and Tables 14–15). In the revised manuscript, we removed the claim of outperforming existing codecs from the list of contributions and noted that the effect of the balancing itself is verified by the controlled experiments. We also noted in Section 6 that our balancing can be applied to lightweight diffusion codecs such as CoD-Lite.

In addition, the decoding cost of WideR is the cost of a single codec that also serves ultra-low rates, which removes the need to deploy separate codecs for ultra-low and conventional rates. Thus we believe that the cost of WideR is better compared with that of EGIC and an ultra-low-rate generative codec together, rather than with EGIC alone.

Reviewer comments.

  • Reviewer gS6J: We are still awaiting the reviewer's reply.
  • Reviewer PVAb: We are still awaiting the reviewer's reply to our follow-up.

We hope this summary helps your assessment. Thank you for your time and effort.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.