- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes context scaling for variable-rate and variable-complexity image compression. It adjusts autoregressive context steps and uses hyperprior information to allocate computation across images.
Critical strengths
- The problem is practical and worth studying.
- The image-dependent context gains are interesting.
- The experiments cover several datasets and recent baselines, with consistent improvements.
Critical weaknesses
- Changing AR context schedules is fairly straightforward. The novelty seems limited.
- The adaptive scheduler lacks comparisons with simple strategies, such as hyperprior-bitrate thresholding.
- MACs do not fully reflect the latency cost of sequential context modeling.
- The full complexity range actually relies on four separately trained models.
Questions for authors
- How does the scheduler compare with simple bitrate-based allocation?
- Why is non-uniform group merging better than uniform merging?
- Does the advantage remain under matched decoding latency?
5 comments
Response to Reviewer 2bSM - Part II
(c) Can the two dimensions be combined? Yes. As Fig. 2 suggests, parameter and context scaling are complementary. We also explored combining MoE-style capacity routing with ContextScale: MoE adjusts active transform capacity, while ContextScale adjusts conditional entropy-modeling computation.
Table 3. Exploratory comparison of capacity and context scaling.
| Metric / target budget | ContextScale-only (reference) | MoE-only | MoE + ContextScale |
|---|---|---|---|
| 450 kMAC/px · ΔBDR (pp) | 0.00 | +5.80 | +1.19 |
| 900 kMAC/px · ΔBDR (pp) | 0.00 | +6.74 | +0.86 |
| 1350 kMAC/px · ΔBDR (pp) | 0.00 | +6.28 | +1.35 |
| Complexity range (kMAC/px) | 353.98–1650.74 | 454.62–1520.31 | 408.50–1562.73 |
| Number of models | 4 | 1 | 1 |
| Complexity control | Context schedules | Active capacity / experts | Capacity + context schedules |
ΔBDR is relative to ContextScale-only at the corresponding budget (lower is better). The ContextScale-only range is the combined coverage of four models; each MoE variant uses one model.
Adding context scaling to MoE reduces the BD-rate gap by 4.61, 5.88, and 4.93 pp at the three budgets, respectively. Although this exploratory single-model combination still trails the four-model ContextScale-only reference by around one percentage point, it shows that context scaling can complement capacity routing rather than replace it.
2. W2 & Q1. Does the scheduler beat simple hyperprior-bitrate thresholding?
We compared Single-threshold (two schedules) and Multi-threshold (multiple schedules) policies calibrated using on held-out images, under matched average MAC budgets. While both baselines use the hyperprior bitrate, our scheduler estimates the expected R–D gain of each candidate schedule and weighs it against its computational cost.
Table 4. Threshold policy minus ContextScale (ours), BD-rate difference (pp).
| Dataset | Single-threshold | Multi-threshold |
|---|---|---|
| Kodak | +0.74 | +0.32 |
| CLIC | +0.61 | +0.24 |
| Tecnick | +0.56 | +0.12 |
| Avg. | +0.63 | +0.23 |
Positive differences favor ContextScale (ours).
Using multiple thresholds narrows the gap considerably, as several cutoffs offer more flexibility than a binary coarse/fine choice. ContextScale (ours) nevertheless retains an approximately 0.23-pp average advantage over Multi-threshold. This suggests that estimating the gain of each schedule, rather than only ranking images by , can allocate computation more effectively. The threshold results also reinforce our observation in Fig. 3 that hyperprior bitrate is a useful indicator of context-scaling gains.
3. Q2. Why use non-uniform rather than uniform group merging?
Our motivation comes from the unequal importance of context groups during autoregressive coding. When adjacent groups are merged, they are coded in parallel and can no longer condition on one another. Since early decoded groups provide context for many subsequent predictions, we preserve finer partitions in early coding stages and merge later groups more aggressively, rather than merging all groups uniformly. This design is consistent with the uneven distribution of latent information observed in ELIC [3] and the importance of informative early groups emphasized by CCA [4].
To verify this design choice, we trained uniform-merging variants under comparable settings and compared both strategies at matched computational complexity.
Table 5. BDR(Non-uniform) − BDR(Uniform), percentage points.
| Evaluated cost (kMAC/px) | Kodak | CLIC | Tecnick | Avg. |
|---|---|---|---|---|
| 385.98 | −0.73 | −1.55 | −1.63 | −1.30 |
| 471.13 | −1.39 | −2.94 | −3.07 | −2.47 |
| 556.27 | −1.44 | −2.55 | −2.96 | −2.32 |
| 641.41 | −0.88 | −1.54 | −1.80 | −1.41 |
| Avg. | −1.11 | −2.14 | −2.36 | −1.87 |
Negative values favor non-uniform merging.
Non-uniform merging consistently achieves lower BD-rate across all three datasets and four evaluated complexity levels, with an average improvement of 1.87 pp. These results support our design of preserving fine-grained conditional dependencies in early coding stages while reducing sequential computation through coarser grouping in later stages.
Response to Reviewer 2bSM - Part III
4. W3 & Q3. Does adaptation still help at matched decoding latency?
An important feature of ContextScale is that its adaptive scheduling framework is not tied to a specific complexity metric. Our method allocates context computation based on the expected R–D gain and computational cost of each candidate schedule. While we use kMACs/pixel in the main paper as a hardware-independent complexity measure, the same framework can accommodate other measurable or estimable costs, such as decoding latency, without changing the underlying scheduling principle.
To verify this, we conducted additional experiments using measured decoding latency as the complexity budget, including neural inference and CPU rANS coding at native image resolutions. We compare our adaptive scheduling strategy with fixed context schedules under matched average decoding latency.
Table 6. ContextScale (ours) versus fixed schedules at matched full decoding time.
| Dataset | Matched decoding time (ms) | BDR(Fixed) − BDR(ContextScale) (pp) |
|---|---|---|
| Kodak | 66.8 | +1.19 |
| Kodak | 71.2 | +1.08 |
| Kodak | 85.7 | +1.03 |
| Tecnick | 289.6 | +1.27 |
| CLIC | 310.9 | +1.30 |
| CLIC | 428.6 | +1.45 |
| Avg. | — | +1.22 |
At all six operating points, ContextScale achieves 1.03–1.45 pp lower BD-rate than fixed scheduling, with an average improvement of 1.22 pp (see Appendix D.2, Fig. 11). Importantly, these results demonstrate that the benefit of our adaptive context-scaling strategy is not limited to MAC-based complexity budgets. The same gain-aware allocation framework remains effective under measured latency constraints, further supporting its flexibility for different computational requirements.
Closing
To summarize, we believe ContextScale makes several distinct contributions. To our knowledge, it is the first unified variable-rate and variable-complexity LIC framework to achieve performance competitive with conventional state-of-the-art LIC methods. We introduce and systematically study context scaling as a new complexity-scaling dimension, and further investigate image-wise scaling behavior to develop an adaptive strategy that allocates computation according to input-dependent gains. Importantly, context scaling is complementary to parameter scaling, generalizes across different conditional entropy models (e.g., HPCM, CHARM, and Checkerboard), and is not restricted to a particular complexity metric, as demonstrated using both kMAC-based and measured-latency budgets.
We have incorporated part of the additional results and analyses into the Appendix, highlighted in blue, and plan to integrate the remaining findings into the main paper following further discussions and peer review. We sincerely appreciate your constructive feedback, which has helped us better clarify and strengthen the contributions of our work.
References
[1] F. Yang, L. Herranz, Y. Cheng, and M. G. Mozerov, “Slimmable Compressive Autoencoders for Practical Neural Image Compression,” CVPR, 2021, pp. 4996–5005.
[2] Y. Gao, R. Feng, Z. Guo, and Z. Chen, “Exploring the Rate-Distortion-Complexity Optimization in Neural Image Compression,” Journal of Visual Communication and Image Representation, vol. 105, art. 104294, 2024.
[3] D. He, Z. Yang, W. Peng, R. Ma, H. Qin, and Y. Wang, “ELIC: Efficient Learned Image Compression With Unevenly Grouped Space-Channel Contextual Adaptive Coding,” CVPR, 2022, pp. 5718–5727.
[4] M. Han, S. Jiang, S. Li, X. Deng, M. Xu, C. Zhu, and S. Gu, “Causal Context Adjustment Loss for Learned Image Compression,” NeurIPS, 2024.
Response to Reviewer 2bSM - Part I
Thank you for your thoughtful review. We address your concerns and questions below.
(a) Contribution and R–D–C performance. We agree that splitting or merging autoregressive (AR) groups is straightforward. Our contribution is not group merging itself, but establishing the granularity of conditional entropy modeling as a controllable computation dimension, characterizing its scaling behavior at both dataset and image levels, and using the observed image-dependent gains for rate–distortion–complexity (R–D–C) allocation. At a fixed rate condition, different context schedules reuse the same transform outputs, so changing context complexity does not require another transform pass.
This differs from capacity-based approaches such as slimmable models [1] or MoE-style routing, which primarily adjust the active transform capacity. RDCO [2] is a closer comparison within entropy modeling: it chooses where to apply spatial autoregressive context, using hyperprior-only or full-context prediction at different positions. ContextScale instead varies how finely latent dependencies are modeled across sequential groups. Both control entropy-model computation, but along different dimensions.
For clarity, we summarize the R–D–C comparisons in Fig. 4 of the main paper below, including the fixed-schedule and oracle references.
Table 1. PSNR BD-rate versus VTM-22.0 (%) at matched average complexity budgets.
ContextScale achieves lower BD-rates than the listed baselines at the reported budgets. More importantly, adaptive scheduling improves over ContextScale (fixed) wherever both are compared, even though they use the same model and schedule family. Rather than assigning the same schedule to every image, it allocates more context computation to images that still benefit from finer conditioning (see also Appendix D.1 for the resulting schedule distributions).
(b) Why four capacities? Each ContextScale model supports multiple rates and context complexities within a single set of weights (see Appendix B.3, Table 3). We use four model capacities to efficiently cover a broad R–D–C range, but context scaling itself does not require multiple models.
Table 2. Complexity coverage within each model (reference forward kMACs/pixel).
In principle, a single model can support a wider complexity range by allowing more AR steps. For example, extending the Small-model configuration from 15 to 32 steps would expand its complexity range from 353.98–726.55 to 353.98–1342.77 kMACs/pixel, without increasing its transform capacity.
However, although additional AR steps enable finer conditional modeling, all context schedules share the same transform outputs. Consequently, the compression gains from additional steps gradually diminish, potentially reflecting the limitations of the shared latent representation. This motivates us to consider parameter scaling and context scaling as complementary dimensions, rather than relying solely on increasing context steps.