- Rating
- 1(−2)
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1(−1)
Read review
Paper summary
ResARC improves ultra-low bitrate autoregressive image compression by synthesizing quantization residuals without extra bits and transmitting compressed generation residuals. Comparisons with generative codecs on DIV2K and CLIC2020 show improved perceptual similarity and distributional fidelity, while ablations support the complementary benefits of both residual branches.
Critical strengths
Meaningful validation of the design. Branch ablations and compensation-strategy comparisons support the complementary benefits of synthesizing quantization residuals and transmitting generation residuals. This strengthens the case that the architecture contributes to the improvements.
Critical weaknesses
- Limited methodological novelty. The contribution primarily lies in decomposing reconstruction errors and integrating established techniques into a residual-aware compression architecture. While the design is coherent and the reported gains are substantial, the methodological advance appears relatively incremental. This limits the strength of the paper’s originality claim.
- The computational cost is not evaluated. ResARC combines a 2B autoregressive backbone, encoder-side suffix-generation replay, a residual codec, and diffusion refinement, but reports no encoding or decoding latency or peak memory. Without these measurements, it is difficult to assess whether the bitrate savings justify the additional computation, limiting the practical significance of the results.
Questions for authors
- Tables 1–3 examine the contributions of the residual branches and model adaptation. Do these controls also account for increased model capacity, or could additional capacity explain part of the performance improvement over ARPC?
- What are the encoding latency, decoding latency, and peak memory at the evaluated resolution? Are measurements available against ARPC and competitive diffusion codecs under identical hardware conditions?
- A key concern is the implicit orthogonal additivity assumption underlying the residual decomposition in Equation 7. The two residuals are inherently coupled — coarser quantization increases both errors by degrading token information — and their frequency spectra overlap substantially. Could the authors justify this assumption, quantify the correlation between the two residuals, and clarify whether the two branches deliver truly complementary gains or involve redundant compensation?
9 comments
Response to Reviewer X3QL(2/2)
Q3. Does the residual decomposition require orthogonality, and do the two branches provide complementary gains?
A3. Thank you for this insightful comment. The two residuals arise from different processes: the quantization residual captures details lost during tokenization, while the generation residual captures suffix generation errors affecting local structures and textures. The residual decomposition in Equation (7) is an exact algebraic identity and does not require orthogonality or disjoint frequency spectra. We further analyze their correlations on DIV2K. For each prefix depth, we compute the angle between the pooled quantization and generation residual vectors. As shown in Figure 3(a), these angles range from 84.0° to 92.5°, suggesting that the pooled residual vectors are approximately orthogonal. Furthermore, Figure 3(b) shows the mean signed Pearson correlation for each residual pair, with lower values for quantization–generation pairs than for pairs of generation residuals at different prefix depths. Table R3.3 also reports a mean absolute Pearson correlation of 0.076 between quantization and generation residuals (Q–G), compared with 0.411 between generation residuals (Within-G) at different transmitted prefix depths for the same image. These results indicate substantially weaker linear correlation between quantization and generation residuals than among generation residuals at different prefix depths.
Furthermore, Table 2 quantitatively shows that the generation-only and quantization-only configurations require 1.90% and 5.38% more bitrate, respectively, than full ResARC at matched DISTS. Together, these results support complementary gains from compensating both residuals.
Table R3.3: Pearson correlations by depth (mean ).
\hline k & \text{Q–G} & \text{Within-G} \\\\ \hline 5 & 0.104 & 0.362 \\\\ 6 & 0.097 & 0.416 \\\\ 7 & 0.077 & 0.444 \\\\ 8 & 0.060 & 0.447 \\\\ 9 & 0.052 & 0.426 \\\\ 10 & 0.068 & 0.370 \\\\ \textbf{All} & \mathbf{0.076} & \mathbf{0.411} \\\\ \hline \end{array}$$ **W1. Limited methodological novelty.** **A4.** Thank you for this comment. To our knowledge, **ResARC is the first to explicitly distinguish and compensate for quantization and generation residuals in autoregressive generative compression**, both of which are key factors affecting perceptual compression performance. The originality of our method lies in linking the explicit formulation of these two residuals to distinct compensation mechanisms based on their different roles in reconstruction. Specifically, we develop compensation strategies tailored to their distinct characteristics, namely, modeling the quantization residual using a DiT and explicitly encoding and transmitting the generation residual, **thereby achieving competitive performance against leading generative codecs.** **References:** [1] PerCo (SD): Open Perceptual Compression. [2] Lossy Compression with Pretrained Diffusion Models. [3] Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior. [4] Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion. [5] AutoRegressive-based Progressive Coding for Ultra-Low Bitrate Image Compression. [6] Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis.After carefully reviewing the rebuttals, my primary concerns regarding methodological novelty and computational cost remain unresolved. Consequently, I am adjusting my score as the trade-offs introduced by this design are, in my view, unacceptable for practical deployment.
- Unacceptable Computational Cost and Latency: The authors' response regarding latency (Table R2.5 / Table 5) reports an encoding time of 2.98–3.23s and a decoding time of 3.39–3.65s on an RTX 3090 GPU. While the authors argue that the additional DiT and generation residual codec (GRC) modules account for less than 2% of the total decoding time, this defense misses the forest for the trees. The baseline itself (Infinity-2B autoregressive backbone) is already prohibitively heavy and slow. Stacking heavy diffusion-based generation and explicit residual coding on top of an already sluggish 2B–3.6B parameter autoregressive model results in an end-to-end decoding latency of over 3.5 seconds per image. In ultra-low bitrate image compression scenarios, a 3.5+ second decoding delay is a critical bottleneck that renders the method practically infeasible. The authors have failed to provide any viable pathway or lightweight alternative for practical deployment.
- Incremental Novelty and Component Stitching: Regarding the concern over technical novelty, the authors position ResARC as the first to explicitly distinguish and compensate for quantization and generation residuals. However, a close inspection reveals that the framework is essentially an engineering composition of existing, off-the-shelf generative components—namely, applying Flow Matching via a DiT for quantization residuals and a traditional analysis/synthesis transform (with a four-pass entropy model) for generation residuals, tied together with an ARPC backbone. The post-hoc geometric analysis (such as the Q-G angle and Pearson correlation in Figure 3) merely rationalizes the architectural choices with descriptive statistics rather than providing a fundamental breakthrough in compression theory or conceptual design. The contribution remains an incremental modular combination rather than a conceptual advancement.
Response to Reviewer X3QL(1/2)
Q1. Could increased model capacity explain the improvements over ARPC, rather than the residual-specific design?
A1. We compare ResARC with ARPC [5] and an enlarged AR prior without our residual-specific design on DIV2K. Specifically, the enlarged AR prior adds two Transformer blocks and rank-64 LoRA to Infinity-2B [6], introducing an additional 230.76M trainable parameters. As shown in Table R3.1, the enlarged AR prior without our residual-specific design requires 19.64% and 262.57% more bitrate than ResARC at matched FID and KID, respectively. Moreover, ResARC achieves better BD-rates across all four metrics than the enlarged AR prior, despite having fewer total parameters (3.64B vs. 3.77B). These results demonstrate that the improvements over ARPC cannot be explained simply by increasing model capacity and support the effectiveness of our residual-specific design.
Table R3.1: Comparison with ARPC and an enlarged AR prior on DIV2K. BD-rate (%) is relative to ResARC.
\hline \text{Method} & \text{Params.}\downarrow & \text{LPIPS}\downarrow & \text{DISTS}\downarrow & \text{FID}\downarrow & \text{KID}\downarrow \\\\ \hline \text{ARPC [5]} & 3.54\mathrm{B} & +40.61 & +75.24 & +104.00 & +271.91 \\\\ \text{Enlarged AR prior} & 3.77\mathrm{B} & +6.38 & +6.35 & +19.64 & +262.57 \\\\ \textbf{ResARC (Ours)} & 3.64\mathrm{B} & \mathbf{0.00} & \mathbf{0.00} & \mathbf{0.00} & \mathbf{0.00} \\\\ \hline \end{array}$$ **W2 & Q2. What are the encoding latency, decoding latency, and peak memory compared with ARPC and competitive diffusion codecs under the same hardware conditions?** **A2.** Thank you for this suggestion. We evaluate the computational efficiency of ResARC against leading generative codecs on DIV2K at 1024 × 1024 resolution, using an RTX 3090 GPU with a batch size of 1. As shown in Table R3.2, ResARC requires 2.98–3.23 s for encoding and 3.39–3.65 s for decoding. **Its decoding latency is comparable to ARPC [5] and competitive with the evaluated diffusion-based codecs.** Moreover, ResARC reduces peak GPU memory usage by 13.8% compared with ARPC (13.62 vs. 15.80 GiB). Importantly, ResARC achieves BD-rate reductions of 28.88%, 42.94%, 50.98%, and 73.11% over ARPC in terms of LPIPS, DISTS, FID, and KID, respectively. These results demonstrate that **ResARC achieves a favorable rate--perception--latency trade-off while reducing peak GPU memory usage compared with ARPC.** Table R3.2: **End-to-end coding latency, peak GPU memory usage, and BD-rate (%) comparison on DIV2K (1024 × 1024).** $$\def\rcell#1{\smash{\begin{array}{c}\hspace{4.4em}\\\\#1\\\\\hspace{4.4em}\end{array}}} \def\rmetrics#1#2#3#4{\begin{array}{cccc}\rcell{#1} & \rcell{#2} & \rcell{#3} & \rcell{#4}\end{array}} \begin{array}{lcccc} \hline \text{Method} & \text{Enc. (s)}\downarrow & \text{Dec. (s)}\downarrow & \begin{array}{c} \text{Peak GPU} \\\\ \text{memory (GiB)}\downarrow \end{array} & \begin{array}{c} \text{BD-rate vs. ARPC }(\\%) \\\\ \hline \rmetrics{\text{LPIPS}\downarrow}{\text{DISTS}\downarrow}{\text{FID}\downarrow}{\text{KID}\downarrow} \end{array} \\\\ \hline \text{PerCo (SD) [1]} & 0.15 & 9.13 & 4.86 & \rmetrics{+190.35}{+639.33}{+770.14}{+2936.40} \\\\ \text{DiffC [2]} & 1.99 & 6.24 & 4.99 & \rmetrics{+31.40}{+359.73}{+417.41}{+792.74} \\\\ \text{DiffEIC [3]} & 0.48 & 22.64 & 9.21 & \rmetrics{+106.16}{+520.97}{+423.82}{+639.93} \\\\ \text{ResULIC [4]} & 0.32 & 4.12 & 9.17 & \rmetrics{+35.52}{+297.70}{+236.68}{+801.30} \\\\ \hline \text{ARPC [5]} & 1.81\text{–}2.56 & 3.57\text{–}5.89 & 15.80 & \rmetrics{0.00}{0.00}{0.00}{0.00} \\\\ \textbf{ResARC (Ours)} & 2.98\text{–}3.23 & 3.39\text{–}3.65 & 13.62 & \rmetrics{\mathbf{-28.88}}{\mathbf{-42.94}}{\mathbf{-50.98}}{\mathbf{-73.11}} \\\\ \hline \end{array}$$