acceptodds
Under review as a conference paper at ICLR 2027

HyPIC: Rethinking Residual Representation for Progressive Image Compression

Abstract

Progressive image compression enables bandwidth-adaptive transmission by successively refining a coarse representation. These refinements can be encoded as residuals, allowing each stage to describe information not captured by the preceding approximation. Residual vector quantization (RVQ) implements this idea in learned codecs by quantizing the remaining latent residual and accumulating codewords to approximate the encoder output. We challenge latent residual approximation as the organizing principle of progressive refinement. Once a prefix establishes a base reconstruction, the remaining information should be encoded in a representation learned for improving the decoded image, without requiring faithful recovery of the original latent residual. To this end, we introduce Subspace-Projected Refinement(SPR), which jointly learns a compact representation of residual information and the directions through which its decoded coefficients update the VQ prefix. SPR is optimized for image rate–distortion, with a coding dimensionality chosen independently of the base latent's channel dimension. Individual coefficients adjust learned correction directions, while their synthesis jointly modifies the original latent channels. This structure leads to Hybrid Progressive Image Compression (HyPIC): VQ establishes the base reconstruction through shared joint patterns, and scalar quantization (SQ) encodes image-specific corrections through the learned SPR coefficients. Experimental results show that HyPIC attains competitive compression performance compared with previous methods. It achieves LPIPS BD-rate savings of up to 75.46% over Control-GIC across the four evaluated datasets. Beyond perceptual quality, HyPIC enables progressive transmission for flexibility, and also delivers faster encoding and decoding compared with ProGIC on GPUs for efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Progressive image compression, Residual representation learning, Hybrid quantization, Ultra-low bitrate compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Li4JOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes HyPIC, a progressive image compression framework that combines Residual Vector Quantization (RVQ) with a novel Subspace-Projected Refinement (SPR) stage. The core idea is to follow N RVQ stages (which establish a shared joint latent prefix via learned codebooks) with a scalar-quantized correction in a learned low-dimensional subspace. The authors motivate this design with a local first-order analysis showing that residual refinement value depends on (i) alignment with the remaining decoder-space error and (ii) the decoded effect of the correction, neither of which is captured by Euclidean residual approximation alone.

The method is evaluated on Kodak, Tecnick, DIV2K, and CLIC 2020, reporting strong perceptual BD-rate gains over HiFiC, MS-ILLM, Control-GIC, PerCo, CTC, CMIC, RDVQ, and ProGIC, with competitive encoding/decoding speed (5.59 ms / 6.39 ms on 512×768 images). Ablations cover VQ depth, activation-free mappings, correction width, comparison against PCA/random/fixed-channel projections, and statistical treatments.

Critical strengths
  1. This paper has a clear and well-motivated design analysis. Section 3.1 and Appendix B provide a clean local decomposition (Eq. 13) separating unreachable feature error from decoder-weighted coefficient error. This gives a principled rationale for learning synthesis directions (B) rather than simply coding the residual, and the distinction is empirically supported by Table 4: decoding the exact encoder latent y (PSNR 22.59, LPIPS 0.1910) is worse than the quantized SPR correction (PSNR 26.02, LPIPS 0.1440). This "exact latent recovery is not optimal" observation is a genuinely useful finding for the community.

  2. This manuscript has strong empirical results across multiple datasets. The BD-rate reductions relative to Control-GIC are substantial (LPIPS: −72.08% Kodak, −64.08% Tecnick, −74.53% DIV2K, −75.46% CLIC). Figure 3 shows consistent dominance across all four datasets. The results are notable given the very low rate regime (0.03–0.20 bpp).

  3. The paer shows excellent computational efficiency that is a meaningful practical contribution.

Critical weaknesses

1、While the paper compares against PCA and random projections (Figure 6), there is no comparison against a well-tuned scalar residual coder (e.g., a hyperprior-based SQ codec operating directly on the 256-dim residual). The "5RVQ bypass" in Table 2 uses the unchanged VQ decoder, which is not a fair SQ baseline. I want to see whether SPR's gains persist against a properly trained SQ refinement stage. The claim that "fixed-dimensional reductions do not reproduce the benefit of learned directions" is supported, but the comparison set could be stronger.

2、The authors state: "As an official ProGIC implementation was unavailable for our evaluation, we use an in-house reimplementation to measure its computational cost on the same hardware." However, ProGIC's RD performance in Figure 3 appears to come from the original paper, while its computational cost comes from the reimplementation. This mixed protocol should be clarified — are the RD curves for ProGIC from the original paper or the reimplementation? If from the original paper, hardware/implementation differences could confound the speed comparison. If from the reimplementation, the RD curve may not reflect ProGIC's true performance.

3、Table 7 uses a different configuration than the main model. The width study uses four-VQ variants trained for only 100,000 updates, while the main model uses three VQ stages trained for 1.5M updates. The statement "We use d = 10 in the three-VQ main model as a practical design choice separately from this four-VQ width comparison" is honest, but the reader cannot tell whether d = 10 is actually near-optimal for the three-VQ model. A width sweep on the same configuration as the main results would be more informative.

4、The paper claims "four progressively decodable operating points from one model," but there is no figure or table showing the rate-distortion trajectory as stages are added (e.g., PSNR/LPIPS at each prefix vs. cumulative bpp). Figure 5 shows the effect of VQ depth, but it appears to compare separate models rather than the progressive decoding of a single model. A plot showing the quality at each operating point (prefix 1, prefix 2, prefix 3, final SPR) would substantiate the progressive claim.

5、Entropy model details are underspecified. Eq. (6) combines a Gaussian score and learned discrete logits, but the paper does not specify:

6、How the Gaussian score is computed (is it a learned Gaussian over codewords? A distance-based score?).

7、Whether the discrete logits are spatial-context-dependent (the text mentions "top and left spatial context" but the architecture in Figure 2b is not fully clear).

8、How the hyperprior features f_k are combined with the codeword scores.

9、Equation numbering: Eq. (10) is referenced in the text but I could not find it in the provided pages (it may be in Appendix B; if so, the reference in Section 3.4 should point there).

10、Table 1 formatting: The table is difficult to parse in the provided PDF (columns run together). Ensure the final version is legible.

11、Figure 1 caption: "Gold outlines mark selected cells" — but the figure shows red stars and dots; the description could be clearer.

12、Typos: "igure 1" (p. 2), "igure 3" (p. 7), "igure 5" (p. 9) — missing "F". "Balle et al., 2016" is misspelled as "Balle" (correct is Ballé). "Toderici et al., 2015" is cited but the reference list shows "Toderici et al., 2015" — consistent, but check the 2017 entry.

13、Reference [Cao et al., 2026]: ProGIC is cited as CVPR 2026; ensure this is a real, forthcoming publication (the year is in the future relative to typical review cycles — this may be a preprint or a placeholder).

14、ProGIC comparison: Are the RD curves in Figure 3 for ProGIC from the original paper or your reimplementation? If from the original paper, how do you ensure the speed comparison in Table 1 is fair?

15、SQ baseline: How does HyPIC compare against a hyperprior-based SQ codec operating on the full 256-dim residual at the same total rate? This would isolate the benefit of the learned subspace from the benefit of scalar quantization itself.

16、Progressive decoding: Can you provide a plot of PSNR/LPIPS vs. cumulative bpp for each of the four operating points from a single model, to substantiate the progressive claim?

17、Activation-free mappings: Why do identity maps outperform SiLU/ReLU? Is this specific to the SPR stage, or does it also hold for the VQ stages?

18、Width selection: Is d = 10 optimal for the three-VQ main model? What happens at d = 4, 8, 16 for the same configuration?

19、Entropy model: Can you provide the exact architecture of the Gaussian score and discrete logit branches in Eq. (6), and the parameter head for δ and σ?

20、Generalization: How does HyPIC perform on out-of-distribution content (e.g., medical images, screenshots, text)? The training set is ImageNet-1k, which may bias toward natural images.

21、Rate range: The evaluated rate range is 0.03–0.20 bpp. How does HyPIC perform at higher rates (e.g., 0.5–1.0 bpp)? Does the advantage persist, or does it diminish?

Questions for authors

Please see the weaknesses part

1 comment

Dear Reviewer Li4J,

Thank you again for your detailed review and for recognizing the motivation behind learning correction directions. We submitted our rebuttal about five days ago, grouping your 21 questions into related topics while retaining their original numbering. With three days remaining in the discussion period, we would sincerely appreciate knowing whether our responses have resolved your concerns.

The main points of our response are:

  • SQ comparisons and SPR evidence: We presented the independently trained Pure SQ results and explained the distinct roles of the full-model comparisons and the appendix’s residual-representation studies.
  • ProGIC comparison: We clarified that both its RD results and computational costs come from our reimplementation, enabling direct runtime comparison with HyPIC on RTX 4090.
  • Progressive reconstruction and model details: We provided the four-stage trajectory from one model and explained the entropy-model conditioning, learned linear mappings, and the setup and interpretation of the width study.
  • Presentation and scope: We addressed the equation, figure, and reference questions, and explained the focus on low-rate natural-image compression.

We value the constructive discussion that is an important part of ICLR’s review process and sincerely hope to continue it with you. Please let us know which points, if any, would benefit from further explanation; we are ready to respond promptly during the remaining discussion period.

Thank you for your time and consideration.

Best regards,
The Authors

Reviewer_j3WvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces HyPIC, a progressive image compression framework that combines vector quantization for a base reconstruction with scalar-quantized refinement coefficients. Its core component, Subspace-Projected Refinement (SPR), learns a compact representation and correction directions optimized for image rate–distortion, rather than faithfully approximating the original latent residual. Experiments on four datasets report competitive compression performance.

Critical strengths
  1. The work addresses a meaningful problem.

  2. The paper reports promising experimental results across four datasets, with competitive perceptual rate–distortion performance and LPIPS BD-rate savings of up to 75.46% over Control-GIC. The faster GPU encoding and decoding compared with ProGIC also suggest practical efficiency benefits.

Critical weaknesses
  1. The experimental comparisons do not cover several recent and relevant methods. In particular, ARPC [1] is a closely related progressive image compression method but is not included in the evaluation. Recent GIC methods, including OneDC [2] and DiT-IC [3], as well as efficiency-oriented approaches such as EF-LIC [4] and CoD-Lite [5], are also absent from the comparisons. Including these methods would help better establish the competitiveness of HyPIC in compression performance and computational efficiency.

  2. The ablation studies could be strengthened to better support the claimed benefits of SPR. Table 2 compares 5RVQ with 3RVQ+SPR, but does not isolate the contribution of SPR from the replacement of RVQ with low-dimensional SQ. In Appendix A.1, the alternative projections are evaluated using a decoder already trained with SPR, rather than being independently optimized. Moreover, the results in Table 4 may partly reflect a mismatch between the decoder's training and evaluation inputs. Additional end-to-end comparisons with existing low-dimensional quantization methods, under consistent entropy models and rate constraints, would help clarify the source of the improvements.

  3. The methodological novelty of SPR could be clarified further in relation to existing approaches. DAC [7] already employs learned low-dimensional projections and reconstruction mappings for residual quantization, while FSQ [8] explores low-dimensional scalar quantization. ProGIC [6] also performs progressive residual quantization with image-level perceptual supervision. SPR combines several similar principles, and its main distinction from these methods is not yet sufficiently clear. The Jacobian-based analysis provides useful motivation, but does not establish a specific theoretical advantage over existing quantization approaches.

[1] Ziyuan Zhang, et al. Autoregressive-based progressive coding for ultra-low bitrate image compression. ICLR, 2026.

[2] Naifu Xue, et al. One-step diffusion-based image compression with semantic distillation. NeurIPS 2025.

[3] Junqi Shi, et al. DiT-IC: Aligned diffusion transformer for efficient image compression. CVPR 2026.

[4] Hao Cao, et al. Efficient learned image compression without entropy coding. ICML 2026.

[5] Zhaoyang Jia, et al. CoD-Lite: Real-time diffusion-based generative image compression. ICML 2026.

[6] Hao Cao, et al. ProGIC: Progressive and lightweight generative image compression with residual vector quantization. arXiv, 2026.

[7] Rithesh Kumar, et al. High-fidelity audio compression with improved RVQGAN. NeurIPS 2023.

[8] Fabian Mentzer, et al. Finite scalar quantization: VQ-VAE made simple. ICLR 2024.

Questions for authors
  1. There appears to be some mismatch between the motivation and the actual technical design, as several key components have already been explored in prior work. Moreover, the current experiments do not clearly demonstrate advantages over recent state-of-the-art methods. Could the authors further clarify the distinct contributions of SPR and strengthen the experimental evidence supporting its effectiveness?

  2. As a minor presentation issue, Figure 1 takes a long time to render in PDF viewers, while the text in Figure 2 is relatively small. Both issues affect the readability of the paper and could benefit from further optimization.

1 comment

Dear Reviewer j3Wv,

Thank you again for your constructive suggestions. We submitted our rebuttal about five days ago. With three days remaining in the discussion period, we would be grateful for your feedback on whether our explanations and additional results have addressed your main concerns.

To briefly summarize our response:

  • Distinction from existing methods: We explained how SPR differs from DAC, FSQ, and ProGIC by learning the residual coordinates and synthesis directions that improve an existing VQ reconstruction through image rate-distortion optimization.
  • Evidence for SPR: We added results from an independently trained, randomly initialized Pure SQ model and explained how this comparison complements the appendix studies of alternative projections, channel selection, and normalization.
  • Recent baselines and presentation: We reported comparisons with ARPC, EF-LIC, CoD-Lite, OneDC, and DiT-IC across 25 Kodak24 operating points, including perceptual metrics and RTX 4090 runtimes. We also outlined the planned improvements to figure readability.

We particularly value ICLR’s tradition of open and constructive author–reviewer dialogue. We sincerely hope to hear your assessment of these responses and to discuss any remaining questions before the discussion closes. Even a brief indication of the points that still need clarification would help us focus our next response.

Thank you again for your time.

Best regards,
The Authors

Reviewer_8wWsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HyPIC, a progressive image compression framework combining residual vector quantization (RVQ) and scalar quantization (SQ). Multiple RVQ stages establish a base latent representation, followed by a Subspace-Projected Refinement module that transforms the remaining latent residual into low-dimensional coefficients. These coefficients are scalar-quantized and mapped back to the original latent space to refine the base representation.

Critical strengths
  1. The proposed method is conceptually simple and easy to follow.

  2. The reported results suggest a favorable trade-off between computational complexity and perceptual quality, as measured by LPIPS and DISTS.

Critical weaknesses
  1. Limited novelty: The proposed method largely follows the RVQ-based progressive coding framework of ProGIC[1], augmenting it with entropy coding of VQ indices and a final scalar-quantized refinement stage. However, entropy modeling and coding of VQ indices have already been explored in prior work, such as UIGC[2] , while hybrid VQ/SQ designs have also been investigated in image compression, including HybridFlow [3] and DLF [4]. Although the specific implementation differs from prior work, I find the methodological contribution incremental and insufficient to establish substantial novelty.

  2. Insufficient comparison with recent methods: The experimental evaluation omits recent diffusion-based image compression methods, including CoD-Lite [5], OneDC [6], and DiT-IC [7]. These methods specifically target efficient generative compression and are therefore relevant to assessing HyPIC's trade-off between perceptual quality and computational cost. Comparisons of perceptual rate–distortion performance and encoding/decoding latency under comparable evaluation settings would provide a more convincing assessment of the proposed method's competitiveness.

Questions for authors
  1. Equation (6) conditions the probability of each VQ index on previously decoded indices, and the text specifies top and left spatial neighbors as context. Does this entail sequential autoregressive decoding across spatial positions within each VQ stage? Such dependencies can limit spatial parallelism and increase decoding latency. Please clarify the decoding schedule and whether the decoding times in Table 1 include the full sequential probability prediction and actual entropy decoding of all transmitted symbols.

  2. Please provide comparisons with CoD-Lite [5], OneDC [6], and DiT-IC [7] in terms of LPIPS, DISTS, and FID, as well as encoding and decoding latency under comparable evaluation settings.

1 comment

Dear Reviewer 8wWs,

Thank you again for your thoughtful review of our paper. We submitted our rebuttal about five days ago and would like to kindly follow up. With three days remaining in the discussion period, we would greatly appreciate hearing whether our responses have addressed your concerns.

For ease of reference, our response focuses on three points:

  • Contribution and relationship to prior work: We explained how SPR learns a residual representation specifically for improving an existing VQ reconstruction, and compared its role with those of ProGIC, UIGC, HybridFlow, and DLF. We also discussed the appendix experiments supporting the learned correction space.
  • Recent-codec comparisons: We added Kodak24 comparisons with CoD-Lite, OneDC, and DiT-IC, reporting LPIPS, DISTS, and encoding/decoding times on RTX 4090.
  • Eq. (6) and decoding: We explained the distinction between the likelihood model and the runtime coding path, and addressed your questions about spatial dependencies and the reported timing scope.

We value ICLR’s tradition of constructive discussion between authors and reviewers and sincerely hope to continue this exchange. Could you let us know which concerns, if any, remain unresolved? We would be happy to respond promptly while the discussion remains open.

Thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.