acceptodds
Withdrawn from ICLR 2027

Tree-VQ: Progressive Image Compression from Pretrained Vector Quantizers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Progressive image compression requires a single embedded representation whose received prefixes can be decoded without re-encoding the source. Modern vector-quantized (VQ) image models provide strong discrete endpoint representations, but conventional flat codeword indices do not define meaningful intermediate states for a neural decoder. We present Tree-VQ, a post-hoc conversion of a pretrained flat VQ tokenizer into a fine-grained, arbitrary-prefix progressive representation while preserving its encoder assignments and every learned leaf vector. The key idea is to organize the original codebook into a balanced binary hierarchy, associate explicit representations with internal nodes, and transmit branch decisions in depth-major order. Consequently, once the image header is available, every payload prefix uniquely specifies a valid latent state: each additional branch bit refines exactly one token, and transmission can therefore be truncated at essentially any payload position rather than only at a small number of stage boundaries. We further adapt one shared decoder on the complete-depth and mixed-depth latent states encountered under such arbitrary truncation, making these densely spaced prefixes useful for reconstruction rather than merely syntactically decodable. On Kodak, Tree-VQ achieves a DISTS-based BD-rate saving of 52.1% relative to ProGIC, while exposing thousands of valid arbitrary-prefix operating points from a single embedded bitstream.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Learned image compression, Tree-structured vector quantization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vhwHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Tree-VQ, which converts a pretrained flat VQ tokenizer post hoc into a progressive image codec that can be decoded from an arbitrary bit position. While keeping the existing codebook and encoder assignments unchanged, the method reorganizes the codebook into a balanced binary tree and transmits branch bits in depth-major order, so that every bitstream prefix corresponds to a valid latent state. Only the decoder is additionally adapted to handle these intermediate states. The experiments report that a single bitstream provides thousands of operating points while achieving perceptual quality competitive with existing generative codecs in the ultra-low bitrate range.

Critical strengths
  • Since every bit position in a single embedded bitstream yields a valid reconstruction state, more than 20,000 operating points are available for a single Kodak image. Transmission can be stopped at any time without re-encoding, and rate control is much finer than in existing progressive VQ methods such as ProGIC, whose granularity is tied to the number of RVQ stages.
  • Below 0.06 bpp, the method achieves competitive LPIPS/DISTS results against several GIC codecs, including diffusion-based ones, showing that a progressive structure and perceptual quality can coexist. It also outperforms RDVQ-adj, a rate-adjustable method from the same family.
  • The encoder and codebook are not retrained; only tree construction and decoder adaptation are added. The method uses fixed-length bits without entropy coding, and the paper transparently reports that arithmetic coding yields only a 0.36% gain.
  • Although the core mechanism is the same as classical progressive VQ (Riskin et al., 1994), the paper's strength lies in its practical completeness: it makes this mechanism work with modern neural tokenizers and nonlinear decoders while achieving strong perceptual quality at ultra-low bitrates.
Critical weaknesses

Novelty

  • The authors distinguish their work from Riskin et al. by stating that "the leaf codebook and encoder assignments come from a pretrained neural tokenizer and remain fixed." However, Riskin's method likewise keeps the codebook and full-search assignments fixed and constructs only the tree post hoc. Riskin's method also uses depth-major transmission, and stopping in the middle of a pass naturally produces the mixed-depth state described in Tree-VQ. Although Riskin does not explicitly discuss mid-pass truncation, the property is the same, since an intermediate codeword is displayed for each received bit. The only change is the source of the codebook (a neural tokenizer), which is a weak distinction in terms of tree construction.
  • The first two contributions claimed in the introduction (and the corresponding key idea in the abstract), which are described in Sec. 3.2–3.3, are overstated. The balanced binary tree, index reassignment, retention of full search, pass-wise bit-by-bit transmission, reconstruction from an intermediate codeword at each bit, and top-down equal partitioning are all present in Riskin et al. The proof in App. C also follows directly from this structure. The contribution should be reframed as "applying a classical full-search progressive transmission tree to a neural tokenizer and adapting a nonlinear decoder to mixed-depth states."

Cost of progressive coding

The comparison with RDVQ-adj demonstrates competitiveness against a rate-adjustable method of the same architectural family. However, because the base tokenizers differ, the contribution and cost of the progressive structure itself are not isolated. Comparisons using the same tokenizer should be provided, for example: (a) the full-depth performance of the original decoder before adaptation, (b) non-progressive models trained per rate with the same tokenizer, and (c) RDVQ-adj-style partial transmission and completion applied to the same tokenizer.

Generality of the method

The paper's central claim is that it converts an existing pretrained tokenizer post hoc, but the method is applied only to a single tokenizer trained by the authors. Results on publicly available tokenizers (e.g., the VQGAN family) are needed to verify the generality of the method.

Model complexity

With about 186M parameters in total, the model is much larger than lightweight codecs such as ProGIC. A comparison and discussion of encoding/decoding time complexity should be included. FLOPs are also not reported.

Missing distribution-level metrics

The evaluation reports only LPIPS, DISTS, and CLIP-IQA, and no distribution-level metric such as FID is provided, even though most of the compared generative codecs report FID and App. E is titled "Realized Rate, Distributional Metrics, and Runtime Tables." Since realism is a key evaluation axis for ultra-low-bitrate generative compression, FID (e.g., following the patch-based protocol commonly used in this field) should be reported.


My rating may be adjusted depending on how the authors address and resolve the concerns raised in my review.

Questions for authors
  1. Riskin et al. (1994) also keep the codebook and full-search assignments fixed, construct the tree post hoc, and transmit bits in depth-major order. Beyond the source of the codebook (a neural tokenizer), what aspects of the tree construction and prefix representation are technically different from Riskin's method?
  2. Can the authors provide comparisons using the same tokenizer to isolate the contribution and cost of the progressive structure, specifically: (a) the full-depth performance of the original decoder before adaptation, (b) non-progressive models trained per rate with the same tokenizer, and (c) RDVQ-adj-style partial transmission and completion applied to the same tokenizer?
  3. Can the authors apply the post-hoc conversion to publicly available pretrained tokenizers (e.g., the VQGAN family) to demonstrate the generality of the method?
  4. Can the authors report encoding/decoding time and FLOPs, and compare them with the baseline codecs, including lightweight codecs such as ProGIC?
  5. Can the authors report a distribution-level metric such as FID?
Reviewer_QyzQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents Tree-VQ method,a post-hoc conversion of a pretrained flat VQ tokenizer into a fine-grained,arbitrary-prefix progressive representation. The method organizes the original codebook into a balanced binary hierarchy,associate explicit representations within ternalnodes,and transmit branch decisions in depth-major order. Compared with prior-arts like ProGIC and GLC, the proposed method shows remarkable coding gains. In the domain of progressive learned image coding, the paper shows some benefits. But the reviewer has some concerns on this topic. For Learned-based image/video coding, there are still a lot of fundamental issues must be solved, like PSNR BD performance, complexity, multiple platform compatiblity, etc. Progressive coding is not important at this stage for learned-based image/video coding. In the paper, no comparison is made between the proposed method and main-streaming industrial standard like VVC-Intra, which can also support progressive coding by multi-layer coding. Such comparisons are mandate since Learned-based image/video coding is much more complicated than traditional video coding. It is useless if it cannot show a better performance than the performance of traditional standard.

Critical strengths

The fine-grained,arbitrary-prefix progressive representation is novel in the domain. Compared with prior-arts like ProGIC and GLC, the proposed method shows remarkable coding gains.

Critical weaknesses

For Learned-based image/video coding, there are still a lot of fundamental issues must be solved, like PSNR BD performance, complexity, multiple platform compatiblity, etc. Progressive coding is not important at this stage for learned-based image/video coding. In the paper, no comparison is made between the proposed method and main-streaming industrial standard like VVC-Intra, which can also support progressive coding by multi-layer coding. Such comparisons are mandate since Learned-based image/video coding is much more complicated than traditional video coding. It is useless if it cannot show a better performance than the performance of traditional standard.

Questions for authors

Please compare the proposed method and VVC-Intra with multi-layer configrations under the common test condition of JVET ([M. Karczewicz, Y. Ye, “Common Test Conditions and evaluation procedures for enhanced compression tool testing”, JVET-Y2017, Jan. 2022.)

Reviewer_3PLrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Tree-VQ, which converts a pretrained flat VQ tokenizer post hoc into a fine-grained progressive image representation. Keeping the encoder and all 4096 leaf codewords fixed, it organizes the codebook into a depth-12 balanced binary tree via recursive balanced 2-means (internal nodes are child means) and serializes branch bits in depth-major order, so that every bit prefix maps to a unique mixed-depth latent state. Only the shared decoder is then fine-tuned with rate-balanced prefix sampling (MSE + LPIPS + GAN). On Kodak, DIV2K, and CLIC2020, the paper reports DISTS BD-rate savings of 52.1% / 49.0% / 53.4% relative to the published ProGIC points, with thousands of bit-level truncation points at ≤0.062 bpp, supported by ablations on tree construction, prefix sampling, and routing.

Critical strengths
  • Good embedded bitstream design. Depth-major serialization makes every bit a valid truncation point and decouples progressive granularity from the number of quantization stages, with a proof of prefix decodability and a full-depth latent identical to the original tokenizer. This is the core contribution and what distinguishes the method from stage-based progressive codecs such as ProGIC.
  • Strong perceptual performance at ultra-low rates. On three datasets, Tree-VQ matches or exceeds strong baselines in LPIPS/DISTS, including billion-parameter diffusion codecs such as OneDC, with ~186M parameters and rates computed from actual serialized bytes. This shows that progressive transmission is achieved at a practically useful quality level.
Critical weaknesses
  • The gains may come from its backbone (RDVQ), not from the tree. Tree-VQ is built on the RDVQ architecture, but the paper never compares with the original RDVQ. Against RDVQ-adj the gain is small (e.g., DISTS 0.0814 vs. 0.0832 in Fig. 11), while the 52% headline is measured against the much weaker ProGIC.
  • Limited novelty. The tree codebook and level-by-level bit transmission are classical TSVQ ideas (Gray et al., 1992, Riskin et al., 1994). The "post-hoc" claim is also weak, because the decoder is retrained and only one self-trained tokenizer is tested.
  • Missing details and weak evaluation. The paper does not say which internal nodes (C or E) the main model uses or which set was used to pick the checkpoint. There is no FID/KID/PSNR.
Questions for authors

Questions that could change my assessment

  • Please add the RD curve of the original RDVQ, the full-depth point of the self-trained flat tokenizer with its original decoder, and LPIPS/DISTS BD-rates against RDVQ and RDVQ-adj. If progressivity comes at nearly no RD cost, I would raise my score.
  • Please add a token-major baseline: transmit full 12-bit indices token by token, replace unreceived tokens with a mask or mean vector, and fine-tune the decoder with the same prefix sampling. Without it, the paper does not show that depth-major refinement, its central design choice, beats spatial progression.
  • The 52% saving is a cross-paper comparison (local pyiqa pipeline vs. published ProGIC points), while your ProGIC* reproduction is used only for plotting. Please report BD-rates against ProGIC*, and LPIPS/DISTS BD-rates against stronger baselines (OneDC, RDVQ-adj, ARPC), given that the Tree-VQ and OneDC curves nearly overlap in Fig. 6.
  • Why do the Tree-VQ curves in Fig. 6 start only at ~0.01 bpp (Kodak) and ~0.017 bpp (DIV2K/CLIC2020), while RDVQ-adj and ARPC extend below 0.005 bpp? How good are prefixes at depth 0–2, and does quality improve monotonically with prefix length? Given the 0.062 bpp cap, please also quantify the practical benefit of bit-level granularity over one operating point per depth plane.

Clarifications

  • Does the main model use internal nodes C or E, and on which set was the checkpoint selected? Can you add FID/KID/PSNR?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.