- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Tree-VQ, which converts a pretrained flat VQ tokenizer post hoc into a progressive image codec that can be decoded from an arbitrary bit position. While keeping the existing codebook and encoder assignments unchanged, the method reorganizes the codebook into a balanced binary tree and transmits branch bits in depth-major order, so that every bitstream prefix corresponds to a valid latent state. Only the decoder is additionally adapted to handle these intermediate states. The experiments report that a single bitstream provides thousands of operating points while achieving perceptual quality competitive with existing generative codecs in the ultra-low bitrate range.
Critical strengths
- Since every bit position in a single embedded bitstream yields a valid reconstruction state, more than 20,000 operating points are available for a single Kodak image. Transmission can be stopped at any time without re-encoding, and rate control is much finer than in existing progressive VQ methods such as ProGIC, whose granularity is tied to the number of RVQ stages.
- Below 0.06 bpp, the method achieves competitive LPIPS/DISTS results against several GIC codecs, including diffusion-based ones, showing that a progressive structure and perceptual quality can coexist. It also outperforms RDVQ-adj, a rate-adjustable method from the same family.
- The encoder and codebook are not retrained; only tree construction and decoder adaptation are added. The method uses fixed-length bits without entropy coding, and the paper transparently reports that arithmetic coding yields only a 0.36% gain.
- Although the core mechanism is the same as classical progressive VQ (Riskin et al., 1994), the paper's strength lies in its practical completeness: it makes this mechanism work with modern neural tokenizers and nonlinear decoders while achieving strong perceptual quality at ultra-low bitrates.
Critical weaknesses
Novelty
- The authors distinguish their work from Riskin et al. by stating that "the leaf codebook and encoder assignments come from a pretrained neural tokenizer and remain fixed." However, Riskin's method likewise keeps the codebook and full-search assignments fixed and constructs only the tree post hoc. Riskin's method also uses depth-major transmission, and stopping in the middle of a pass naturally produces the mixed-depth state described in Tree-VQ. Although Riskin does not explicitly discuss mid-pass truncation, the property is the same, since an intermediate codeword is displayed for each received bit. The only change is the source of the codebook (a neural tokenizer), which is a weak distinction in terms of tree construction.
- The first two contributions claimed in the introduction (and the corresponding key idea in the abstract), which are described in Sec. 3.2–3.3, are overstated. The balanced binary tree, index reassignment, retention of full search, pass-wise bit-by-bit transmission, reconstruction from an intermediate codeword at each bit, and top-down equal partitioning are all present in Riskin et al. The proof in App. C also follows directly from this structure. The contribution should be reframed as "applying a classical full-search progressive transmission tree to a neural tokenizer and adapting a nonlinear decoder to mixed-depth states."
Cost of progressive coding
The comparison with RDVQ-adj demonstrates competitiveness against a rate-adjustable method of the same architectural family. However, because the base tokenizers differ, the contribution and cost of the progressive structure itself are not isolated. Comparisons using the same tokenizer should be provided, for example: (a) the full-depth performance of the original decoder before adaptation, (b) non-progressive models trained per rate with the same tokenizer, and (c) RDVQ-adj-style partial transmission and completion applied to the same tokenizer.
Generality of the method
The paper's central claim is that it converts an existing pretrained tokenizer post hoc, but the method is applied only to a single tokenizer trained by the authors. Results on publicly available tokenizers (e.g., the VQGAN family) are needed to verify the generality of the method.
Model complexity
With about 186M parameters in total, the model is much larger than lightweight codecs such as ProGIC. A comparison and discussion of encoding/decoding time complexity should be included. FLOPs are also not reported.
Missing distribution-level metrics
The evaluation reports only LPIPS, DISTS, and CLIP-IQA, and no distribution-level metric such as FID is provided, even though most of the compared generative codecs report FID and App. E is titled "Realized Rate, Distributional Metrics, and Runtime Tables." Since realism is a key evaluation axis for ultra-low-bitrate generative compression, FID (e.g., following the patch-based protocol commonly used in this field) should be reported.
My rating may be adjusted depending on how the authors address and resolve the concerns raised in my review.
Questions for authors
- Riskin et al. (1994) also keep the codebook and full-search assignments fixed, construct the tree post hoc, and transmit bits in depth-major order. Beyond the source of the codebook (a neural tokenizer), what aspects of the tree construction and prefix representation are technically different from Riskin's method?
- Can the authors provide comparisons using the same tokenizer to isolate the contribution and cost of the progressive structure, specifically: (a) the full-depth performance of the original decoder before adaptation, (b) non-progressive models trained per rate with the same tokenizer, and (c) RDVQ-adj-style partial transmission and completion applied to the same tokenizer?
- Can the authors apply the post-hoc conversion to publicly available pretrained tokenizers (e.g., the VQGAN family) to demonstrate the generality of the method?
- Can the authors report encoding/decoding time and FLOPs, and compare them with the baseline codecs, including lightweight codecs such as ProGIC?
- Can the authors report a distribution-level metric such as FID?