- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Geometric Binary Quantization (GBQ) for visual tokenization. The authors' approach is motivated by the observation that existing lookup-free binary quantization methods (such as LFQ) typically constrain discrete codes to a fixed, axis-aligned binary hypercube and assign signs to each latent dimension independently, resulting in a fixed geometric structure for the quantizer itself. GBQ partitions the latent representation into small blocks and applies a learnable linear transformation to the full binary state space within each block, thereby constructing a structured local codebook, and performs precise quantization via block-wise nearest-neighbor search.
Critical strengths
The experiments are comprehensive. Reconstruction evaluation of the visual tokenizer was primarily conducted on ImageNet-1K, with comparisons made against LFQ, WeTok, and other visual tokenizers; additionally, zero-shot reconstruction was tested on MS-COCO, and generation experiments were performed using the BitDance framework.
Critical weaknesses
-
The paper overstates its novelty. As I understand it, GBQ can be viewed as PQ + SIMVQ + LFQ; specifically, the proposed GBQ essentially groups the LFQ codebook and introduces a linear layer within each group, similar to the approach in SimVQ. In other words, the GBQ formulation is very close to the existing SimVQ method, differing only by the addition of a grouped/product quantization structure and the specialization of SimVQ’s generic base codebook into the binary format used in LFQ. This issue affects Equations 6, 7, and 8; while the authors seem intent on highlighting the differences from SimVQ, anyone familiar with the field would recognize the underlying similarity.
-
Following up on the first point, the current experiments fail to address a more critical question: the paper lacks a "grouped SimVQ" baseline that directly matches GBQ, as well as a rigorously controlled comparison against product/grouped VQ methods that freely learn local code vectors. Without these comparisons, the performance gains attributed to this quantization method could simply stem from the use of SimVQ and a larger codebook, thereby significantly diminishing the authors' contribution.
-
I am particularly puzzled by the lack of visualizations. Typical VQ papers include numerous visualizations to demonstrate effectiveness; however, this paper offers very few visualizations for the quantization reconstruction component and none at all for the generation experiments. Furthermore, there is no appendix; notably, the authors have not explicitly stated that the code will be open-sourced, nor have they provided an appendix.
Questions for authors
My primary focus is on the relationship between GBQ and existing methods that combine grouped/product quantization with SimVQ-style codebook reparameterization. At a high level, GBQ partitions latent vectors into blocks and parameterizes each local codebook as a linear transformation of fixed binary states. Could the authors clarify the fundamental difference between this approach and independently applying SimVQ-style shared linear transformations within each group? More importantly, have you provided controlled experimental results comparing GBQ against such "grouped SimVQ" baselines, or against "grouped/product VQ" baselines that utilize identical block sizes, numbers of local candidates, backbone networks, and training budgets? Such comparisons are crucial for my assessment: determining whether the performance gains stem from the proposed binary geometric structure or simply from the introduction of local vector assignment and codebook reparameterization.
In LFQ-style tokenization, the binary identity corresponds directly to the quantized latent vector; in contrast, with GBQ, the generator predicts a binary identity , yet the decoder receives a transformed representation . Given that BitDance models and generates binary representations, and that GBQ deliberately alters the geometric structure associated with these binary identities. Could the authors discuss whether this geometric mismatch complicates downstream token generation? While current BitDance results demonstrate that GBQ remains compatible with the generator, the improvement in generation performance is relatively modest compared to the gains in reconstruction performance. Any analysis or existing evidence linking the learned geometric structure to downstream generation quality would help clarify whether GBQ enhances visual tokenization beyond mere reconstruction.
7 comments
As far as I am aware, ICLR allows for the submission of revised manuscripts during the rebuttal phase; authors may upload a revised version, incorporating the relevant additional content, before providing their response to reviewers. Visualization cannot be perceived within text.
Thank you for the helpful suggestion. We have added further qualitative reconstruction comparisons with WeTok, PQ, and Grouped SimVQ, along with representative GBQ generation samples, all included in Appendix E of the revised manuscript. We hope these additional visualizations help clarify the reconstruction improvements and provide a more complete qualitative evaluation.
We thank the reviewer for the detailed and insightful comments. We would like to clarify the motivation, design, and contributions of GBQ.
W1, W2 & Q1: Relation to SimVQ and Product Quantization
Thank you for your insightful comments. LFQ-based visual tokenizers such as WeTok enable efficient scalar assignment, but their fixed code geometry cannot adapt to encoder features, limiting reconstruction quality.
Inspired by SimVQ, we introduce learnable linear transformations of binary codes. Block-wise factorization further enables local coupling while preserving tractable exact assignment, extending independent scalar assignment to block-wise vector assignment.
To evaluate this design, we conducted controlled comparisons with PQ and Grouped SimVQ under matched settings at 50K training steps.
GBQ outperforms both PQ and Grouped SimVQ in PSNR and rFID.
We further compared GBQ and Grouped SimVQ using the diagnostics introduced in Fig. 1: Relative Quantization Residual, measuring relative code–feature distance, and STE–Reference Gradient Alignment, measuring the cosine similarity between STE and continuous-latent reference gradients.
GBQ achieves lower quantization residual and better gradient alignment, consistent with its reconstruction advantage.
We attribute this in part to the binary basis construction inherited from LFQ. Unlike randomly sampled bases, which may introduce sampling variability and uneven coverage, the complete binary state set provides a regular, symmetric, and deterministic structure. GBQ also uses a compact shared transformation instead of independently storing every code vector, facilitating codebook reuse, transfer, and block-wise modification.
More importantly, GBQ provides a precise geometric characterization of binary-variable coupling. The off-diagonal entries of the Gram matrix determine interactions between binary variables, while block size controls their scope. At block size one, GBQ naturally reduces to separable scalar assignment. This provides a controllable transition from scalar to coupled vector assignment while retaining tractable exact quantization.
We will further clarify the motivation and structural properties of GBQ in the revision.
W3: Appendix and Qualitative Results
Thank you for highlighting the importance of reproducibility and qualitative evaluation. We would like to respectfully clarify that our original submission already includes Appendices A–D, covering implementation details and reconstruction visualizations.
Appendix A provides the model architecture, training hyperparameters, and evaluation settings. Appendix D (Fig. 3) presents GBQ–WeTok reconstruction comparisons on ImageNet validation images, including fine structures, edges, and local textures.
Appendix B analyzes code geometry, assignment separability, representational capacity, and complexity, while Appendix C provides additional diagnostics, training analyses, and ablations.
We also appreciate your suggestion to include more visualizations. Since generation differences are relatively subtle, we previously focused on quantitative evaluation. We will complement these results with additional reconstruction comparisons and representative generated images in the revision.
Q2: Reconstruction Robustness and Downstream Generation
Thank you for raising this important question. During generation, predicted tokens may contain errors, affecting the images recovered by the tokenizer. Evaluating reconstruction under token perturbations therefore helps examine how learned code geometry may influence downstream generation.
We evaluated GBQ and WeTok by randomly flipping bits in their binary tokens. We sampled 2,000 ImageNet validation images and averaged results over three independently trained seeds per method. We tested four settings: no perturbation, independent bit flips at 1% and 5%, and exactly one random bit flip per token.
GBQ maintains higher reconstruction PSNR under all perturbation settings. Its reconstruction advantage thus persists even when binary tokens contain errors, providing further evidence that learned code geometry can benefit the decoding of imperfect token predictions in downstream generation.
Explicitly incorporating bit-flip robustness into tokenizer training is an interesting direction that we leave for future work.