- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces ChannelTok, a flexible-length visual tokenizer that treats each latent channel map, rather than each spatial location, as a visual token. It uses a lightweight VQGAN-style CNN–Transformer autoencoder, stochastic prefix masking to order channels from global structure to fine detail, and independent Binary Spherical Quantization for active channels; varying the retained prefix length then provides different compression levels without retraining and also supports variable-length autoregressive generation. Unlike spatial token dropping, ChannelTok preserves the full spatial grid inside every retained channel and progressively removes global feature dimensions.On ImageNet, ChannelTok achieves an rFID of 3.70 at 256 tokens while using 159M parameters and decoding 8.6× faster than FlexTok, whose rFID is 2.97. At the full 512-token budget, it reaches an rFID of 2.92 while retaining the same lightweight, single-pass decoder.
Critical strengths
- The channel-as-token perspective is fresh and interesting. Each token represents a full feature map rather than a local patch, moving beyond the standard spatial-token paradigm and enabling natural coarse-to-fine rate control.
- Random tail dropping encourages channels to organize by semantic importance, which is similar in spirit to PCA, although the hierarchy here is learned and nonlinear. Padding inactive channels also keeps the autoencoder tensors regular, making the design friendly to parallel computation.
- The experiments show a strong quality–efficiency trade-off. ChannelTok uses a lightweight, single-pass decoder to achieve reconstruction quality close to the best baseline while being much smaller and faster.
Critical weaknesses
- The evaluation is limited to ImageNet at 256×256 and does not really stress-test fine-detail preservation. Results on higher-resolution images or detail-sensitive cases such as small objects, rendered text, and documents would make the claim more convincing.
- The method does not actually choose based on the complexity of each image. It samples randomly during training and requires users to specify it at inference, so the method is flexible-length rather than truly content-adaptive;
Questions for authors
- How should the spatial feature-map size and the number of retained channels be balanced under a fixed bitrate or compute budget? Is there a principled way to choose the best and , rather than fixing the spatial size and tuning only the channel count?
2 comments
Thanks for the detailed responses. My concerns have been addressed and I would keep my original rating (3: Weak acceptance).
Response to Reviewer ZGgb
We're grateful for your suggestions and overall support for ChannelTok. Thank you for recognising its novelty and architectural compactness, and for appreciating our experiments. We address your concerns below.
Thank you for the suggestion! We agree that, from a deployment standpoint, a practical tokenizer should handle varying resolutions. We trained and evaluated at 256×256 to follow the standard protocol shared by all the methods we compare against, and with academic compute, this single full run was what we could complete for this submission. Supporting resolutions beyond 256×256 is something we have been thinking about for future versions, and the channel-wise formulation lends itself naturally to it, since the spatial size of each channel map h×w and the number of channels k are independent. Higher resolutions could therefore scale h×w while k continues to control the coarse-to-fine budget.
To stress-test fine detail, we have added qualitative reconstructions of scene text from TextOCR and ICDAR 2013 alongside small objects from COCO in Fig. 18, as well as tiled 512×512 reconstructions in Fig. 19 (App. G). Large text becomes legible from around 96–128 tokens and small objects sharpen progressively as channels are added.
The reviewer is right to point this out, and we're sorry for the misleading phrasing. We have revised "adaptive" to "flexible-length" throughout the paper. Reviewer A5CJ raised the same point, and we address it at length in our response to them, including the specific passages we revised. We kindly refer the reviewer there for more details.
This is an interesting question! In our design, we fix h×w as an empirical choice and vary k. This is similar to design choices in spatial tokenizers, which fix the channel dimension of the latent and vary the number of spatial tokens.
What makes this convenient is that, with h×w and the quantizer fixed, every token carries the same amount of information, one 16-bit BSQ index, so the bitrate budget and the token budget are tied together through k. The bitrate is 16k bits, and the token budget, which determines downstream AR compute, is k itself. Choosing k thus sets both at once.
We chose the per-token size by balancing two effects. A larger codebook packs more information into each token, which improves reconstruction at a fixed token budget. However, it also enlarges the embedding and output layers of any downstream generator. With 216 codes, each token carries 16 bits, the same as FlexTok, while our vocabulary intentionally stays well below those of modern LLMs, such as Llama 3 with 128k and Qwen3 with 152k, so that adding our tokens to such a model remains a moderate change.
Choosing h×w and k jointly for a given budget or image is a promising extension, but it is closer to adaptive allocation, which serves a different purpose. We wish to give the user predictable control over the budget. A sweep over h×w and codebook size under matched bitrates is something we look forward to exploring in future work.
Once again, thank you for your thoughtful review and support!