acceptodds
Under review as a conference paper at ICLR 2027

FeaTok: Feature-Loss Tokenization for Image Generation

Abstract

Discrete image generators learn the distribution of tokens produced by an image tokenizer. These token representations should be easy for generators to model, not merely enable faithful image reconstruction. Yet these tokenizers are commonly trained to reconstruct pixels, which prioritizes low-level fidelity and may disrupt representation structure useful for generation. Motivated by feature-predictive self-supervised learning, we introduce feature-loss tokenization (FeaTok), which trains discrete tokens by reconstructing pretrained features rather than pixels. Ablations across SigLIP, SigLIP2, DINOv2, and DINOv3 show better generative performance with FeaTok. In class-conditional generation, FeaTok advances the quality-compute Pareto frontier, reducing gFDr6 at by up to over the strongest discrete baselines and outperforming the strongest unguided continuous baseline (RAEv2) with less sampling compute. FeaTok also achieves higher human-preference reward-model scores than prior art without reward-based training and surpasses continuous and discrete text-conditional baselines on three benchmarks with only fine-tuning updates after pre-training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
image generation, discrete tokenizer

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_iVs1OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FeaTok, a discrete image tokenizer for image generation in which the discrete representation is shaped only by reconstruction losses to pretrained visual features rather than by pixel reconstruction. A feature decoder reconstructs multi-layer features from frozen SigLIP, SigLIP2, DINOv2, or DINOv3 encoders, while a pixel decoder is trained on stop-gradient tokens solely to render generated token sequences back into images. The method also discretizes the encoder’s global representation and generates this global token before the patch tokens, providing image-level context for autoregressive generation. Experiments on ImageNet 256×256 and 512×512 use BAR-style autoregressive generators with masked-bit modeling and compare against discrete and continuous baselines using gFDr6, gFID, reward-model scores, and sampling-cost measurements. The authors report that FeaTok improves over prior discrete models, reaches competitive quality with much lower sampling cost than continuous generators, and transfers to text-to-image generation after limited fine-tuning. Ablations study pixel, feature, and mixed supervision; feature depth; global-token augmentation; reconstruction quality; and token-structure statistics.

Critical strengths

Strong controlled empirical study of tokenizer objectives. The paper systematically compares pixel-only, feature-only, and mixed pixel-feature supervision across four pretrained encoders, while largely keeping the BAR-style generator fixed. This makes the central empirical claim—that feature-only supervision can improve generative learnability—more credible than a single-model SOTA comparison.

Strong quality-compute tradeoff. FeaTok-L-DINOv3 reaches gFDr6 3.13 on ImageNet 256×256 using only 1.27 TFLOPs per image, compared with RAEv2 at 3.95 using 47.72 TFLOPs without guidance. This substantial sampling-cost advantage is practically important for discrete image generation and multimodal systems, even though RAEv2 remains stronger under full guidance.

Critical weaknesses
  1. Shaping tokens with pretrained visual features has been explored in prior work such as DINO-Tok and TA-Tok, and the global-token idea has also appeared in Holistic Tokenizer. Thus, the contribution of this paper is more like a strong combination of existing ideas. The novelty is incremental and not sufficiently isolated from close prior work.

  2. The DINO/SigLIP encoders and reward models were pretrained on large external datasets, which may overlap with the evaluation images or prompts.

  3. More methods that use features from pretrained encoders as auxiliary training signals should be added, such as DINO-Tok and TA-Tok, to demonstrate the advantage of the proposed “feature-only objectives” over these methods. Comparisons should include both the main experiments and downstream text-to-image generation.

Questions for authors

The experiments show that although FeaTok achieves overall SOTA performance, it is weaker on the Inception and SigLIP2 sub-metrics, especially in the experiments corresponding to Tables 3 and 4. Have the authors analyzed this issue?

7 comments
AuthorsOpenReview ↗

Response to reviewer iVs1 (Part 1/5)

We thank the reviewer for recognizing our controlled tokenizer study and FeaTok’s quality–compute trade-off, achieving the SOTA performance.

We address the concerns of the reviewer in the following bullet points.

W1: Novelty

We would like to use the rebuttal to clarify our novelty. FeaTok argues that pixel losses should not shape the discrete tokens during the tokenizer training for image generation. Feature losses but not pixel losses should be the loss to shape the discrete tokens, and a separate pixel decoder is trained with pixel losses using detached discrete tokens. This approach is different from the prevailing design choice shared by recent models, including DINO-Tok [2] and BAR, which assumes image reconstruction losses are beneficial to train useful discrete tokens for generation. TA-Tok [3] uses feature supervision alone to train its semantic tokens, but for the generation task, it still employs a separate generative de-tokenizer to obtain VQ-VAE style discrete tokens shaped by pixel losses. Our work demonstrates that pixel losses are not necessary and can be detrimental to shape discrete tokens in image generation tasks where we care about image details not only semantics. We find the results surprising that achieving state-of-the-art image generation from discrete tokens does not require the tokenizer encoder to leverage pixel-level reconstruction losses.

We do not claim the general idea of global conditioning as novel. As acknowledged in the manuscript, prior work already follows global-to-local generation strategies. We will revise the manuscript to make this distinction explicit. Our main contribution is learning discrete tokens through feature losses without allowing pixel reconstruction gradients to shape them. Global conditioning is a natural addition to this framework because the pretrained visual encoder already provides an image-level representation. RCG [10] and DLC [11] demonstrate the benefits of representation conditioning, while FlexTok [12] uses a related coarse-to-fine ordering of visual tokens. Our implementation requires little additional machinery. Hita [1] learns holistic representations through attention between learnable queries, VQGAN patch embeddings, and DINOv2 features, then fuses the quantized holistic and patch representations before image decoding. FeaTok instead directly quantizes the pretrained encoder’s existing global feature with one additional quantizer. The resulting token conditions patch generation, while the image decoder receives only the patch codes. At the system level, FeaTok-L achieves lower reported gFID with a smaller generator than both Hita variants listed in Table iVs1-2.

AuthorsOpenReview ↗

Response to reviewer iVs1 (Part 2/5)

W2: Overlap in evaluation

We acknowledge that overlap between external pretraining data and evaluation images or prompts may exist and cannot be easily ruled out, and we will clarify this limitation in the revision. Our use of pretrained visual representations follows common practice in the literature. Our baselines also use pretrained DINO or SigLIP representations, including BAR [6], REPA [7], RAE [8], and RAEv2 [9]. FeaTok achieves lower gFDr6 than BAR-L and RAE at both 256×256 and 512×512, with and without guidance. FeaTok also achieves better reward scores compared to these baselines.

We also tried to minimize the bias introduced by the data overlaps from pretrained visual features.

First, we report three human-preference reward models with different backbones. PickScore and HPSv2 are fine-tuned from CLIP, while HPSv3 uses Qwen2-VL-7B. FeaTok-L-DINOv3 scores higher than BAR-L, RAE, and RAEv2 on all three metrics (Fig. 5 of the manuscript). In particular, its HPSv3 score is 4.38, compared with 3.69, 3.76, and 4.31, respectively, showing that the preference gains also hold with a different reward-model backbone.

Second, we report gFDr4 by excluding DINOv2 and SigLIP2 from the six-space average and expand the comparison in Tables iVs1-1a and iVs1-1b. At 256×256, FeaTok-L-DINOv3 has the lowest unguided gFDr4 among the listed methods. With guidance, it remains better than BAR, REPA, REPA-E, and RAE, while RAEv2 is better similar to the gFDr6 metric. At 512×512, FeaTok-L-DINOv3 has the lowest gFDr4 among the available comparisons in both guidance modes. Thus, the advantage over BAR and RAE persists after removing DINOv2 and SigLIP2 from evaluation. We will revise the manuscript to include a discussion.

Table iVs1-1a. ImageNet-256 gFDr4, averaging the normalized Fréchet distances in Inception-v3, ConvNeXt V2, MAE, and CLIP spaces. Sampling configurations follow manuscript Tables 1 and 2. Guided continuous-baseline scores use the published per-space values in [9]. Lower is better. Bold marks the best score in each column.

ModelWithout guidance ↓With guidance ↓
BAR-B [6]4.073.85
BAR-L [6]3.673.37
SiT-XL/2-REPA [7]9.204.64
REPA-E-XL/13.682.70
RAE-XL [8]4.323.41
RAEv2-XL [9]3.912.32
FeaTok-B-DINOv32.922.77
FeaTok-L-DINOv32.572.57

Table iVs1-1b. ImageNet-512 gFDr4 in the same four feature spaces, using manuscript Table 4 sampling configurations. Lower is better. Bold marks the best score in each column.

ModelWithout guidance ↓With guidance ↓
BAR-L [6]2.762.39
RAE-XL [8]2.922.18
FeaTok-L-DINOv31.741.51
AuthorsOpenReview ↗

Response to reviewer iVs1 (Part 3/5)

W3: Additional baselines

Thank you for pointing us to more baselines. Table iVs1-2 compares the methods at 256×256, without and with guidance. We use Hita’s reported FID and DINO-Tok’s published gFID. We computed Hita and DINO-Tok’s gFDr6 with the authors’ sampler and the released weights. We also invite the reviewer to consult Table fGxQ-2, where we added AliTok, EOSTok, and other gFID baselines in response to another reviewer’s request.

Table iVs1-2. Class-conditional ImageNet-256 comparisons with Hita and DINO-Tok. Paired entries are without / with guidance. Hita and DINO-Tok gFID are reported results, while Hita and DINO-Tok gFDr6 are measured by us with released weights and samplers. Lower is better.

ModelGenerator paramsgFID ↓gFDr6 ↓
Hita-V-2B, paper2.0B— / 2.5912.95 / 6.54
Hita-U-2B, repository2.0B— / 1.82 (1.69 with CFG-free guidance)4.35 / 3.87
DINO-Tok VQ-XXL1.4B reported— / 2.4416.33 / 10.08
FeaTok-B-DINOv30.4B1.34 / 1.073.58 / 3.39
FeaTok-L-DINOv31.1B1.15 / 1.053.13 / 3.08

For TA-Tok, there are no results reported using the class-conditional ImageNet setup, and no checkpoint of this ImageNet setup is released. We used their released tokenizer and trained a base-size generative model on 169-token sequences using the same 250K-step recipe as our patch-only ablation. Since they use a VQ-style discretizer instead of FSQ as ours, we tested both a categorical head and a bitwise head. Table iVs1-3 reports the best measured 50K-sample result for each head. The categorical head reaches gFID 5.27 and gFDr6 7.95, while the bitwise head reaches 6.94 and 10.00. FeaTok-DINOv3 with the patch-only BAR-B generator achieves 1.48 and 4.28.

Table iVs1-3. Class-conditional ImageNet-1k generative performance of TA-Tok following our training recipe, compared with patch-only FeaTok-DINOv3. Both TA-Tok rows use de-tokenizer CFG 4.0 following the released setting. The categorical head uses T=1.0, top-k=1200 and top-p=0.95. The bitwise head uses T=0.5 and generator CFG 5.0. Both are tuned during inference. Lower gFID and gFDr6 are better.

ModelGenerator + generative de-tokenizerGuidancegFID ↓gFDr6 ↓
TA-Tok + BAR-B, categorical head0.3B + 0.65BDe-tokenizer only5.277.95
TA-Tok + BAR-B, bitwise head0.4B + 0.65BGenerator + de-tokenizer6.9410.00
FeaTok-DINOv3 + BAR-B, patch-only0.4B + N/AOff1.484.28

We also evaluated reconstruction on all 50K ImageNet validation images using the reconstruction evaluation protocol of Table 12 in our manuscript and TA-Tok’s released 256 px de-tokenizer at CFG 4.0. Table iVs1-4 shows that FeaTok reconstructs more faithfully even when TA-Tok uses its native, denser token grid.

Table iVs1-4. ImageNet-256 reconstruction on 50K validation images using the protocol of Table 12 in our manuscript. TA-Tok uses its released de-tokenizer at CFG 4.0 and either 169 or 729 tokens. Lower rFID and rFDr6, and higher PSNR and SSIM, are better.

TokenizerTokensrFID ↓rFDr6 ↓PSNR ↑SSIM ↑
TA-Tok1694.927.0013.000.318
TA-Tok7293.726.3615.560.417
FeaTok-DINOv32560.992.0817.450.424

For text-to-image generation, FeaTok leads BAR and RAEv2 after fine-tuning (Table 5 of our manuscript). As for Tar, the t2i model released with TA-Tok, Table iVs1-5 shows that our method also exceeds Tar-1.5B and approaches Tar-7B on both shared benchmarks, using a smaller tokenizer-training corpus and a direct pixel decoder. We are training a matched TA-Tok model using FeaTok’s generator backbone, data, and schedule, and will post its results if they are ready before the end of the discussion.

Table iVs1-5. Text-to-image system comparison on GenEval and DPG-Bench. Tar scores are reported results. FeaTok scores are after fine-tuning. Tokenizer, de-tokenizer, and generator training data differ across systems. Higher scores are better.

SystemTokenizer / generative de-tokenizer dataGenerator training dataGenEval ↑DPG-Bench ↑
Tar-1.5B, reported200M / 50M LAION images100M mixed samples, including 23M text–image pairs0.7682.96
Tar-7B, reported200M / 50M LAION images100M mixed samples, including 23M text–image pairs0.8484.19
FeaTok, BAR-L + Qwen3-0.6B1.28M ImageNet images / —~36M BLIP-3o images (Long-Caption, Short-Caption, JourneyDB), then BLIP-3o-60K0.83783.4
Reviewer_t36pOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FeaTok, a discrete image tokenizer that addresses the mismatch between reconstruction fidelity and generative learnability. Instead of optimizing tokens for pixel reconstruction, FeaTok learns discrete representations by reconstructing pretrained visual features, while training a separate pixel decoder on detached tokens. It also introduces a quantized global token to guide autoregressive patch generation. Experiments on ImageNet-1K show that FeaTok outperforms existing discrete generators and achieves better unguided generation quality than RAEv2 with 38× less sampling compute. The authors demonstrate that feature-based supervision can produce more generation-friendly discrete representations without pixel reconstruction losses shaping the tokens.

Critical strengths
  1. Meaningful insight into tokenizer design. The paper shows that feature-only supervision can improve generation quality despite lower reconstruction fidelity, challenging the conventional pixel-reconstruction objective.

  2. Well-controlled ablations. Comparisons across four pretrained encoders and different supervision objectives help isolate the benefits of feature-based tokenization and global conditioning.

  3. Strong quality–compute trade-off. FeaTok achieves better unguided gFDr6 than existing discrete baselines and RAEv2 on ImageNet 256×256, using approximately 38× less sampling compute than RAEv2.

Critical weaknesses
  1. Insufficient evidence for generative learnability. The paper attributes its improvements to more learnable token distributions, but Figure 3b only provides a component-wise spatial mutual information proxy. Higher spatial dependence does not necessarily imply easier autoregressive modeling. Generator training curves or entropy-aware prediction diagnostics would provide more direct evidence for this central claim.

  2. Unclear contribution of encoder regularization. Equation (2) includes an encoder regularization loss that preserves pretrained features, in addition to the feature reconstruction objective. However, the paper does not isolate its contribution. An ablation without this regularizer would help determine whether the improvements primarily come from feature reconstruction or pretrained representation preservation.

  3. Limited analysis of global token conditioning. Figure 3c demonstrates improvements from global augmentation, but it remains unclear whether these gains arise from the proposed global-to-local modeling strategy or simply from introducing an additional image-level latent variable. Controls using shuffled global tokens or alternative global conditioning mechanisms would help clarify its contribution.

  4. Reconstruction degradation limits the broader claims. Table 12 shows substantial reconstruction degradation under feature-only supervision, particularly for DINOv3 (PSNR 20.61 → 17.45; SSIM 0.545 → 0.424). While this trade-off benefits generation, its implications for the proposed unified multimodal applications remain unexplored. Evaluations on fine-grained visual tasks or a clearer discussion of these limitations would strengthen the paper.

Questions for authors

See weakness.

5 comments
AuthorsOpenReview ↗

Response to reviewer t36p (Part 1/3)

We thank the reviewer for acknowledging our insights on tokenizer design, the well-controlled ablations, and our strong empirical results, particularly on the quality-compute pareto fronts.

We respond to the weaknesses raised by the reviewer below.

W1: Insufficient evidence for generative learnability

Thank you for raising this point and for suggesting additional analyses. We define generative learnability as how easily a generator fits the image distribution at a given budget (Sec. 1, line 57), and we measure it the way similar to how self-supervised representations are evaluated: by downstream performance (gFDr6) under a fixed protocol rather than by an intrinsic statistic. For discrete token image generation, the protocol is the sample quality of a fixed generator (BAR-B, 250K steps, fixed sampler) trained on the image tokens, presented as the controlled ablation of Fig. 3a, and Fig. 3b is a diagnostic.

As suggested by the reviewer, we add two more diagnostics related to learnability. First, we pulled out the in-training evaluation logs for every pixel vs. feature generator pair of Fig. 3a (10K samples, no guidance, random generation order, so values sit above the 50K numbers). Table t36p-1 reports the gFID gap between the feature trained tokens and pixel trained tokens, where negative means the feature tokens are better. The learning advantage appears early and holds through training. Feature tokens lead at every step for DINOv2 and DINOv3 (by 6.3 gFID after 50K steps for DINOv3), overtake pixel tokens at 150K steps for SigLIP, and SigLIP2 is the one encoder where pixel tokens stay ahead, the same exception as in Fig. 3a.

Table t36p-1. In-training gFID differences between feature-trained and pixel-trained tokens at matched training steps. Evaluations use 10K samples, no guidance, and random generation order. Negative values favor feature-trained tokens.

Training steps50K100K150K200K250K
SigLIP+0.46+0.12−0.16−0.10−0.14
SigLIP2+1.41+0.67+0.33+0.16+0.15
DINOv2−0.83−0.50−0.69−0.89−1.53
DINOv3−6.29−2.68−2.02−4.17−5.47

Second, we add an entropy-aware prediction diagnostic following the reviewer’s suggestion. For each tokenizer, a baseline predicts the bits of a target patch token using only its class and spatial position. A matched MLP predictor additionally receives a neighbouring token. We evaluate both on 25K images held out from probe fitting and compute the normalized prediction gain: 100 × (L_base − L_context) / L_base, where L_base and L_context are their bit-prediction cross-entropies. We average this percentage over the four immediate-neighbour directions. A higher score means that neighbouring tokens make the target bits easier to predict. This is a predictor-dependent measure of local predictability. Table t36p-2 shows that feature supervision raises this score for the three encoders whose gFID improves and lowers it for SigLIP2, whose gFID worsens. Together, these diagnostics provide complementary evidence that helps explain the generation-quality gains observed in our controlled ablations.

Table t36p-2. Normalized prediction gain and patch-only generation quality for pixel versus feature supervision. The gain is 100 × (L_base − L_context) / L_base, averaged over four immediate-neighbour directions using MLP probes and 25K images held out from probe fitting. Higher prediction gains and lower gFID are better.

encoderNormalized prediction gain, pixel → featurepatch-only gFID, pixel → feature
SigLIP1.46% → 1.61%1.88 → 1.70
SigLIP21.11% → 1.01%1.68 → 1.80
DINOv21.88% → 3.21%1.76 → 1.63
DINOv31.80% → 3.98%2.92 → 1.48
AuthorsOpenReview ↗

Response to reviewer t36p (Part 2/3)

W2: Unclear contribution of encoder regularization

We follow previous work in regularizing the encoder when it is fine-tuned [1, 2], and the pixel baselines in Table 11 use the same regularizer, so the pixel-versus-feature comparison in the paper already holds it fixed. To isolate its contribution, we trained two more tokenizers using DINOv3 pretrained features and the corresponding generative models during the rebuttal. One tokenizer is trained with pixel losses only, and the other with feature losses only, both without the regularization term in the loss. Table t36p-3 reports the generative results (gFID and gFDr6) of the corresponding generative models. With the pixel loss, without using the regularizer degrades the generation by a big margin (gFID 4.73). Switching to feature loss, still without regularization, the learned tokens generate far better than the pixel variant without regularization (gFID 1.64 compared to 4.73). The regularizer then adds a small gain (1.48 patch-only, and 1.39 against 1.45 with the global token). Going from the unregularized pixel tokenizer to FeaTok, 95% of the gFID gain comes from the feature loss and 5% from the regularizer, and the split is similar for gFDr6.

Table t36p-3. DINOv3 tokenizer objective and encoder-regularization ablations with BAR-B generation. We report gFID / gFDr6 without guidance. The final column additionally enables global-augmented generation. Lower is better.

targetPixel, no reg, patch-onlyFeature, no reg, patch-onlyFeature, + reg, patch-onlyFeature, + reg, global augmented
gFID / gFDr64.73 / 10.571.64 / 4.551.48 / 4.281.39 / 4.13

W3: Limited analysis of global token conditioning

To test whether the global token helps because of the global-to-local order or simply because it adds an image-level latent variable, we trained three additional generators during the rebuttal, with the DINOv3 tokenizer, the BAR-B generator and the sampler fixed, the same as the FeaTok-B-DINOv3 generator in Table 1 of the manuscript. In the first generator variant, the global token is replaced by the global token of a random other image of the same class (noted as shuffled in Table t36p-4), which keeps the class information but removes the image-specific content. In the second and third, the real global token is added after half of the patch tokens or after all of them, which breaks the global-to-local order. Table t36p-4 shows that FeaTok performs the best among these alternatives. The shuffled token retains about 23% of the gFDr6 improvement over patch-only generation. Both global-last and global-middle perform worse than patch-only. During training, predicting the global token still updates the same network that predicts the patches. However, the token cannot condition any patches in global-last and can condition only the remaining half in global-middle. We suspect that learning this additional task makes patch prediction harder, without the full conditioning benefit of generating the global token first. Consistent with this explanation, a global-last control with the global prediction loss disabled closely matched the patch-only training-loss curve. These results support the importance of providing image-specific global information before patch generation.

Table t36p-4. DINOv3 global-token controls with BAR-B generation. Shuffled global tokens come from another image of the same class. Global-middle and global-last change when the global token is generated relative to the patch tokens. Lower gFID and gFDr6 are better.

Metricpatch-onlyFeaTok with global tokenshuffled globalglobal-lastglobal-middle
gFID1.481.331.462.062.20
gFDr64.284.024.225.495.71
AuthorsOpenReview ↗

Response to reviewer t36p (Part 3/3)

W4: Reconstruction degradation limits the broader claims

Thank you for raising this important point. We acknowledge that our tokenizers have lower pixel-reconstruction metrics such as PSNR and SSIM, as presented faithfully in the Appendix. Unified multimodal models solve image generation and understanding at the same time, and how well semantic tokens like ours serve both tasks in such a model remains an open question. From the literature, the lower pixel-reconstruction metrics doees not necessarily limit the unified model performance including image understanding, for two reasons. First, reconstructing the features of a pretrained encoder rather than its pixels is an established way to build discrete tokens for image understanding. It is used from masked image modeling with BEiT v2 [3] to multimodal LLMs such as SEED [4] and TA-Tok [5]. Second, pixel reconstruction performance does not correlate with understanding performance in the literature. In ILLUME+ [8], a semantic encoder with much worse reconstruction than a pixel encoder (rFID 5.48 against 2.08, PSNR 15.7 against 21.9) scores 16 points higher on MMBench (46.9 against 30.7). In VQRAE [2], the variant with the best reconstruction collapses on understanding (MMBench 22.3 against 64.9). A controlled study of tokenizers in unified multimodal models finds that reconstruction fidelity and multimodal learnability can diverge in both directions: a tokenizer with better rFID (0.51 against 0.81) scores lower on VQAv2, while one with worse rFID (2.23 against 1.86) scores higher (61.28 against 57.21) [7]. For feature reconstruction, a recent study likewise finds that tokens with an order of magnitude larger quantization error can still give better understanding [6].

Since FeaTok already improves image generation using semantic tokens, we believe that training a unified model on its tokens is a promising direction. We will mention this question in a discussion section on how FeaTok can be used to train a unified multimodal model.

[1] Chen, B., et al. (2026). Aligning visual foundation encoders to tokenizers for diffusion models. ICLR. https://proceedings.iclr.cc/paper_files/paper/2026/hash/c7138635035501eb71b0adf6ddc319d6-Abstract-Conference.html

[2] Du, S., et al. (2026). VQRAE: Representation quantization autoencoders for multimodal understanding, generation and reconstruction. CVPR. https://openaccess.thecvf.com/content/CVPR2026/html/Du_VQRAE_Representation_Quantization_Autoencoders_for_Multimodal_Understanding_Generation_and_Reconstruction_CVPR_2026_paper.html

[3] Peng, Z., et al. (2022). BEiT v2: Masked image modeling with vector-quantized visual tokenizers. arXiv:2208.06366. https://arxiv.org/abs/2208.06366

[4] Ge, Y., et al. (2024). Making LLaMA SEE and draw with SEED tokenizer. ICLR. https://proceedings.iclr.cc/paper_files/paper/2024/hash/97011c648eda678424f9292dadeae72e-Abstract-Conference.html

[5] Han, J., et al. (2025). Vision as a dialect: Unifying visual understanding and generation via text-aligned representations. NeurIPS. https://proceedings.neurips.cc/paper_files/paper/2025/hash/e87b1e06be8c3594c810e8991e77ea40-Abstract-Conference.html

[6] Zou, J., et al. (2026). Can discrete tokens serve as semantically lossless representations of visual signals? arXiv:2610.37662. https://arxiv.org/abs/2610.37662

[7] Li, S., et al. (2026). Studying image tokenizers as visual languages in unified multimodal models. arXiv:2609.09143. https://arxiv.org/abs/2609.09143

[8] Huang, R., et al. (2026). ILLUME+: Illuminating unified MLLM with dual visual tokenization and diffusion refinement. CVPR. https://arxiv.org/abs/2504.01934

Reviewer_fGxQOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes learning discrete representations exclusively with a feature loss (FeaTok), without relying on pixel losses for that. The goal is to reconstruct frozen visual features that come from good encoders that were pre-trained in a self-supervised manner. The key design choices are (1) the target for the predicted visual features, which is the average of four encoder layers, (2) training a pixel decoder jointly (but having stopgrad so that it does not change the token distribution) and (3) the encoder's global representation (quantized to be a discrete representation) serves as the prior for the AR generator. The method achieves SOTA results compared to other discrete tokenizers on ImageNet-1k 256 and 512 resolutions, while it beats most recent continuous baselines using less sampling FLOPs.

Critical strengths
  • The idea is simple, interesting, and well-motivated at the beginning of the paper.
  • The method achieves strong results on both class-conditional ImageNet and text-to-image generation, where it outperforms relevant and/or recent discrete and continuous baselines.
  • I like that multiple strong SSL encoders have been ablated, as well as pixel, mixed and feature targets.
Critical weaknesses
  • The 512x512 class-conditional ImageNet-1k generation has only two baselines: BAR-L and RAE. There are several recent and very competitive discrete baselines that could be added such as AliTok (ICLR 2026) [1] and EOSTok (ICML 2026 Spotlight) [2]. For example, AliTok reports the performance on both 256 and 512 class-conditional ImageNet generation and they have the weights and code released. Also, EOSTok reports gFID of 1.48 without CFG. Could you add these baselines in Table 1 and 4?
  • To my understanding, the sampling settings for the baselines in Table 1 are optimized for gFID, but the ones for FeaTok are selected for gFDr6. You mention in the caption of Table 1 taht "Baselines use their released sampling settings, and App. F reports FeaTok at gFID-selected setting". Could you tune at least BAR-L [3] for gFDr6 (in the same way you did it for FeaTok) and report the result for comparison?
  • Could you add WeTok (ICLR 2026) [4] in the Related Work section (i.e., for proposing the noise-conditioned decoder that allows for better reconstruction at lower compression rates)?

[1] Wu, P., Zhu, K., Liu, Y., Tang, L., Yang, J., Peng, Y., ... & Zha, Z. J. (2026, April). Towards sequence modeling alignment between tokenizer and autoregressive model. In International Conference on Learning Representations (Vol. 2026, pp. 132194-132217). [2] Chu, W., Zhang, B., Han, J., Li, Y., Yang, L., Yue, Y., & Guo, Q. (2026). End-to-end autoregressive image generation with 1d semantic tokenizer. arXiv preprint arXiv:2605.00503. [3] Yu, Q., Liu, Q., He, J., Zhang, X., Liu, Y., Chen, L. C., & Chen, X. (2026). Autoregressive image generation with masked bit modeling. arXiv preprint arXiv:2602.09024. [4] Zhuang, S., Guo, Y., Wang, F., Fu, C., Huang, Z., Tian, Z., ... & Wang, Y. (2026, April). Wetok: Powerful discrete tokenization for high-fidelity visual reconstruction. In International Conference on Learning Representations (Vol. 2026, pp. 6284-6314).

Questions for authors

See weaknesses section.

4 comments
AuthorsOpenReview ↗

Response to reviewer fGxQ (Part 1/2)

We thank reviewer fGxQ for their thoughtful and detailed review. We are happy that the reviewer found our simple, interesting and well-motivated approach achieves strong results on both class-conditional ImageNet and text-to-image generation compared to relevant discrete and continuous baselines. We also appreciate the reviewer’s recognition of our detailed ablations across multiple strong SSL encoders.

We address the concerns of the reviewer in the following bullet points.

  • Thank you for suggesting more baselines. AliTok [1] released checkpoints only at 256x256 resolution and EOSTok [2] released none, so neither can be evaluated with gFDr6 at 512x512 resolution. At 256x256 we evaluated AliTok's released checkpoints ourselves using both their released samplers and samplers tuned for gFDr6. Table fGxQ-1 compares these results with FeaTok-B/L-DINOv3. Both FeaTok sizes retain lower gFDr6 than AliTok-L and AliTok-XL, with and without guidance. We will add these rows to Table 1 of our manuscript.

Table fGxQ-1. ImageNet-256 gFDr6 with released and gFDr6-tuned samplers. Released denotes AliTok's published sampling settings. Tuned denotes the best gFDr6 setting found in our search. Lower is better.

ModelReleased (w/o guidance)Released (w guidance)Tuned (w/o guidance)Tuned (w guidance)
AliTok-L [1]8.426.247.765.48
AliTok-XL [1]7.226.416.615.26
FeaTok-B-DINOv3——3.583.39
FeaTok-L-DINOv3——3.133.08

We also directly compare the gFID with these additional baselines at both 256 and 512 resolutions in Table fGxQ-2. FeaTok achieves the best gFID among these baselines at both 256 and 512 resolution, with and without guidance.

Table fGxQ-2. Reported class-conditional ImageNet gFID at 256×256 and 512×512, without and with guidance. FeaTok uses gFID-selected sampling settings. The default guidance type is classifier-free guidance. gFID is the lower the better.

methodgenerator params256x256 gFID (w/o guidance)256x256 gFID (w guidance)512x512 gFID (w/o guidance)512x512 gFID (w guidance)
DINO-Tok VQ XXL (2027) [3]1.4B-2.44--
EOSTok-L (2026) [2]312M1.741.35 (autoguidance)1.98-
EOSTok-H (2026) [2]644M1.481.38 (autoguidance)--
AliTok-L (2026) [1]318M1.981.38-1.39
AliTok-XL (2026) [1]662M1.881.28--
FeaTok-B (DINOv3)0.4B1.341.07--
FeaTok-L (DINOv3)1.1B1.151.051.601.21
AuthorsOpenReview ↗

Response to reviewer fGxQ (Part 2/2)

  • Following the suggestion, we tuned the sampling settings of the released BAR-B and BAR-L for gFDr6 with the same procedure as for FeaTok, and reported them in Tables fGxQ-3 and fGxQ-4 next to all FeaTok variants at their manuscript Table 1 settings. FeaTok with SigLIP2, DINOv2 or DINOv3 matches or improves on tuned BAR in every setting, and FeaTok-DINOv3 improves on it in all four. The exception is FeaTok-SigLIP, similar to the results in the other settings already presented in the manuscript. We will add Tables fGxQ-3 and fGxQ-4 to the appendix discussion of sampling selection.

Table fGxQ-3. ImageNet-256 generation without guidance. Sampling hyperparameters are tuned using the gFDr6 metric. Columns report gFDr6 and its six normalized Fréchet-distance components. Lower is better.

modelgFDr6incep.cvnx2dino2maesiglip2clip
BAR-B, gFDr6-best4.461.331.234.085.348.306.47
FeaTok-B-SigLIP4.461.580.894.495.868.005.94
FeaTok-B-SigLIP24.001.420.983.794.877.525.41
FeaTok-B-DINOv24.400.921.364.124.938.316.74
FeaTok-B-DINOv33.581.090.823.304.626.505.15
BAR-L, gFDr6-best3.571.320.873.144.266.205.62
FeaTok-L-SigLIP4.001.490.803.945.097.125.56
FeaTok-L-SigLIP23.451.110.823.374.046.295.11
FeaTok-L-DINOv23.470.870.833.114.136.335.52
FeaTok-L-DINOv33.130.960.682.954.005.574.62

Table fGxQ-4. ImageNet-256 generation with guidance. Sampling hyperparameters are tuned using the gFDr6 metric. Columns report gFDr6 and its six normalized Fréchet-distance components. Lower is better.

modelgFDr6incep.cvnx2dino2maesiglip2clip
BAR-B, gFDr6-best3.951.310.883.434.877.196.02
FeaTok-B-SigLIP4.141.280.753.965.517.715.63
FeaTok-B-SigLIP23.461.300.863.094.146.295.05
FeaTok-B-DINOv23.561.150.893.004.216.395.71
FeaTok-B-DINOv33.391.060.703.004.436.284.91
BAR-L, gFDr6-best3.291.460.922.673.915.435.33
FeaTok-L-SigLIP3.921.520.853.804.827.015.54
FeaTok-L-SigLIP23.271.120.782.983.756.074.94
FeaTok-L-DINOv23.200.950.742.693.775.795.28
FeaTok-L-DINOv33.081.090.822.933.735.284.65
  • Thank you for suggesting this paper. We are revising our manuscript and will add WeTok [4] to the related work. WeTok complements our related work by introducing a noise-conditioned generative decoder, which its authors motivated by the ambiguity of reconstruction at high compression ratios.

[1] Wu, P., et al. (2026). Towards sequence modeling alignment between tokenizer and autoregressive model. ICLR. https://openreview.net/pdf?id=GT3obnZ5Fk

[2] Chu, W., et al. (2026). End-to-end autoregressive image generation with 1D semantic tokenizer. ICML. https://proceedings.mlr.press/v306/chu26b.html

[3] Jia, M., et al. (2027). DINO-Tok: Adapting DINO for visual tokenizers. WACV (accepted). https://arxiv.org/abs/2511.20565

[4] Zhuang, S., et al. (2026). WeTok: Powerful discrete tokenization for high-fidelity visual reconstruction. ICLR. https://proceedings.iclr.cc/paper_files/paper/2026/hash/0b2a58f2b5943b9c356f10db32073275-Abstract-Conference.html

Thanks for the additional (extensive) experiments, they address my concerns. Just one small observation in Table fGxQ-2 to report the number of params in M for FeaTok-B (DINOv3) (similar to the all other models with less than 1B params). Regardless, I'm going to raise my score from 2 to 3. Good luck for the rest of the rebuttal!

Thank you very much for your response. We appreciate your detailed review and constructive feedback. For the camera ready, we will change the number of params to M for models with size less than 1B params. Thank you again for your review!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.