1. Limited methodological novelty and insufficient investigation of the underlying mechanisms.
The main innovations largely consist of combining and modifying existing techniques rather than introducing a fundamentally new modeling approach.
The shared-encoder continuous/discrete design closely follows previous unified visual tokenization frameworks, such as MANZANO[1]. The main tokenizer contribution consists of applying caption supervision and masked feature prediction before quantization, while retaining existing post-quantization objectives.
More importantly, directly supervising pre-quantization features is not a fresh idea in visual tokenization. For instance, DualToken[2] applies a self-distillation objective to the second-to-last layer features before quantization to preserve semantic representations. UniWeTok[3] also explicitly investigates supervision before and after quantization and introduces Pre-Post Distillation to improve representation learning. The authors should discuss these related works and clarify how their approach differs.
Besides, moving these objectives to the pre-quantization latent is a reasonable design choice, but the paper does not provide a sufficiently deep explanation of why this placement fundamentally improves discrete representation learning. This would also help distinguish the paper from prior works with similar technical contributions.
In particular, gradients from post-quantization objectives already propagate to the upstream encoder through STE. It remains unclear whether the improvements primarily arise from avoiding gradient approximation through quantization, introducing an additional optimization pathway, changing the distribution of supervision across features, or providing stronger visual reconstruction signals.
Similarly, TAN largely builds upon existing text-scale-aware normalization techniques. Its principal extension is caption-supervised vocabulary adaptation followed by transferring the resulting embeddings into the LLM. While useful, the conceptual distinction from existing normalization and vocabulary adaptation strategies is modest.
Overall, the contribution currently appears closer to a carefully engineered combination of existing techniques than a substantially new representation-learning or multimodal modeling paradigm. A more convincing analysis of the underlying mechanisms and a clearer distinction from closely related prior work would strengthen its significance.
2. Insufficient experimental evidence
The tokenizer evaluation in Table 3(a) relies primarily on PSNR, SSIM, and code-use perplexity. These measurements are useful but insufficient to comprehensively characterize the quality of visual tokens for autoregressive image generation.
In particular, code-use perplexity only measures the diversity of token utilization and does not directly establish better semantic organization, information preservation, or autoregressive predictability. Likewise, PSNR and SSIM are limited measures of perceptual reconstruction quality, especially when the reconstruction process involves a pretrained generative decoder. Additional metrics such as rFID, LPIPS, and token prediction performance would provide a more complete picture.
Moreover, Table 3 does not directly isolate the contribution of pre-quantization caption supervision under a matched unified-training setting. All its variants already include both pre- and post-quantization caption supervision, while the largest downstream improvements from the full tokenizer and TAN are demonstrated using substantially simplified training configurations.
Additionally, the headline results involve extensive multi-stage training, a pretrained diffusion-based image decoder, and large-scale internal and synthetic datasets. The comparisons in Tables 1 and 2 therefore do not isolate the contribution of the proposed techniques from differences in training data, decoder quality, and computational resources. Table 3(c) provides useful component-level evidence, but it does not include a complete factorial analysis of pre-caption, pre-MIM, D2C, and TAN, making it difficult to attribute the final performance gains precisely.
Consequently, while the empirical results are impressive, the evidence does not yet establish how much of the full-system improvement can be attributed specifically to the proposed contributions rather than the strong pretrained decoder, training data, and extensive downstream optimization.
[1] MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer, Arxiv 2025
[2] DualToken: Towards Unifying Visual Understanding and Generation with Dual Visual Vocabularies, ICLR 2026
[3] UniWeTok: An Unified Binary Tokenizer with Codebook Size 2128 for Unified Multimodal Large Language Model, Arxiv 2026
Dear Reviewer i1tM,
We sincerely thank you for your time and thoughtful feedback.
Performance without pre-caption. We trained a tokenizer with post-quantization caption supervision only and ran the same joint training as in Table 3(c). The setting is identical to the first row of Table 3(c) (L2 initialization,no D2C, no VAE-latent prediction, same downstream data and schedule), except that pre-caption is removed.
| Tokenizer | GenEval | DPG | TextVQA | MMB-EN |
With pre-caption, GenEval increases by 0.04 and DPG by 1.56. The understanding results are mixed (TextVQA -0.52, MMB-EN +1.02), so we do not draw a conclusion for understanding. This is reasonable, since understanding uses the continuous branch and pre-caption is applied to the discrete branch. This comparison has two limitations. It is done on the simplest configuration (no D2C, no VAE-latent prediction) rather than on the full model, and the caption pathways are sampled differently in the two tokenizers (1/2 each for continuous and post without pre-caption, 1/3 each for the three pathways with it), which may also contribute to the difference.
VAE-latent prediction without masking. We trained a tokenizer with the mask ratio set to 0, so the auxiliary Transformer predicts the VAE latents from the complete pre-quantization latent. All other settings are the same as the last row of Table 3(c), including decoder training, reconstruction evaluation and downstream training with TAN initialization. In the paper the mask ratio is sampled from [0.1, 0.5].
The results without masking are close to those with masking. For the tokenizer, masking gives slightly better PSNR (+0.22), rFID (3.2 vs. 3.5) and LPIPS (0.40 vs. 0.43), and slightly lower SSIM. For the downstream model, the differences on all four benchmarks are below 0.3 and do not have a consistent direction. This shows that the improvement mainly comes from predicting VAE latents before quantization, and masking is not necessary. We will change the name of this objective from "masked VAE-latent prediction" to "VAE-latent prediction" and add both results to Tables 3(a) and 3(c).
Where the gains come from. From the table above, the additional visual supervision is useful by itself: applied after quantization, it improves all reconstruction metrics. Applying the same objective before quantization gives a larger improvement (PSNR 18.25 vs. 16.50, LPIPS 0.40 vs. 0.49), and these two rows use the same pathway sampling and loss weights. Masking has a small effect, as discussed above. As another control, doubling the weight of the post-quantization caption loss does not improve the results (PSNR 15.06 vs. 15.02 in Table 3(a)). We did not test all combinations: the two positions are compared with masking, and masking is only ablated for the pre-quantization position.
Random initialization followed by the same caption training. We ran this baseline and a variant where the random vocabulary is first rescaled to the text-embedding norm. In both cases only the token indices from the tokenizer are used. The caption training uses the same data, number of steps and optimization settings as TAN in Table 3(b), and in both cases the auxiliary LLM starts from the pretrained Qwen3-0.6B. After that, we transfer only the vocabulary to a newly loaded Qwen3-0.6B and run the same T2I training.
Norm is the average token norm of the transferred vocabulary, and caption loss is measured at the end of caption training. The caption loss of the two baselines (0.86 and 0.83) is close to that of TAN (0.78), which means the LLM can already use these vocabularies for captioning. However, they do not improve generation over random initialization without caption training (0.12 and 0.14 vs. 0.15). TAN is higher than this baseline by 0.26 to 0.28. These results suggest that the improvement of TAN mainly comes from the vocabulary learned by the tokenizer, and caption adaptation further improves it from 0.34 to 0.40.