acceptodds
Under review as a conference paper at ICLR 2027

A Latent Distribution Perspective on Evaluating and Improving Latent Generative Models

Abstract

In latent generative models, reconstruction quality is often assumed to correlate with generative performance. However, reconstruction FID (rFID) can exhibit weak or even negative correlation with generation FID (gFID). We attribute this misalignment to a latent distribution mismatch: reconstruction evaluates the decoder on encoder-induced latents, whereas generation uses the same decoder on latents produced by the generative model. To characterize this shift, we introduce generation-aware reconstruction (GAR), which constructs a continuous trajectory from standard reconstruction toward generation by perturbing encoder latents with noise and denoising them through the generative model before decoding. GAR probes the decoder behavior along this trajectory, making the transition from encoder to generation-time latent distributions observable and diagnosable. The resulting trajectory-based diagnostic, GAR-FID, exhibits strong empirical correlation with gFID across diverse tokenizers and scales. Importantly, intermediate GAR latents become more generation-aware while preserving correspondence with their source images, thereby retaining paired supervision that is absent for fully generated latents. This correspondence enables decoder adaptation on intermediate GAR latents, consistently improving generative quality across model scales. Overall, latent distribution mismatch provides a useful perspective for evaluating and improving latent generative models.

Primary area:
generative models
Keywords:
latent generative models, visual tokenizers, latent distribution mismatch, generation-aware reconstruction, generative evaluation, decoder adaptation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_FikZOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper adress the problem of decoders being trained on denoising tasks and being drop-in solutions for generative tasks. The authors motivate their solution by stating that there is mostly non correlation between reconstruction FID and generation FID. For this they present a solution to adapt the decoder for the generation task by fine-tuning it in a Generative-Aware Reconstruction (GAR) task, where the image is encoded into a latent space and then its perturbed by some noise and then decoded. The paper presents analysis on the latent distribution mismatch and decoder behaviour finishing with experiments that show how GAR improves generation. The main contributions of the paper are: identifying the problem of using reconstruction decoder in a generative task, introduce GAR as a method that constructs a trajectory from reconstruction to generation.

Critical strengths
  • The motivation of the problem and solution are easy to follow and well explained.
  • The paper shows that using decoders from reconstruction can hurt the performance of generative models and how it can be easily prevented improving FID of the images generated.
  • The in-depth analysis (Section 3 and 4) present results that corroborate with the motivation showing that the problem is impacting generative tasks.
Critical weaknesses
  • Despite being informative and explaining the problem, the initial analysis occupy most of the main pages making the final experiments rely mostly on the Annex.
  • The decoder adaption could be presented with more depth on ablations such as training with different noise levels and presenting failure cases and possible justifications for them.
Questions for authors
  • The work focuses on adapting the decoder with a fixed range of noise levels (0.25-0.45). What would happen if the adaption is done with lower ranges of noise like 0.10-0.25 or only with high levels of noise?
  • What do you think enables the decoder to perform better on examples such as Fig.14 but then struggling on samples such as Fig.17? It would be interesting to have a failure case analysis to understand if the problem still related with the capability of decoding latents produced during the generation.

Suggestions:

  • In Figure 3 change one of the legend for latents from diagonal left to diagonal right, by doing that the overlap is much easier to understand.
  • Figure 3 and Figure 5 captions should have more vertical space so it is easier to read and distinguish from the main text
  • The arrows inside table 2 should follow the header by being green and pointing down, otherwise it is too difficult to understand.
5 comments
AuthorsOpenReview ↗

Response to Reviewer FikZ (1/3)

We appreciate that the reviewer found our motivation and solution clear and easy to follow, recognized that reconstruction-trained decoders can hurt generative performance and that decoder adaptation readily improves generation FID, and found that our in-depth analyses in Sections 3 and 4 corroborate this motivation. Thank you for taking the time to review our paper and for your thoughtful feedback. We address your questions and concerns below.


R 3.1 Despite being informative and explaining the problem, the initial analysis occupy most of the main pages making the final experiments rely mostly on the Annex.

Thank you for this suggestion. Sections 3 and 4 establish the latent distribution mismatch, show that the decoder amplifies it, verify that GAR moves decoder inputs from Pe\mathcal{P}_e toward Pg\mathcal{P}_g, and relate this shift to generation quality across 13 tokenizers. Decoder adaptation (DA) builds directly on these findings, so we gave them most of the main pages, and several DA results ended up in the Appendix.

We agree that the experiments should be more visible in the main text. Within the page limit, the revision adds the following DA experiments to the main text:

  • Section 7: a noise-range ablation for CFG-aware DA (Table 4), DA gains in six representation spaces beyond Inception (Table 3), and results on MAEToK, a multi-step generator with a different tokenizer.
  • Section 8 (new): controlled experiments on the training latents of DA, including a new baseline that fine-tunes the decoder on directly perturbed encoder latents. On iMF-B/2, this baseline worsens gFID with CFG from 3.47 to 9.91, while DA improves it to 3.01 (Table 5).

Section 7 also points to the remaining analyses in the Appendix, including the system-level comparison (Table 18) and the failure-case analysis (Appendix L).

R 3.2 The decoder adaption could be presented with more depth on ablations such as training with different noise levels and presenting failure cases and possible justifications for them.

Thank you for this suggestion.

(1) Noise-level ablation

This ablation is already included in the original submission, in Table 13 of Appendix L (Table 16 in the revision), which we may not have pointed to clearly enough from the main text. It varies only the noise level used for decoder adaptation (DA), fixed at ηt∈{0.0,…,0.6}\eta_t \in \{0.0, \dots, 0.6\}, under the DA w/o CFG setting (GAR latents without CFG, 10 epochs). We report gFID w/o CFG (↓) under the OpenAI protocol:

Training latentsiMF-B/2iMF-M/2iMF-L/2iMF-XL/2
w/o DA16.4111.929.269.41
Clean (ηt=0.0\eta_t = 0.0)15.2711.308.388.82
ηt=0.1\eta_t = 0.114.7611.038.198.64
ηt=0.2\eta_t = 0.213.5010.157.898.03
ηt=0.3\eta_t = 0.312.869.997.577.84
ηt=0.4\eta_t = 0.412.089.527.517.24
ηt=0.5\eta_t = 0.511.589.187.367.19
ηt=0.6\eta_t = 0.611.889.347.417.40

The trend is the same at all four scales. As ηt\eta_t increases, the training latents move from the encoder distribution toward the generation-time distribution, and gFID improves up to ηt=0.5\eta_t = 0.5. At ηt=0.6\eta_t = 0.6, gFID is worse than at 0.50.5 at every scale, because the latents keep too little correspondence with their source images for the paired supervision to stay reliable. rFID shows this directly: on iMF-B/2, it rises from 1.20 at ηt=0.5\eta_t = 0.5 to 1.81 at ηt=0.6\eta_t = 0.6.

AuthorsOpenReview ↗

Response to Reviewer FikZ (2/3)

(2) Failure cases

We observe three cases where DA does not help or comes at a cost. In each, the outcome depends on which latent distribution the decoder is trained on.

To examine the first two, we added a new experiment in the revision. On iMF-B/2, we follow one-epoch CFG-aware DA and keep the objectives, discriminator, optimizer, and number of decoder updates fixed, changing only the latents used to train the decoder: clean encoder latents, encoder latents with added noise but no generator denoising (znoisyt=atze+btϵz^t_{\mathrm{noisy}} = a_t z_e + b_t \epsilon), or GAR latents denoised by the generator. We report gFID under the OpenAI protocol:

Training latents (1 epoch)gFID w/ CFG ↓gFID w/o CFG ↓
None (w/o DA)3.4716.41
Clean (ηt=0\eta_t = 0)3.7416.21
Directly perturbed, ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45]9.9121.68
GAR (DA), ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45]3.0113.21
  • Directly perturbed latents. With the same noise range as DA, direct noise worsens gFID, whereas GAR latents improve it. Direct noise pushes the latents toward an isotropic Gaussian, not toward the generation-time distribution, so the decoder is trained on perturbations it does not see during generation.
  • Clean latents. Fine-tuning on clean encoder latents worsens gFID with CFG and gives only a small gain without CFG. Without generation-time latents, the fine-tuning objectives alone do not improve generation.
  • Reconstruction on encoder latents. DA lowers standard reconstruction quality: on iMF-B/2, FID on encoder latents rises from 0.73 to 0.85, while FID on latents closer to the generation-time distribution improves, for example from 4.38 to 3.10 at ηt=0.5\eta_t = 0.5 (Table 17). The adapted decoder works better on generation-time latents and slightly worse on encoder latents.

These results are summarized in Section 8 (Table 5), and the full results, including fixed noise levels from 0.1 to 1.0 for directly perturbed latents, are given in Appendix O (Table 21).

R 3.3 The work focuses on adapting the decoder with a fixed range of noise levels (0.25-0.45). What would happen if the adaption is done with lower ranges of noise like 0.10-0.25 or only with high levels of noise?

Thank you for this question. In addition to the fixed-noise ablation without CFG in R 3.2, we ran CFG-aware DA with different noise ranges. All settings follow our main CFG-aware DA protocol (one epoch, GAR latents generated with CFG) and change only the noise range. We report gFID with CFG under the OpenAI protocol:

Noise rangeiMF-B/2iMF-M/2iMF-L/2iMF-XL/2
w/o DA3.472.401.901.80
[0.10,0.25][0.10, 0.25]3.502.381.841.85
[0.10,0.30][0.10, 0.30]3.412.321.851.81
[0.25,0.45][0.25, 0.45] (default)3.012.191.701.61
[0.40,0.60][0.40, 0.60]3.043.212.981.98
  • Low noise. With [0.10,0.25][0.10, 0.25], the GAR latents stay close to the encoder distribution and give the decoder little exposure to generation-time latents. The effect is small and mixed: gFID improves slightly on iMF-M/2 and iMF-L/2 but is slightly worse on iMF-B/2 and iMF-XL/2. Extending the range to [0.10,0.30][0.10, 0.30] gives small gains at most scales (e.g., 3.47 → 3.41 on iMF-B/2), still far from the default range.
  • High noise only. With [0.40,0.60][0.40, 0.60], gFID on iMF-B/2 stays close to the default range (3.04), but becomes worse than the original decoder on iMF-M/2, iMF-L/2, and iMF-XL/2. This is another failure case of DA: at high noise levels, the GAR latents keep too little correspondence with their source images, so the paired supervision no longer matches the content in the latents.

Compared with the setting without CFG in R 3.2, where a fixed ηt=0.5\eta_t = 0.5 worked best, the useful range is lower with CFG. This is why our default range [0.25,0.45][0.25, 0.45], which gives the best results here at all four scales, was chosen for CFG-aware DA. These results have been added to Section 7 (Table 4).

AuthorsOpenReview ↗

Response to Reviewer FikZ (3/3)

R 3.4 What do you think enables the decoder to perform better on examples such as Fig.14 but then struggling on samples such as Fig.17? It would be interesting to have a failure case analysis to understand if the problem still related with the capability of decoding latents produced during the generation.

Thank you for this question. In Figures 13-24, the two samples in each column are decoded from the same generated latent zgz_g; only the decoder parameters differ, before and after DA. DA therefore changes how a given zgz_g is rendered, but not what zgz_g encodes. We see two sources of error:

  • Decoder-side errors (e.g., Fig. 14). In samples such as the agama in Fig. 14, the generated latent already encodes a plausible image, and the remaining artifacts come mainly from decoding, such as smoothed textures and inconsistent fine details. These are the errors DA targets, so the adapted decoder produces sharper and more coherent details.
  • Generator-side errors (e.g., Fig. 17). In the weaker samples of Fig. 17 (golden retriever), structural distortions, such as irregular faces or body shapes, appear in both the original and the adapted decoding of the same zgz_g. These errors are therefore more likely to come from the generated latent itself.

The training setup of DA explains why it can fix the first kind of error but not the second. DA is trained on GAR latents with ηt∈[0.25,0.45]\eta_t \in [0.25, 0.45], where the global structure is still largely determined by the source image (Figure 8), so the paired supervision mainly teaches the decoder local appearance. Global structure is formed at higher noise levels, where the latents keep too little correspondence with their source images for paired supervision to work, as shown in R 3.2 and R 3.3. DA can thus refine local appearance but is unlikely to correct global structure.

For these failure cases, we therefore think the remaining problem lies mainly in the generated latents rather than in the decoder's ability to decode them. Such errors are better addressed by improving the generative model, which is complementary to DA (Section 5). We have added this discussion to Appendix L.

R 3.5 In Figure 3 change one of the legend for latents from diagonal left to diagonal right, by doing that the overlap is much easier to understand. Figure 3 and Figure 5 captions should have more vertical space so it is easier to read and distinguish from the main text. The arrows inside table 2 should follow the header by being green and pointing down, otherwise it is too difficult to understand.

Thank you for pointing these out. We have revised the figures and table according to your suggestions.


Please let us know if you have any remaining questions.


Reviewer_nSLjOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies misalignment between good reconstruction quality and good generation quality. In order to focus on the difference between encoder-induced latent and starting point of generator, it suggests generation-aware reconstruction (GAR). This metric evaluates the outputs at different noise levels. Furthermore, authors propose using intermediate latents to fine-tune the decoder while maintaining the encoder and generator freezes. The empirical results demonstrate improved generation quality, including without CFG.

Critical strengths
  1. This paper clearly shows a study of the substantial gap between generation quality and reconstruction quality.
  • The difference between encoder-induced latent and generator-produced latents presents a practical perspective on why reconstruction quality cannot guarantee the generation quality. Well-motivated study on the generator and decoder component is a strength, as it identifies that the performance of generation quality depends not only the quality of the generated latents but also on how the decoder adapts them. This induces the reformulation of both the evaluation and improvement of generation quality.
  1. The proposed decoder adaptation is practical and shows empirical improvements
  • The method enhances the generation quality while maintaining the encoder and generator being freezed, without introducing additional inference steps. In real world scenario, it is a practical approach for improving the performance of generative models without retraining the latent generative models.
Critical weaknesses
  1. Distinction from closely related work requires clarification.
  • "Image Tokenizer Needs Post-Training" [1] already shows decoder-only post-training to adapt to generated latents, including SDEdit-based generation for diffusion models. This appears closely related to GAR adaptation. The paper should clarify the differences, ideally supported by a direct experimental comparison. Also, "Latent Denoising Makes Good Tokenizers" [2] trains tokenizers to reconstruct images from corrupted latents. A brief discussion of what generator-based denoising adds beyond direct latent corruption training would further clarify the contribution.
  1. Insufficient isolation of the benefit from decoder adaptation.
  • In Table 13, it shows a decoder only fine-tuning baseline using the clean latents. However, it still remain unclear whether the improvement is induced by generator-aware latent adaptation or by the robustness to latent perturbation itself. More specifically, [2] has already demonstrated this robustness from the fine-tuning tokenizer with corrupted latents. Fair comparison against decoder only fine-tuning on directly corrupted encoder latents without generator denoising, would help identify the benefit of the proposed method.

[1] Kai Qiu, Xiang Li, Hao Chen, Jason Kuen, Xiaohao Xu, Jiuxiang Gu, Yinyi Luo, Bhiksha Raj, Zhe Lin, and Marios Savvides. Image Tokenizer Needs Post-Training

[2] Jiawei Yang, Tianhong Li, Lijie Fan, Yonglong Tian, and Yue Wang. Latent Denoising Makes Good Tokenizers

Questions for authors
  1. How does GAR adaptation differ from previous decoder post-training?
  • How does the proposed method differ from [1], particularly in how training settings are constructed and how does the decoder trained? Direct comparison between GAR and [1] would help clarify the contribution of adaption method. Also, what additional benefit comes from using generator-denoised latents, beyond learning robustness to latent corruption as suggested in [2]? Clarifying these connections would help establish the unique contribution of GAR.
4 comments
AuthorsOpenReview ↗

Response to Reviewer nSLj (1/4)

We appreciate that the reviewer found our study of the gap between reconstruction and generation quality well-motivated, and recognized our key insight that generation quality depends not only on the generated latents but also on how the decoder adapts to them. We are also glad that the reviewer considered our decoder adaptation practical for real-world use, as it improves generation quality without retraining the latent generative model or adding inference cost. Thank you for your thoughtful review. We address your questions and concerns below.


R 2.1 Distinction from closely related work requires clarification. "Image Tokenizer Needs Post-Training" [1] already shows decoder-only post-training to adapt to generated latents, including SDEdit-based generation for diffusion models. This appears closely related to GAR adaptation. The paper should clarify the differences, ideally supported by a direct experimental comparison. Also, "Latent Denoising Makes Good Tokenizers" [2] trains tokenizers to reconstruct images from corrupted latents. A brief discussion of what generator-based denoising adds beyond direct latent corruption training would further clarify the contribution.

Thank you for pointing us to these works. In the revision, we discuss SDEdit [4] in Section 2.2, [1] in Section 6, and [2] in Section 8, and give a detailed comparison in Appendix P. We address the three points below.

(1) Relation to [1]

[1] mainly studies discrete tokenizers with autoregressive generators. Its main contributions are latent perturbation during tokenizer training, the pFID metric, and a decoder post-training stage in which a preservation ratio σ\sigma replaces part of the generated tokens with ground-truth tokens. For diffusion models, its Appendix A.5 only relates σ\sigma to the noise level of SDEdit [4], defining σ=1−t/T\sigma = 1 - t/T by analogy with the autoregressive case. It does not specify the post-training procedure for diffusion models, such as how training latents are constructed and paired, the sampler, the use of guidance, or the training settings. We agree that this analogy is conceptually close to decoder adaptation (DA) on GAR latents. SDEdit itself edits images in pixel space and has no encoder or decoder, so it does not involve decoder training. The main post-training method of [1] replaces generated tokens with ground-truth tokens for autoregressive models; the noise-then-denoise operation in latent space appears only in its brief analogy for diffusion models in Appendix A.5. Our work differs from [1] in two respects.

Analysis and diagnosis. Our main contribution is analytical. We attribute the reconstruction–generation gap to the mismatch between encoder latents Pe\mathcal{P}_e and generated latents Pg\mathcal{P}_g, show that the decoder amplifies this mismatch in its deeper features (Section 3.1), and use GAR as a controlled path from Pe\mathcal{P}_e to Pg\mathcal{P}_g with the decoder fixed. Along this path, Latent-FD to Pg\mathcal{P}_g decreases (Figure 4), and GAR-FID rises from near rFID toward its generation endpoint for every tokenizer–generator pair (Tables 7 and 8). GAR-FID correlates strongly with gFID across 14 tokenizers and two generator scales (Table 1). pFID in [1], by comparison, perturbs tokens without the generator and is designed for discrete tokenizers.

DA for diffusion and flow models. [1] does not describe a post-training method for diffusion models, and its released code does not include one. We design and evaluate DA for continuous diffusion and flow generators:

  • Noise sampling. We sample ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45] instead of fixing one strength, based on the trade-off between generation awareness and source correspondence in Table 16, and the CFG-aware noise-range ablation in Table 4.
  • CFG-aware construction. We generate GAR latents with guidance to match inference. One epoch of CFG-aware DA matches or outperforms 10 epochs of DA without CFG at all iMF scales (Table 2).
  • Isolating the training distribution. With the same objectives, fine-tuning on clean encoder latents improves gFID w/o CFG on iMF-B/2 from 16.41 to 15.27, while GAR latents at ηt=0.5\eta_t = 0.5 reach 11.58 (Table 16). Most of the gain comes from the training latents, not from fine-tuning itself.
  • Breadth. DA improves gFID at four iMF scales, with and without CFG and under two evaluation protocols, with stable gains across three random seeds (Tables 2 and 15). It also improves MeanFlow-XL/2 and the distilled DMF-XL/2+ (Table 18).
AuthorsOpenReview ↗

Response to Reviewer nSLj (2/4)

(2) Comparison with [1] on MAEToK

Because [1] does not specify its post-training procedure for diffusion models, we could not reimplement it reliably on iMF. We instead compare on MAEToK [3], for which [1] reports post-training results with the official MAEToK checkpoint (its Table 10(b), baseline gFID 1.74, close to the 1.73 reported by MAEToK). We start from the same official MAEToK decoder. [1] pairs it with LightningDiT, but only the MAEToK SiT-XL checkpoint is publicly available, so we apply CFG-aware DA to MAEToK with SiT-XL. The two settings share the tokenizer and differ only in the generator and its baseline, so we compare the change from each baseline. [1] trains on 200K images for 10 or 20 epochs, which equals 1.56 or 3.12 epochs of ImageNet (1.28M images); we report epochs on the same scale.

MethodGeneratorEpochsgFID w/ CFG ↓Change
BaselineLightningDiT–1.74–
[1], σ=0.7\sigma = 0.7LightningDiT1.56 / 3.122.00 / 2.23+0.26 / +0.49
[1], σ=0.8\sigma = 0.8LightningDiT1.56 / 3.121.81 / 2.12+0.07 / +0.38
[1], σ=0.9\sigma = 0.9LightningDiT1.56 / 3.121.70 / 1.85−0.04 / +0.11
BaselineSiT-XL–1.67–
CFG-aware DA (ours)SiT-XL0.21.60−0.07
CFG-aware DA (ours)SiT-XL11.55−0.12

The results of [1] are taken from its Table 10(b). On the official MAEToK, [1] worsens gFID in five of six settings, and its only gain is small (−0.04 at σ=0.9\sigma = 0.9 with 1.56 epochs); longer training makes results worse. DA improves gFID at both training budgets, already after 0.2 epochs. Since the generators and baselines differ, this is a reference comparison, not a strictly controlled one. This comparison is reported in Appendix P (Table 23), and the MAEToK result is also reported in Section 7.

(3) What generator-based denoising adds beyond direct latent corruption [2]

Both [2] and GAR start from corrupted latents znoisyt=atze+btϵz^t_{\mathrm{noisy}} = a_t z_e + b_t \epsilon. [2] reconstructs from znoisytz^t_{\mathrm{noisy}} directly, and its distribution moves toward N(0,I)\mathcal{N}(0, I) as noise increases. GAR first denoises it with the generator, so the resulting latents move toward Pg\mathcal{P}_g (Figure 4). Training on GAR latents therefore exposes the decoder to the shift it meets at generation time; Gaussian corruption does not occur during generation. [2] reports a consistent finding in its Appendix A.3: with the encoder frozen, decoder-only training with latent noise on MAR-VAE worsens FID with CFG, and latent denoising helps only when the encoder and decoder are trained jointly. Joint training, however, changes the latent space the generator was trained on, so it does not apply to our setting, where the encoder and generator are frozen. We give a controlled comparison in R 2.2.

[1] Image Tokenizer Needs Post-Training. arXiv, 2025.

[2] Latent Denoising Makes Good Visual Tokenizers. ICLR, 2026.

[3] Masked Autoencoders Are Effective Tokenizers for Diffusion Models. ICML, 2025.

[4] SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. ICLR, 2022.

AuthorsOpenReview ↗

Response to Reviewer nSLj (3/4)

R 2.2 Insufficient isolation of the benefit from decoder adaptation. In Table 13, it shows a decoder only fine-tuning baseline using the clean latents. However, it still remain unclear whether the improvement is induced by generator-aware latent adaptation or by the robustness to latent perturbation itself. More specifically, [2] has already demonstrated this robustness from the fine-tuning tokenizer with corrupted latents. Fair comparison against decoder only fine-tuning on directly corrupted encoder latents without generator denoising, would help identify the benefit of the proposed method.

Thank you for this suggestion. We have added the requested baseline.

(1) Decoder-only fine-tuning on directly corrupted encoder latents

The baseline follows one-epoch CFG-aware decoder adaptation (DA) with the same objectives and loss weights, discriminator, optimizer, learning rate, batch size, and number of decoder updates. Only the training latents differ: DA uses GAR latents denoised by the generator with CFG, while the baseline uses directly corrupted encoder latents znoisyt=atze+btϵz^t_{\mathrm{noisy}} = a_t z_e + b_t \epsilon, as in [2]. We train one decoder per noise setting on iMF-B/2 and report gFID under the OpenAI protocol.

Training latents (1 epoch)gFID w/ CFG ↓gFID w/o CFG ↓
None (w/o DA)3.4716.41
Clean (ηt=0\eta_t = 0)3.7416.21
Corrupted, ηt=0.1\eta_t = 0.16.4016.66
Corrupted, ηt=0.2\eta_t = 0.212.4625.34
Corrupted, ηt=0.3\eta_t = 0.319.7537.75
Corrupted, ηt=0.4\eta_t = 0.424.6247.21
Corrupted, ηt=0.5\eta_t = 0.529.5157.07
Corrupted, ηt=0.6\eta_t = 0.639.3868.86
Corrupted, ηt=0.7\eta_t = 0.747.0780.08
Corrupted, ηt=0.8\eta_t = 0.896.93117.40
Corrupted, ηt=0.9\eta_t = 0.9110.94121.59
Corrupted, ηt=1.0\eta_t = 1.0153.32154.08
Corrupted, ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45]9.9121.68
GAR (DA), ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45]3.0113.21

With the same noise range ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45], corrupted latents worsen gFID from 3.47 to 9.91 with CFG and from 16.41 to 21.68 without CFG, while GAR latents improve it to 3.01 and 13.21. The two settings differ only in whether the generator denoises the corrupted latents. Every fixed corruption level is also worse than the original decoder, and more so as noise grows. Clean-latent fine-tuning gives no clear gain (3.74 with CFG, 16.21 without), so the objectives alone do not explain the improvement. Robustness to latent perturbation therefore does not explain the gain of DA in this setting; the gain comes from training on generator-denoised latents.

(2) Why direct corruption does not help here

As discussed in R 2.1 (3), corrupted latents move toward N(0,I)\mathcal{N}(0, I) as noise increases, whereas GAR latents move toward the generation-time distribution Pg\mathcal{P}_g. Training on corrupted latents prepares the decoder for perturbations it does not see during generation. This matches the finding of [2] in its Appendix A.3 that decoder-only training with latent noise worsens FID with CFG when the encoder is frozen; robustness from latent denoising in [2] relies on training the encoder and decoder jointly, which changes the latent space the generator was trained on.

The same pattern appears on the evaluation side. We fed corrupted encoder latents directly to the decoder, without generator denoising, and computed FID on the outputs (nFID) for the 13 tokenizers with SiT-B. Across noise levels from 0.1 to 0.9, the SRCC between nFID and gFID ranges from only −0.15 to 0.43, whereas GAR-FID at ηt=0.8\eta_t = 0.8 reaches 0.96 without CFG and 0.94 with CFG. A decoder's robustness to Gaussian corruption thus says little about its behavior on generated latents. The full nFID results are provided in Appendix N.

These results have been added to Section 8 (Table 5) and Appendices N and O.

AuthorsOpenReview ↗

Response to Reviewer nSLj (4/4)

R 2.3 How does GAR adaptation differ from previous decoder post-training? How does the proposed method differ from [1], particularly in how training settings are constructed and how does the decoder trained? Direct comparison between GAR and [1] would help clarify the contribution of adaption method. Also, what additional benefit comes from using generator-denoised latents, beyond learning robustness to latent corruption as suggested in [2]? Clarifying these connections would help establish the unique contribution of GAR.

Thank you for these questions. R 2.1 discusses how our analysis and DA design differ from [1], and R 2.2 addresses the benefit beyond robustness to latent corruption. Here we compare the training setups of [1] and DA directly, and summarize the other two points.

(1) Training setups of [1] and DA

The table below summarizes how training latents are constructed and how the decoder is trained, based on Section 4.3 and Appendices A.5-A.6 of [1] and Section 6 and Appendix K of our paper. This comparison is included as Table 22 in Appendix P.

[1], autoregressive[1], diffusion (Appendix A.5)DA (ours)
GeneratorsLlamaGen, RARMAEToK with LightningDiTiMF (four scales), MeanFlow, DMF; MAEToK with SiT-XL
Training latentsGenerated images re-encoded by the encoderNot specifiedGenerator outputs from noised encoder latents, used directly
Link to source imageFraction σ\sigma of tokens replaced by ground truthσ=1−t/T\sigma = 1 - t/T via SDEditNoise level ηt\eta_t before generator denoising
Noise / preservation levelOne fixed σ\sigma per run, tuned per tokenizerOne fixed σ\sigma per runηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45] sampled per example
Guidance when building latentsNot specifiedNot specifiedCFG-aware (matches inference)
Decoder objectivesOriginal tokenizer losses, pretrained discriminator reusedNot specifiedPixel, LPIPS, adversarial (DINO-based discriminator), and Gram losses
Training data200K images, 10–20 epochs200K images, 10–20 epochsImageNet, 1 epoch (CFG-aware) or 10 epochs

Both methods freeze the encoder and update only the decoder. The main differences are that DA takes the generator's latents directly rather than re-encoding generated images, samples the noise level instead of fixing it, and builds training latents with guidance to match inference. Each of these choices is supported by our ablations: the noise sweeps in Tables 4 and 16 and the comparison of CFG-aware DA with DA without CFG in Table 2.

(2) Direct comparison with [1]

As shown in R 2.1 (2), on the same official MAEToK decoder, [1] worsens gFID in five of six settings reported in its Table 10(b), with a best change of −0.04, while CFG-aware DA improves gFID from 1.67 to 1.55 after one epoch. Because the generators differ (LightningDiT for [1], SiT-XL for ours, the only public MAEToK generator), we treat this as a reference comparison.

(3) Benefit beyond robustness to latent corruption [2]

As shown in R 2.2, with the same objectives, number of updates, and noise range, decoder fine-tuning on directly corrupted encoder latents worsens gFID on iMF-B/2 from 3.47 to 9.91 with CFG, while DA on generator-denoised latents improves it to 3.01. The benefit therefore comes from training on latents that move toward the generation-time distribution Pg\mathcal{P}_g, not from robustness to Gaussian corruption, which does not occur during generation.


Please let us know if you have any remaining questions. If our clarifications and additional experiments have satisfactorily addressed your questions and concerns, we would deeply appreciate it if you considered recommending our paper to be presented at the conference.


Reviewer_RxvfOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Generation-aware reconstruction (GAR) adds noise to encoder latents and denoises them with a fixed generator. GAR-FID measures decoded distributions along this path. Decoder adaptation uses intermediate latents with paired image targets. The encoder and generator remain fixed. Essentially, this method fine-tunes tokenizer decoders to make it more robust to imperfectly generated latents.

Critical strengths
  1. The metric study covers 13 tokenizers at two SiT scales, plus RAE at SiT-XL. Rank correlations reach 0.94-0.99 at noise level 0.8. Leave-one-out checks support robustness.

  2. Adaptation adds no inference stage. Gains occur under two FID protocols and several generators. CFG-aware adaptation improves iMF-XL/2 from 1.80 to 1.61. The appendix reports evaluation variability and reconstruction trade-offs. The appendix also provides a clean-latent adaptation baseline and noise ablations. These controls strengthen the empirical case for adaptation on intermediate GAR latents.

Critical weaknesses
  1. My main concern is practical tokenizer selection before expensive generator training. GAR-FID requires a trained generator in each candidate tokenizer's latent space. It therefore evaluates an existing tokenizer-generator pair, rather than a tokenizer independently. If the goal is to select a tokenizer for a new large model, this creates a circular dependency: the proposed evaluation requires a model that the selection decision is meant to precede. Appendix G appropriately describes GAR as a diagnostic. That role is useful, but it leaves the tokenizer-selection question unresolved. The runtime comparison also excludes the cost of obtaining those generators.

  2. Using small proxy generators could reduce this cost, but transfer to larger models needs a clearer test. Denoising errors can change with model capacity and training budget. A tokenizer that handles one small model's errors well need not be best after scaling. Tables 4-5 provide encouraging B-to-XL ranking evidence. Both B GAR-FID and B gFID select DM-VAE, also best at XL (calculated from these tables). I wonder if other generative models, e.g., autoregressive diffusion models (MAR or RAR), will also observe a similar trend.

  3. Decoder adaptation is useful, but its specific advantage over simpler corruption training remains unclear. Table 13 includes clean-latent fine-tuning, but no direct latent-noise decoder baseline with matched losses and updates.

Questions for authors
  1. For a new tokenizer with no trained generator, what evaluation procedure and total compute budget do you recommend before committing to large-scale training?

  2. Under a fixed proxy-training budget, does SiT-B GAR-FID select better SiT-XL tokenizers than SiT-B gFID, with the noise level chosen without XL results?

  3. How does GAR adaptation compare with matched direct latent-noise decoder training?

  4. Do adaptation gains persist under metrics based on diverse representations, such as FDr^6 [1] or a comparable panel of feature-space distances? Does GAR-FID also predict rankings under these metrics?

9 comments
AuthorsOpenReview ↗

Response to Reviewer Rxvf(1/7)

Thank you for your thoughtful review. We appreciate your recognition of GAR-FID's strong rank correlations across diverse tokenizers and generator scales, supported by leave-one-out checks, and of the decoder adaptation gains across generators and evaluation protocols. We are also glad that you found the clean-latent baseline and noise ablations to strengthen the case for adaptation on intermediate GAR latents. We address your concerns below.


R 1.1 My main concern is practical tokenizer selection before expensive generator training. GAR-FID requires a trained generator in each candidate tokenizer's latent space. It therefore evaluates an existing tokenizer-generator pair, rather than a tokenizer independently. If the goal is to select a tokenizer for a new large model, this creates a circular dependency: the proposed evaluation requires a model that the selection decision is meant to precede. Appendix G appropriately describes GAR as a diagnostic. That role is useful, but it leaves the tokenizer-selection question unresolved. The runtime comparison also excludes the cost of obtaining those generators.

Thank you for clearly articulating your main concern.

(1) Scope of GAR-FID and revisions to the manuscript

We agree that GAR-FID requires a trained generator in each tokenizer's latent space. It evaluates an existing tokenizer–generator pair, so it cannot select a tokenizer before generator training, nor was it designed to. Our objective is to understand the reconstruction–generation performance gap from a latent distribution perspective, and to use this understanding to improve generation in existing systems.

Several phrasings in the original submission could suggest a selection use, and we have revised them:

  • Abstract and Introduction: GAR-FID is now described as a trajectory-based diagnostic of existing tokenizer-generator systems.
  • Section 2.2: we state that GAR-FID is defined for a given tokenizer-generator pair and diagnoses an existing system rather than a tokenizer in isolation.
  • Section 4.2: we state that iFID requires no trained generator and thus applies before generator training, whereas GAR-FID requires one. The higher correlation of GAR-FID is obtained with access to the generator, so the comparison concerns the analysis of existing systems rather than tokenizer selection.
  • Section 4.2 (computational cost) and Appendix F: runtime comparisons assume the generator is already trained and exclude its training cost.
  • Conclusion: "a reliable proxy for generative performance" now describes GAR-FID as a diagnostic of existing systems, and "tokenizer evaluation and training" is narrowed to analyzing and training decoders.
  • Appendix G (Limitations): we add that GAR-FID is not designed for tokenizer selection before generator training.
AuthorsOpenReview ↗

Response to Reviewer Rxvf(2/7)

(2) A generator-free variant of GAR

Although tokenizer selection before generator training is beyond the scope of our paper, we took the reviewer's question seriously and investigated whether GAR could be made generator-free. We removed the generator-based denoising step and fed the perturbed encoder latents znoisyt=atze+btϵz^t_{\mathrm{noisy}} = a_t z_e + b_t \epsilon directly into the decoder, computing FID on the outputs (denoted nFID). Computing nFID requires no generator. The gFID values, with and without CFG, are taken from the SiT-B models in Table 7 and are used only to assess the relationship between nFID and generation performance.

TokenizerrFID ↓gFID w/o CFG ↓gFID w/ CFG ↓nFID (0.1) ↓nFID (0.2) ↓nFID (0.3) ↓nFID (0.4) ↓nFID (0.5) ↓nFID (0.6) ↓nFID (0.7) ↓nFID (0.8) ↓nFID (0.9) ↓
SD-VAE0.7346.649.914.0817.0372.31176.90258.27318.42369.97417.39400.76
IN-VAE0.2648.2810.871.847.8929.9593.98216.74338.02372.97355.26342.92
FLUX-VAE0.1662.9214.385.1434.76100.68176.49243.75299.18345.21377.70397.07
QwenImage-VAE1.5248.0310.974.3423.3383.95179.25261.53321.06354.09363.81363.72
SD3-VAE0.2151.5811.892.4512.2244.80130.00254.70353.56440.66497.12515.65
EQ-VAE0.5937.499.3236.65146.19224.60273.50293.17307.53319.69329.76334.38
VA-VAE-160.3018.886.0122.75211.08307.27359.64380.42386.33392.51402.49412.74
VA-VAE-640.1533.567.767.1986.36251.54358.26415.80449.60467.08467.40459.32
SOFT-VQ0.5929.487.4811.51112.50219.28256.53268.81272.28273.35270.17260.70
MAE-TOK0.6113.755.6911.93186.72307.54324.48286.20256.51245.67260.37279.43
DE-TOK0.5920.176.951.661.943.6611.5153.24166.12280.14383.81459.63
DM-VAE0.728.384.892.243.7211.8349.07156.94298.95422.93484.75488.58
REPAE-VAE0.5625.756.464.1818.6165.53143.52202.97285.00376.08391.40410.72
MetricPCC w/o CFGSRCC w/o CFGPCC w/ CFGSRCC w/ CFG
rFID−0.11−0.36−0.11−0.30
nFID (ηt=0.1\eta_t=0.1)−0.16−0.14−0.16−0.14
nFID (ηt=0.2\eta_t=0.2)−0.38−0.15−0.38−0.15
nFID (ηt=0.3\eta_t=0.3)−0.28−0.13−0.31−0.13
nFID (ηt=0.4\eta_t=0.4)−0.09−0.14−0.16−0.13
nFID (ηt=0.5\eta_t=0.5)0.14−0.020.04−0.01
nFID (ηt=0.6\eta_t=0.6)0.270.430.170.41
nFID (ηt=0.7\eta_t=0.7)0.200.100.130.05
nFID (ηt=0.8\eta_t=0.8)0.100.020.080.02
nFID (ηt=0.9\eta_t=0.9)0.01−0.100.03−0.08
GAR-FID (ηt=0.8\eta_t=0.8)0.980.960.960.94

Without generator-based denoising, the relationship with generation performance is substantially weaker. The SRCC between nFID and gFID ranges from −0.15 to 0.43 across noise levels, whereas GAR-FID at ηt=0.8\eta_t = 0.8 achieves SRCCs of 0.96 without CFG and 0.94 with CFG. This negative result is informative. Direct noise injection probes the decoder's robustness to prescribed noise, whereas generator-based denoising probes the decoder's behavior on latents shaped by the actual generator. Removing the generator from GAR removes most of its correlation with gFID, which indicates that the generator-induced latent distribution is a key factor that this simple perturbation fails to capture. The full nFID results have been added to Appendix N (Tables 19 and 20).

(3) What GAR is designed to do: relating the latent distribution mismatch to the performance gap

Our paper asks why the same decoder performs differently in reconstruction and generation. In reconstruction, the decoder receives encoder latents ze∼Pez_e \sim \mathcal{P}_e; in generation, it receives generator latents zg∼Pgz_g \sim \mathcal{P}_g. rFID and gFID measure the two ends of the resulting gap, and Section 3.3 shows that a larger Latent-FD(Pe,Pg\mathcal{P}_e, \mathcal{P}_g) goes with worse gFID, but this correlation does not show that the mismatch causes the gap. GAR allows a more direct test: we change the latent distribution and keep everything else fixed. We report two such interventions, together with controls for other explanations.

AuthorsOpenReview ↗

Response to Reviewer Rxvf(3/7)

Intervention 1: changing only the decoder's input distribution. Along the GAR path, the decoder, source images, reference set, and evaluation protocol are the same at every noise level; only the input distribution moves from Pe\mathcal{P}_e (ηt=0\eta_t = 0) toward Pg\mathcal{P}_g (ηt=1\eta_t = 1). For SD-VAE with iMF-B/2, as ηt\eta_t increases, Latent-FD(Pgt,Pg\mathcal{P}_g^t, \mathcal{P}_g) decreases monotonically (Figure 4), and the gap between GAR-FID(ηt\eta_t) and GAR-FID(ηt=1\eta_t = 1) shrinks monotonically (Figure 10b). We use GAR-FID(ηt=1\eta_t = 1) as the endpoint instead of gFID because it shares the reference set (Appendix H). GAR-FID is also monotonic for all 13 tokenizers with SiT-B and all 14 with SiT-XL (Tables 7 and 8). We check two other explanations:

  • Loss of source correspondence. The rise of GAR-FID might come from outputs drifting away from their sources. But FID compares distributions rather than image pairs, so losing correspondence does not by itself raise FID: an ideal generator loses all correspondence at ηt=1\eta_t = 1 and can still reach near-zero FID. iMF-B/2 and iMF-XL/2 lose correspondence at the same rate in pixel, structural, and perceptual terms, with nearly overlapping GAR-PSNR, GAR-SSIM, and GAR-LPIPS curves (Figures 9 and 10a). Yet their GAR-FID differs substantially (e.g., 4.38 vs. 2.02 at ηt=0.5\eta_t = 0.5, Figure 6), in the same order as their latent mismatch (Figure 11) and gFID. The difference therefore comes from the generator-induced latent distribution.
  • Any departure from Pe\mathcal{P}_e. Moving the input away from Pe\mathcal{P}_e by direct noise injection, without the generator, does not reproduce this behavior: nFID reaches SRCCs with gFID of only −0.15 to 0.43 (see (2)). Generation quality follows the shift toward Pg\mathcal{P}_g, rather than any shift away from Pe\mathcal{P}_e.

Intervention 2: changing only the decoder's training distribution. If the mismatch causes the gap, training the decoder on latents closer to Pg\mathcal{P}_g should improve generation more as the latents get closer, until paired supervision becomes unreliable. We fix the encoder, generator, objectives, and training budget, and vary only the training latents.

  • Dose-response. In Table 16, where only the noise level of the training latents varies, gFID w/o CFG decreases monotonically from ηt=0.0\eta_t = 0.0 to 0.50.5 at all four iMF scales (e.g., 15.27 → 11.58 on iMF-B/2 and 8.82 → 7.19 on iMF-XL/2). The ηt=0.0\eta_t = 0.0 row uses clean latents with the same objectives and budget, so it gives the effect of fine-tuning alone (16.41 → 15.27 on iMF-B/2); the further gain to 11.58 comes from moving the training latents toward Pg\mathcal{P}_g. Above ηt=0.5\eta_t = 0.5, the gains reverse as source correspondence weakens (Appendix E).
  • Generator denoising versus direct perturbation. In R 1.3, we compare decoder adaptation with a baseline trained on directly perturbed encoder latents znoisyt=atze+btϵz^t_{\mathrm{noisy}} = a_t z_e + b_t \epsilon, using the same noise range ηt∼U[0.25,0.45]\eta_t \sim U[0.25, 0.45], objectives, discriminator, optimizer, and number of decoder updates. The only difference is whether the generator denoises the perturbed latents. Training on directly perturbed latents substantially worsens gFID, while training on GAR latents improves it (full results in R 1.3). Robustness to latent perturbation therefore does not explain the gain.
  • Opposite effects on the two distributions. If decoder adaptation simply made the decoder better overall, reconstruction on Pe\mathcal{P}_e should not degrade. Instead, the effect depends on where the decoder is evaluated. Across training noise levels (Table 16), moving the training latents toward Pg\mathcal{P}_g improves gFID but raises rFID on iMF-B/2 from 0.58 to 1.20 (training ηt\eta_t from 0.0 to 0.5). For one adapted decoder evaluated at different noise levels (Table 17), FID worsens on inputs at or near Pe\mathcal{P}_e (0.73 → 0.85 at ηt=0\eta_t = 0; 0.90 → 0.93 at ηt=0.1\eta_t = 0.1) and improves once inputs are close enough to Pg\mathcal{P}_g (1.28 → 1.10 at ηt=0.2\eta_t = 0.2; 4.38 → 3.10 at ηt=0.5\eta_t = 0.5). The decoder works best on the distribution it is trained on, as expected if the gap is driven by the input distribution.

In both interventions, the performance gap changes when we change the mismatch directly with other factors fixed. We see this as empirical evidence that the latent distribution mismatch plays a causal role in the reconstruction–generation gap. It is not a formal proof, and other factors may also contribute; Proposition 1 gives a complementary bound under Gaussian and Lipschitz assumptions. In the revised manuscript, both interventions are presented in a new Section 8, with the direct-perturbation baseline in Table 5; the full analysis, nFID results, and direct-perturbation results are given in Appendices M, N, and O, respectively.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

Thank you to all the reviewers for their valuable feedback. We are grateful to reviewers nSLj and FikZ for recognizing the clear motivation of our work and the importance of understanding the mismatch between encoder-induced and generator-produced latents. We also thank reviewer Rxvf for highlighting the strong rank correlations and robustness checks in our GAR-FID study, as well as the ablations supporting adaptation on intermediate GAR latents. Finally, we appreciate all reviewers for recognizing the practical value of decoder adaptation, which improves generation quality while keeping the encoder and generator fixed and adding no inference cost.

We have revised the manuscript accordingly; changes are marked in blue. The main revisions are:

  1. Scope of GAR-FID (Rxvf). We now describe GAR-FID as a diagnostic of existing tokenizer-generator systems rather than a tool for tokenizer selection before generator training (Abstract, Introduction, Sections 2.2 and 4.2, Conclusion, and Appendices F and G). Section 4.2 also notes that generator-free metrics such as iFID apply before generator training, and that runtime comparisons exclude the cost of training the generator.
  2. Controlled interventions on the latent distribution (Rxvf, nSLj). A new Section 8 tests whether the latent distribution mismatch drives the reconstruction-generation gap by changing only the decoder's input distribution or only its training latents. It includes a new baseline that fine-tunes the decoder on directly corrupted encoder latents with matched objectives and updates (Table 5): this baseline worsens gFID, whereas DA on generator-denoised latents improves it. Full results are in Appendices M–O, including the generator-free nFID analysis.
  3. Relation to prior work (nSLj). We discuss SDEdit (Section 2.2), decoder post-training for autoregressive generators (Section 6), and latent denoising tokenizers (Section 8). Appendix P compares training setups and reports a comparison on the official MAETok checkpoint, where DA improves gFID from 1.67 to 1.55.
  4. Additional experiments in the main text (Rxvf, FikZ). Section 7 now reports DA gains in six representation spaces and their aggregate FDr$^6$ (Table 3), a noise-range ablation for CFG-aware DA (Table 4), and results on MAETok. Section 4.2 shows that GAR-FID also predicts generation rankings in five representation spaces beyond Inception (Appendix Q).
  5. Analysis and presentation (FikZ). We add a failure-case analysis of DA (Appendix L), revise the legend of Figure 3, and use green downward arrows in Table 2.

Table, figure, and appendix numbers in our responses refer to the revised manuscript. We thank the reviewers again for their thoughtful comments, which have helped us clarify the scope, evidence, and generality of our framework.

AuthorsOpenReview ↗

Discussion Summary for the Area Chair

Dear Area Chair,

We thank the reviewers for a constructive discussion. All three reviewers raised their scores after the rebuttal: Rxvf (2→3), nSLj (2→3), and FikZ (3→4). The final ratings are 3/3/4, all recommending acceptance, and each reviewer stated that their concerns were addressed.

Our paper explains why reconstruction quality does not predict generation quality: the decoder is trained on encoder latents but receives generated latents at inference. GAR constructs a controlled path between these two distributions. With everything else fixed, the Latent-FD to encoder latents increases monotonically along this path, and GAR-FID, measured against the reconstruction source images, rises with it in perfect rank agreement (Spearman ρ=1.0\rho = 1.0). This gives empirical evidence that directly links the latent distribution mismatch to the FID gap. Building on this, GAR-FID tracks gFID with rank correlations of 0.94-0.99 across 13 tokenizers. Decoder adaptation (DA) then improves generation with the encoder and generator frozen and no inference cost.

The rebuttal resolved the main concerns with new experiments:

  • Benefit beyond corruption training (Rxvf, nSLj). We compared DA against fine-tuning on directly corrupted latents with matched objectives and updates. Corrupted latents worsen gFID from 3.47 to 9.91, while DA improves it to 3.01. nSLj called this "convincing proof" that the gain is not explained by robustness to corruption.
  • Relation to prior post-training (nSLj). On the official MAETok checkpoint, DA improves gFID from 1.67 to 1.55. Prior post-training worsens gFID in five of its six reported settings.
  • Generality (Rxvf). DA gains hold in six representation spaces, reducing FDr$^6$ by 13–18%. GAR-FID also predicts generation rankings in all six spaces.
  • Depth of DA analysis (FikZ). We added noise-range ablations and a failure-case analysis.

Rxvf's concern about tokenizer selection was resolved by clarifying scope: GAR-FID diagnoses existing tokenizer-generator systems. Following FikZ's final suggestion, the revised main text now also summarizes the failure-case analysis.

We believe the paper offers a well-supported explanation of the reconstruction-generation gap and a practical fix.

Sincerely, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.