acceptodds
Under review as a conference paper at ICLR 2027

CUDiT: Cubic Uniform Diffusion for Efficient Generation of High-Dimensional Discrete Visual Representations

Abstract

Discrete visual generation casts image synthesis as token prediction, providing a modeling interface shared with language and a natural path toward unified multimodal architectures. Recent visualtokenizers pursue larger effective vocabularies to reduce quantization error and higher-dimensional representations to retain the semantic information required for visual understanding. Conventional quantization methods face substantial challenges when scaling to high-dimensional representations and large codebooks. Dimension-wise quantization provides a simple and effective route to near-lossless quantization, but induces an enormous and highly redundant product space. The enormous code space makes direct joint categorical modeling intractable, so practical generation must rely on parallel factorization. The existing absorbing-diffusion approach uses large sampling budgets in this space and reports substantially lower quality at fewer steps. Our analysis shows that the resulting mismatch between the product of coordinate-wise marginals and the joint distribution becomes increasingly severe as fewer sampling steps require larger parallel updates. To address this problem, we propose CUDiT (Cubic Uniform Diffusion Transformer), a full-support discrete diffusion framework for efficient parallel generation of high-dimensional representation cubes. By denoising through continuous-time Markov chain dynamics over noisy token states, avoids the severe degradation that arises when a model trained only on clean tokens encounters the intrinsic noise of parallel generation. On ImageNet-256, -XL achieves gFID in steps, delivering comparable generation quality with up to two orders of magnitude fewer estimated backbone FLOPs than CubiD.

Primary area:
generative models
Keywords:
image generation, diffusion model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_DxumOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets efficient generation of high-dimensional discrete visual tokens (16×16×768 cubes from dimension-wise quantized DINOv2/RAE features). The prior method, CubiD, uses mask diffusion and needs up to 1536 steps, degrading sharply with fewer. The authors attribute this to parallel factorization error: independently sampling many correlated coordinates produces inconsistent tokens that a mask-trained denoiser never sees during training. They propose CUDiT, which replaces mask corruption with uniform-noise Discrete Flow Matching so the model learns from noisy content states. On ImageNet 256×256, CUDiT-XL achieves gFID 1.99 in 64 steps, matching CubiD-XL at 1536 steps with roughly 40× lower estimated compute. An ablation suggests the gain comes from the training distribution rather than the ability to revise tokens.

Critical strengths
  1. Important research problem. Discrete generation on high-dimensional semantic tokens is a promising route toward unified multimodal models, but existing methods such as CubiD [3] require up to 1536 sampling steps for high-quality results, which limits practical use. Reducing this cost is a timely and well-motivated problem, and progress here could directly benefit future unified architectures.

  2. Strong empirical efficiency gains. CUDiT-XL reaches gFID 1.99 in 64 steps, matching CubiD-XL at 1536 steps with roughly 40× fewer estimated FLOPs, and remains stable across 32 to 128 steps. The revisability ablation also offers a useful insight: training on noisy content states, rather than the ability to revise tokens alone, appears to drive the improvement. This makes the paper a valuable empirical reference for this setting.

Critical weaknesses
  1. Limited novelty. The factorization analysis closely follows ReDi [1], which already characterizes factorization error via conditional total correlation and shows it grows in few-step generation. The method is also close to URSA [2], which applies uniform discrete diffusion with a similar timestep shift to visual generation and already shows its advantage over masked diffusion. CUDiT's genuine increments are the extension to high-dimensional cubes and the revisability ablation, but without discussing these works, the contribution is overstated and hard to assess.

  2. Unfair baseline comparison. The masked baselines may not be at their best. Stronger samplers for masked diffusion (e.g., confidence-based unmasking or remasking) are not considered, and in the key ablation (Table 6) the mask variants reach gFID ~26 at 64 steps, far worse than the ~9 CubiD[3] reports at 64 steps in its own ablation. The reported gap may therefore be partly inflated.

  3. Limited few-step range. The lowest budget reported is 32 steps. Results at 16 or 8 steps would better support the few-step claim, as the few-step literature typically targets this regime.

Minor:

  1. I found the text font size in the main pipeline figure (Fig. 3) is too small and is hard to read.

  2. The paper does not report the compute required for training (e.g., hardware or GPU hours).

Reference:

[1] Yoo et al. ReDi: Rectified Discrete Flow. NeurIPS 2025.

[2] Deng et al. Uniform Discrete Diffusion With Metric Path for Video Generation. ICLR 2026.

[3] Wang et al. Cubic Discrete Diffusion: Discrete Visual Generation on High-Dimensional Representation Tokens. CVPR 2026.

Questions for authors
  1. How does CUDiT differ from ReDi in its factorization analysis and from URSA in its method? Please clarify the specific increment beyond these works.

  2. Why do the mask variants (gFID 26 at 64 steps) perform much worse than CubiD’s own reported result (9 at 64 steps)? Did they use CubiD’s original masking schedule and hyperparameters? Please also clarify how the CTMC-Mask variant revises previously revealed tokens under an absorbing path.

  3. Could you report CubiD at 64 steps with stronger samplers (e.g., confidence-based unmasking or remasking)? If the gap to CUDiT remains large, this would substantially strengthen the main claim.

  4. How does CUDiT perform at 16 or 8 steps?

10 comments
AuthorsOpenReview ↗

Response to Reviewer Dxum (part 1/4)

We sincerely thank the reviewer for the thoughtful and constructive review, and for recognizing the importance of efficient discrete generation for high-dimensional semantic representations and its potential relevance to future unified multimodal architectures. We greatly appreciate the recognition of CUDiT's substantial improvements in sampling efficiency and generation quality, as well as the insights from our controlled ablations highlighting the importance of training on noisy content states. All changes are marked in green in the revised manuscript.

[W1&Q1] How does CUDiT differ from ReDi in its factorization analysis and from URSA in its method? Please clarify the specific increment beyond these works.

[A1] We thank the reviewer for highlighting ReDi and URSA. We acknowledge that conditional total correlation (TC), uniform discrete diffusion, and CTMC sampling have been explored in prior work. CUDiT addresses a distinct challenge in high-dimensional, dimension-wise quantized visual representations. We clarify the differences below.

(1) Comparison with ReDi: Factorization analysis and methodology.

ReDi analyzes factorization error through Conditional TC and addresses it via iterative coupling rectification. In contrast, CUDiT focuses on the severe factorization error arising from parallel sampling across both spatial and channel dimensions in cubic representations. A 768×16×16768\times16\times16 cube contains 196,608 categorical coordinates, introducing 768 times more prediction coordinates than spatial-only tokenization at the same resolution. Strong cross-channel dependencies make independent parallel sampling particularly challenging.

We further develop a linear-code model showing that, even with exact coordinate-wise marginals, the probability of obtaining a jointly valid sample can decrease exponentially when the parallel update size exceeds the intrinsic dimension. Unlike ReDi, which directly reduces Conditional TC through coupling rectification, CUDiT mitigates the consequences of factorization errors through full-support denoising, without iterative coupling updates or additional distillation.

(2) Comparison with URSA: Problem setting and methodology.

URSA studies uniform discrete diffusion and CTMC-based refinement for image and video generation. CUDiT instead targets high-dimensional discrete semantic representations produced by dimension-wise quantization, whose enormous and highly redundant product space makes few-step parallel generation particularly difficult.

Our key observation is that parallel factorization produces noisy, potentially inconsistent intermediate states, while absorbing-mask diffusion trains primarily on clean tokens and masks. Full-support uniform corruption addresses this train–inference mismatch by exposing the denoiser to noisy content states during training, improving robustness to sampling errors. We do not claim uniform diffusion or CTMC sampling as new primitives; rather, we identify and validate their effectiveness in this specific representation regime.

(3) Experimental validation.

Our controlled ablation distinguishes full-support corruption from token revisability. Under the same irreversible sampler, switching the corruption source from mask to uniform improves gFID from 26.64 to 6.52. Holding uniform corruption fixed, enabling revisability provides an additional improvement from 6.52 to 3.95. These results identify full-support uniform corruption as the larger contributor while also showing that revisability provides a meaningful additional benefit. Furthermore, CUDiT-XL achieves gFID 1.99 in 64 steps, compared with CubiD-XL's 2.06 in 1,536 steps.

In summary, CUDiT's contribution beyond ReDi and URSA lies in (i) identifying and quantitatively characterizing the factorization bottleneck in high-dimensional cubic representations, (ii) mitigating its consequences through full-support noisy-content training, and (iii) experimentally separating the larger gain from corruption support from the additional benefit provided by token revisability. We will clarify these distinctions in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer Dxum (part 2/4)

[W2&Q2] Why do the mask variants (gFID 26 at 64 steps) perform much worse than CubiD’s own reported result (9 at 64 steps)? Did they use CubiD’s original masking schedule and hyperparameters? Please also clarify how the CTMC-Mask variant revises previously revealed tokens under an absorbing path.

[A2.1]

Why do the mask variants (gFID 26 at 64 steps) perform much worse than CubiD’s own reported result (9 at 64 steps)? Did they use CubiD’s original masking schedule and hyperparameters?

We thank the reviewer for pointing out this discrepancy. We clarify that the mask-based variants in Table 6 are controlled ablations within our CUDiT framework, rather than faithful reproductions of CubiD. Their purpose is to isolate the effects of token revisability and corruption support under matched experimental conditions. The main differences from CubiD are:

  1. Architecture and training: CubiD uses a bidirectional Transformer for masked-token prediction, whereas our variants adopt a LightningDiT-B backbone with explicit timestep conditioning and a CTMC-based training objective.

  2. Sampling: CubiD progressively unmasks tokens without revision, while our CTMC-Mask variant allows previously generated tokens to be revised. We additionally introduce an Irreversible Mask variant to isolate the effect of revisability.

  3. Scheduler: All our variants share the same time-shift schedule, 64-step sampling budget, and temperature (τ=1\tau=1), rather than CubiD's original cosine schedule, to ensure controlled comparisons.

    These differences may explain the discrepancy in absolute gFID values. We acknowledge that the shared time-shift schedule may disadvantage mask-based generation. To further examine this factor, we conduct additional experiments using a cosine-based schedule for the mask variants:

MethodNoise schedulerSourceRevisablegFID↓IS↑
Irreversible Masktime-shiftMaskNo26.6487.7
Irreversible MaskcosineMaskNo14.8128.4
CTMC-Masktime-shiftMaskYes25.3195.19
CTMC-MaskcosineMaskYes11.5144.0
CTMC-Uniformtime-shiftUniformYes3.95192.2

Using the cosine schedule, which is better suited to mask diffusion, substantially improves generation quality, although a considerable performance gap remains compared with the uniform diffusion approach.

  1. Training budget and capacity: Our ablation models use a 0.6B-parameter backbone trained from scratch for 100 epochs, whereas CubiD's reported few-step result uses a larger backbone (0.9B) trained for 150 epochs. The corresponding intermediate checkpoint is not publicly available, preventing exact reproduction under our protocol.

Our 64-step evaluation using the publicly released CubiD-Huge-800epochs checkpoint and code yields a gFID of 21.17, compared with the 9.14 reported in CubiD's ablation study. The original 64-step ablation used a different checkpoint, which is not publicly available; therefore, this is not an exact like-for-like reproduction. Nevertheless, we carefully verified the public experimental configuration and repeated the 64-step evaluation multiple times, obtaining consistent results. Using the same public checkpoint and configuration at the default 1536 steps also produces results closely matching those reported in CubiD's main table (Table 5), corroborating our reproduction protocol on the publicly available assets. Taken together, these results suggest that, with the released checkpoint and code, CubiD's few-step degradation could be substantially more severe than the original ablation indicates. This discrepancy reinforces the practical importance of robust few-step generation and the performance gains achieved by CUDiT.

AuthorsOpenReview ↗

Response to Reviewer Dxum (part 3/4)

[A2.2]

Please also clarify how the CTMC-Mask variant revises previously revealed tokens under an absorbing path.

Thank you for the careful reading. The CTMC-Mask variant uses an absorbing-mask path for training corruption: training states contain clean content and MASK, without full-support content corruption. At inference, however, our sampler removes the monotonic freeze and additionally permits previously revealed coordinates to undergo content-to-content revisions. We do not claim that this modified inference sampler strictly follows the canonical reverse CTMC induced by the standard absorbing-mask probability path; it is designed to isolate the effect of revisability while keeping mask-based training fixed.

Concretely:

  1. Training corruption path. The forward corruption follows an absorbing mixture of clean content and the mask token (no full-support content corruption). Inference starts from all-MASK.
  2. Inference update. At each Euler step the denoiser predicts a clean posterior; we sample a target token x1x_1 and form jump rates toward that token. A site jumps with probability 1−exp⁡(−h⋅intensity)1-\exp(-h\cdot\mathrm{intensity}).
  3. How revision happens. Because the inference sampler does not impose the irreversible constraint, already-revealed content tokens remain eligible to jump. When they jump, they are typically rewritten content → content toward the newly sampled x1x_1.

By contrast, Irreversible Mask uses the same absorbing-mask training corruption but forbids further jumps once a site leaves MASK, so revealed tokens stay fixed.

Reviewer_p8DFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper targets fast generation of dimension-wise quantized representation cubes. Following CubiD, an image is encoded by a frozen DINOv2-B encoder into 16×16×768 features. Each scalar is then quantized independently to one of 8 levels, giving 196,608 categorical coordinates that a RAE decoder maps back to pixels. CubiD generates these cubes with absorbing mask diffusion using up to 1536 steps, and its quality drops quickly with fewer steps (gFID 2.02 at 1536 steps versus 8.25 at 128 for CubiD-XXL). The authors attribute this to parallel factorization error. Sampling many correlated coordinates from their marginals in one step deviates from the joint distribution by the conditional total correlation, and a linear-code example shows this can become severe once the block size exceeds the intrinsic dimension. The resulting inconsistent tokens are then fed to a denoiser that was trained only on clean tokens and masks.

CUDiT replaces the absorbing path with the uniform-source mixture path of discrete flow matching, so every training state contains legal but partly random content. It samples with Euler steps of the corresponding CTMC, which lets any coordinate be resampled later. It also uses the generalized KL objective of Gat et al., CubiD's spatial packing into 256 tokens of width 768 with per-channel 8-way heads, a LightningDiT backbone and a dimension-aware time-shift schedule. On class-conditional ImageNet 256 without guidance, CUDiT-XL (1.62B) reaches gFID 1.99 with 64 steps and τ = 0.75, against 2.06 for CubiD-XL with 1536 steps. The paper reports up to two orders of magnitude lower estimated backbone compute. Ablations cover the noise schedule, temperature, number of steps and model size, plus a controlled comparison. In it, allowing mask predictions to be revised changes gFID only from 26.64 to 25.31, while switching to uniform corruption gives 3.95.

Critical strengths
  • The paper addresses a real bottleneck for high-dimensional discrete tokens, and the gain is large. Representations of this kind are attractive for unified models, but at 1536 steps they are expensive to sample. CUDiT reaches a slightly better gFID than CubiD-XL with 24 times fewer steps, and it degrades gracefully as the budget shrinks: CUDiT-L gets 2.35 at 32 steps, while CubiD-XXL is already at 8.25 with 128 steps. Results improve monotonically from B to XL. For anyone who wants to use dimension-wise quantized features in practice, this makes the representation much more usable.

  • The ablations are informative and mostly well controlled. Table 6 trains three variants from scratch with the same backbone, data, schedule and step budget, and changes only the source distribution and whether sampled tokens can be revised. This gives direct evidence that, under mask training, adding the ability to rewrite tokens does little, while training on noisy content states makes the difference. The schedule study shows that the dimension-aware time shift is essential in this setting (3.95 against 10.53 for a linear schedule). The temperature sweep gives a clear U-shaped curve with the best gFID near τ = 0.75. Together these form a recipe others can reuse.

  • The analysis gives a clear account of why mask diffusion degrades with fewer steps. Framing the error of parallel sampling as conditional total correlation makes the mechanism easy to understand. So does the linear-code illustration, where exact marginals still produce mostly invalid samples: only 0.5% are valid when 16 coordinates with 8 degrees of freedom are sampled at once. The paper is also careful about scope. Appendix B states that the analysis does not show that uniform corruption removes the factorization error, and the empirical claims are tied to the ablation rather than to the theory.

Critical weaknesses
  • The related work describes absorbing transitions as the predominant choice in visual generation, but corrupting image tokens with random replacements to counter error accumulation has clear precedents. VQ-Diffusion introduced a mask-and-replace strategy explicitly "to avoid the accumulation of errors", yet it appears in this paper only as a row of Table 2. Uniform corruption of image data goes back to multinomial diffusion and the uniform kernels of D3PM. Campbell et al. (2022) applied continuous-time discrete diffusion to images, and Token-Critic and DPC already add correction to masked image generation. Recent work, mostly on text, such as GIDD, Duo, ReMDM and discrete copula diffusion, studies the same combination of uniform noise, self-correction and factorization error. Every component of CUDiT also comes from prior work: the path, loss and sampler from Gat et al., the packing from CubiD, and the schedule from RAE. The contribution is therefore the finding that a known corruption helps much more in this regime. The paper should present it that way and explain why the effect is so large here, for example because of the 196,608 coordinates or the per-coordinate vocabulary of only 8 values.

  • Several comparisons needed to support the explanation are missing. The authors mention that adding random perturbations to visible tokens during mask training gave limited improvement, but report no numbers, and this design is close to mask-and-replace. A masked model with a corrector or remasking at 64 steps would be a natural comparison. So would an ordinal noise kernel, since the 8 levels are ordered while both uniform noise and the categorical loss treat them as unordered. Table 6 is also missing one cell: uniform training with an irreversible sampler that freezes each coordinate after its first update. Correcting errors plausibly needs both noisy-state training and the ability to rewrite tokens. Table 6 shows only that revisability alone is not enough, so it does not yet establish that revisability plays no important role.

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer p8DF (part 1/3)

We sincerely thank the reviewer for the thorough and constructive review, and for recognizing the substantial improvements in sampling efficiency and generation quality achieved by CUDiT, the informative and carefully controlled ablations that validate our design choices, and our analysis of parallel factorization error through conditional total correlation. We particularly appreciate the recognition of the practical relevance of our approach for high-dimensional discrete representations and the careful discussion of the scope and limitations of our theoretical analysis. All changes are marked in green in the revised manuscript.

[W1] The related work describes absorbing transitions as the predominant choice in visual generation, but corrupting image tokens with random replacements to counter error accumulation has clear precedents. VQ-Diffusion introduced a mask-and-replace strategy explicitly "to avoid the accumulation of errors", yet it appears in this paper only as a row of Table 2. Uniform corruption of image data goes back to multinomial diffusion and the uniform kernels of D3PM. Campbell et al. (2022) applied continuous-time discrete diffusion to images, and Token-Critic and DPC already add correction to masked image generation. Recent work, mostly on text, such as GIDD, Duo, ReMDM and discrete copula diffusion, studies the same combination of uniform noise, self-correction and factorization error. Every component of CUDiT also comes from prior work: the path, loss and sampler from Gat et al., the packing from CubiD, and the schedule from RAE. The contribution is therefore the finding that a known corruption helps much more in this regime. The paper should present it that way and explain why the effect is so large here, for example because of the 196,608 coordinates or the per-coordinate vocabulary of only 8 values.

We sincerely thank the reviewer for the highly constructive feedback on the presentation and clarity of our manuscript. In response to these valuable suggestions, we have made the following revisions:

[A1.1]

The related work describes absorbing transitions as the predominant choice in visual generation, but corrupting image tokens with random replacements to counter error accumulation has clear precedents. VQ-Diffusion introduced a mask-and-replace strategy explicitly "to avoid the accumulation of errors", yet it appears in this paper only as a row of Table 2. Uniform corruption of image data goes back to multinomial diffusion and the uniform kernels of D3PM. Campbell et al. (2022) applied continuous-time discrete diffusion to images, and Token-Critic and DPC already add correction to masked image generation. Recent work, mostly on text, such as GIDD, Duo, ReMDM and discrete copula diffusion, studies the same combination of uniform noise, self-correction and factorization error.

We agree with your suggestion to expand and clarify the discussion of prior work. We have updated the Related Work section to explicitly acknowledge and emphasize prior studies on uniform and non-absorbing discrete diffusion, error accumulation and correction mechanisms in masked generation, and approaches addressing the limitations of parallel factorization, including VQ-Diffusion, Token-Critic, DPC, GIDD, ReMDM, Duo, and Discrete Copula Diffusion.

[A1.2]

Every component of CUDiT also comes from prior work: the path, loss and sampler from Gat et al., the packing from CubiD, and the schedule from RAE. The contribution is therefore the finding that a known corruption helps much more in this regime. The paper should present it that way and explain why the effect is so large here, for example because of the 196,608 coordinates or the per-coordinate vocabulary of only 8 values.

We agree that uniform diffusion and its underlying mathematical framework are established techniques rather than original contributions of this work.Our paper instead focuses on identifying and analyzing the practical challenges of modeling high-dimensional representations produced by cubic quantization, a promising approach to discrete visual representation learning. We further investigate the underlying causes of these challenges and provide a mathematical characterization of parallel factorization error through conditional total correlation. Guided by this analysis, we address these challenges using established techniques and conduct controlled ablation studies to validate their effectiveness.

We have revised the contribution statements in the updated manuscript to more accurately reflect the novelty and scope of our work, emphasizing our analysis and mathematical characterization of the challenges arising from high-dimensional cubic representations, as well as the empirical effectiveness of applying full-support uniform diffusion in this setting.

AuthorsOpenReview ↗

Response to Reviewer p8DF (part 2/3)

[W2] Several comparisons needed to support the explanation are missing. The authors mention that adding random perturbations to visible tokens during mask training gave limited improvement, but report no numbers, and this design is close to mask-and-replace. A masked model with a corrector or remasking at 64 steps would be a natural comparison. So would an ordinal noise kernel, since the 8 levels are ordered while both uniform noise and the categorical loss treat them as unordered. Table 6 is also missing one cell: uniform training with an irreversible sampler that freezes each coordinate after its first update. Correcting errors plausibly needs both noisy-state training and the ability to rewrite tokens. Table 6 shows only that revisability alone is not enough, so it does not yet establish that revisability plays no important role.**

[A2.1]

The authors mention that adding random perturbations to visible tokens during mask training gave limited improvement, but report no numbers.

Thank you for the careful reading. We train two models with the same parameter count (0.6B) for 100K steps on a subset of ImageNet, following identical training recipes, with the only difference being noise injection. For the noise-injected variant, after standard per-element masking, each sample draws a corruption ratio r∼Uniform[0,0.2]r \sim \mathrm{Uniform}[0, 0.2] and replaces a fraction rr of its unmasked positions with values sampled uniformly from the discrete codebook. Masked positions and reconstruction targets remain unchanged, and the model continues to predict only masked tokens using cross-entropy loss. Noise is applied only during training.

We evaluate both models under identical conditions using 64-step inference. The noise-injected model exhibits a slight degradation in FID but a substantial improvement in IS. These results suggest that simply introducing noisy content tokens during training does not fundamentally resolve the few-step generation problem.

Noise ratiogFIDIS ↑
012.73148.2
0.213.59171.8

[A2.2]

A masked model with a corrector or remasking at 64 steps would be a natural comparison. So would an ordinal noise kernel, since the 8 levels are ordered while both uniform noise and the categorical loss treat them as unordered.

We thank the reviewer for this insightful suggestion. We evaluate the publicly available CubiD-Huge checkpoint using 64-step inference with three sampling strategies: the default random mask schedule, confidence-based position selection, and remasking. The results are presented below. Both alternative strategies exhibit severe degradation in generation quality compared with the default cosine schedule, failing to produce meaningful images. These findings highlight the distinctive characteristics of cubic-quantized tokens compared with conventional quantized tokens, suggesting that preserving randomness in the selection of positions to unmask is crucial, and that allowing previously unmasked tokens to be revised through remasking does not resolve the few-step generation problem.

SamplerDescriptiongFID ↓IS ↑
Random order (default)CubiD’s original fixed random order + progressive unmasking21.17106.04
Confidence-based unmaskingMaskGIT-style: each step reveals the top-N still-masked positions by token confidence63.3420.51
Revisable remaskingKeep top-K confident positions; remask the rest so early mistakes can be revised89.6510.99

And we agree that an ordinal noise kernel is a promising direction, as it could explicitly exploit the ordered structure of the eight quantization levels, which is not captured by our current uniform corruption formulation. Exploring such structured corruption processes may further improve discrete generation, and we consider this a valuable direction for future work.

AuthorsOpenReview ↗

Response to Reviewer p8DF (part 3/3)

[A2.3]

Table 6 is also missing one cell: uniform training with an irreversible sampler that freezes each coordinate after its first update. Correcting errors plausibly needs both noisy-state training and the ability to rewrite tokens. Table 6 shows only that revisability alone is not enough, so it does not yet establish that revisability plays no important role.

We thank the reviewer for this question. We agree that the original ablation did not fully isolate the contribution of token revisability under uniform corruption. We have conducted the additional experiment using uniform training with an irreversible sampler that freezes each coordinate after its first update. The results are presented below and have been incorporated into the revised manuscript. Under the same irreversible sampler, changing the corruption source from mask to uniform improves gFID from 26.64 to 6.52. Holding uniform corruption fixed, enabling revisability further improves gFID from 6.52 to 3.95. Thus, full-support uniform corruption accounts for the larger improvement, while revisability provides a clear additional benefit.

MethodSourceRevisablegFID↓IS↑
Irreversible MaskMaskNo26.6487.7
CTMC-MaskMaskYes25.3195.19
Irreversible UniformUniformNo6.52181.3
CTMC-UniformUniformYes3.95192.2
Reviewer_TDZsOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CUDiT studies few-step generation of high-dimensional discrete visual representations obtained by dimension-wise quantization of frozen DINOv2 features. It replaces absorbing-mask corruption with a full-support uniform source and learns CTMC denoising under a shifted probability path and a rate-weighted generalized KL objective. The paper motivates the approach through parallel factorization error and tests the role of corruption support versus token revisability. On ImageNet-256, CUDiT-XL reports generation FID 1.99 in 64 steps, compared with 2.06 in 1,536 steps for CubiD-XL.

Critical strengths
  1. The reported quality-versus-step tradeoff is substantial and practically relevant for high-dimensional discrete generation. CUDiT-XL obtains 1.99 FID in 64 steps without classifier-free guidance (Table 2), while CUDiT-L remains competitive at 32 steps with 2.35 FID (Table 5). These results support the usefulness of the proposed corruption/training formulation beyond a single operating point.

  2. The controlled corruption ablation is particularly informative. Table 6 uses the same backbone, representation, optimizer, time-shift schedule, 100-epoch training budget, 64 sampling steps, and temperature 1. Irreversible Mask, CTMC-Mask, and CTMC-Uniform obtain FIDs of 26.64, 25.31, and 3.95, respectively. This strongly supports an effect of full-support corruption beyond merely enabling token revisions under the tested protocol. Appendix B also appropriately states that the stylized analysis does not prove elimination of factorization error or exact matching of learned-sampler errors.

Critical weaknesses
  1. The finite-step sampling algorithm needs a more precise specification. Equations (8) and (10) use an Euler update of the form delta + hu. Valid continuous-time rates alone do not guarantee a valid categorical distribution for an arbitrary finite step: the probability of retaining the current token is 1 - hlambda_t*(1-p_current), where lambda_t = kappa'_t/(1-kappa_t), and this must be nonnegative. Figure 3 indicates that all positions are updated at the final step, but the text does not fully specify the time grid, any rate clipping or alternative discretization, and how the final update relates to the stated formula. This matters for reproducing the central 64-step result. It is an implementation clarification, not evidence that the actual code samples negative probabilities.

  2. The practical efficiency and baseline sensitivity remain incompletely characterized. The XL comparison establishes 24 times fewer steps and about 40.5 times lower estimated backbone FLOPs (1,430.2 versus 35.3 TFLOPs), rather than measured end-to-end acceleration. The large output heads, memory costs, and representation decoding can affect realized speed. Also, Table 6 fixes the same time-shift schedule across all corruption variants; this is a useful control, but it does not establish that the mask variants remain as far behind when their schedules are tuned independently. These limitations constrain the strength of the efficiency and mechanism claims, while leaving a strong empirical result in the reported setting.

Questions for authors
  1. Please provide the actual 64-step sampling pseudocode, including the time grid, handling of hlambda_t(1-p_current) greater than 1, and the final all-position update shown in Figure 3. Does the implementation use the Euler formula as written, or a modified transition rule?

  2. What are the end-to-end latency and peak memory relative to CubiD on the same device and batch size, including output heads and decoding? How sensitive are the mask results in Table 6 to separately tuned noise schedules? These clarifications would determine how broadly the reported efficiency and corruption comparison can be interpreted.

4 comments
AuthorsOpenReview ↗

Response to Reviewer TDZs (part 1/2)

We sincerely thank the reviewer for the thorough and constructive review, and for recognizing the substantial quality–efficiency tradeoff achieved by CUDiT, the practical benefits of full-support uniform corruption demonstrated through controlled ablations, and the careful discussion of the scope and limitations of our theoretical analysis. We greatly appreciate these encouraging assessments and insightful suggestions. All changes are marked in green in the revised manuscript.

[W1&Q1] Please provide the actual 64-step sampling pseudocode, including the time grid, handling of hlambda_t(1-p_current) greater than 1, and the final all-position update shown in Figure 3. Does the implementation use the Euler formula as written, or a modified transition rule?

[A1] We thank the reviewer for requesting a more precise specification of the finite-step sampler. We clarify the actual 64-step sampling procedure below and have highlighted the relevant implementation details with comments.

(1) A 64-step sampling pseudocode.

N = 64                 # number of sampling steps (NFE)
time_eps = 5e-3        # truncate schedule away from t=1
T = 0.75               # sampling temperature
V = 8                  # 3-bit vocabulary

# ---- time grid ----
t0, t1 = 0.0, 1.0 - time_eps
h_nom = 1.0 / N
n_steps = ceil((t1 - t0) / h_nom)

t_grid = [t0 + h_nom * i for i in range(n_steps)] + [t1]

# ---- initialization ----
x = sample_x0()  # Unif{0,...,V-1}^{C×H×W}

for i in range(n_steps):
    t = t_grid[i]
    h = t_grid[i + 1] - t_grid[i]

    # Predict and independently sample a proposal at each position
    p = softmax(model(x, t) / T, dim=-1)
    x1 = categorical(p)

    if i == n_steps - 1:
        # Final all-position update
        x = x1
    else:
        # Linear path: kappa_t = t
        lam = 1.0 / (1.0 - t)

        # Hard-proposal jump rate, with zero self-transition rate
        intensity = lam * (x1 != x)

        # h*lam < 1 under the specified time grid
        # No clipping is required, even for Euler probabilities
        # Exponential jump probability (modified Euler rule)
        jump_prob = 1.0 - exp(-h * intensity)

        # Independent Bernoulli decisions for all positions
        jump_mask = rand_like(x) < jump_prob
        x = where(jump_mask, x1, x)

(2) Euler discretization versus the implemented transition rule.

The implementation uses a modified first-order CTMC sampling rule rather than the literal Euler update δ+hu\delta+hu.

Specifically, we first sample a hard proposal x^∗1∼p∗θ(⋅∣xt,t)\hat{x}*1\sim p*\theta(\cdot\mid x_t,t), then jump toward this proposal with probability 1−exp⁡(−hλt)1-\exp(-h\lambda_t) if it differs from the current token.

After marginalizing over the sampled proposal, the resulting transition probability for j≠xtj\neq x_t is

=(1-e^{-h\lambda_t})p_\theta(j\mid x_t,t),$$ whereas the literal Euler discretization gives $$P_{\mathrm{Euler}}(x_t\to j) =h\lambda_t p_\theta(j\mid x_t,t).$$ Since $1-e^{-h\lambda_t}=h\lambda_t+O(h^2)$, the two transition rules agree to first order but differ at finite step sizes. The exponential rule produces valid transition probabilities and is slightly more conservative than Euler. For our time grid and $\lambda_t=1/(1-t)$, we have $h\lambda_t<1$, so even the literal Euler probabilities would remain valid without clipping. The exponential rule is therefore an implementation choice rather than a necessary correction for probability overflow. Finally, the last step directly resamples all positions from the predicted categorical distribution, consistent with Figure 3. This terminal update is applied separately from the preceding CTMC transitions.
AuthorsOpenReview ↗

Response to Reviewer TDZs (part 2/2)

[W2&Q2] What are the end-to-end latency and peak memory relative to CubiD on the same device and batch size, including output heads and decoding? How sensitive are the mask results in Table 6 to separately tuned noise schedules? These clarifications would determine how broadly the reported efficiency and corruption comparison can be interpreted.

[A2.1]

What are the end-to-end latency and peak memory relative to CubiD on the same device and batch size, including output heads and decoding?

Thank you for the careful reading. We measure end-to-end latency and peak memory relative to CubiD on the same single A100-SXM-80GB with batch size 16, including output heads and decoding.

Setting#ParamsStepsgFIDLatency (s/img)Rel. latencyPeak alloc (GB)Rel. peak mem
CubiD-Huge3.7B15362.0220.50081.000×16.4161.000×
CubiD-Huge3.7B5122.636.90450.337×16.4171.000×
CubiD-Huge3.7B2563.963.45830.169×16.4171.000×
CUDiT-B0.6B642.390.52410.026×5.2710.321×
CUDiT-L1.0B1281.990.61310.030×6.6010.402×
CUDiT-L1.0B642.110.30760.015×6.6010.402×
CUDiT-L1.0B322.350.15550.008×6.6010.402×
CUDiT-XL1.6B641.990.42020.020×8.7650.534×

At comparable high image generation quality, CUDiT-L@128 achieves a 33× speedup in end-to-end inference latency over CubiD-Huge@1536, while reducing peak memory usage to just 40% of CubiD's. Furthermore, CUDiT-XL@64 achieves a 50× speedup in end-to-end latency compared to CubiD-Huge@1536, while reducing peak memory usage by 46%.

[A2.2]

How sensitive are the mask results in Table 6 to separately tuned noise schedules?

We thank the reviewer for raising this question. We acknowledge that the shared time-shift schedule in Table 6 may disadvantage mask-based generation, as it was originally designed for our uniform diffusion model rather than independently optimized for each variant. To assess the sensitivity of the mask-based results to the noise schedule, we conducted additional experiments using a cosine-based schedule, following CubiD's original setting, while keeping the remaining experimental configurations unchanged. The results are presented below:

MethodNoise schedulerSourceRevisablegFID↓IS↑
Irreversible Masktime-shiftMaskNo26.6487.7
Irreversible MaskcosineMaskNo14.8128.4
CTMC-Masktime-shiftMaskYes25.3195.19
CTMC-MaskcosineMaskYes11.5144.0
CTMC-Uniformtime-shiftUniformYes3.95192.2

Using the cosine schedule, which is better suited to mask diffusion, substantially improves generation quality, although a considerable performance gap remains compared with the uniform diffusion approach.

thanks for the new reply! my concerns are resolved.

Thank you again for your time and for your positive assessment of our work. We are very glad to hear that our clarifications have fully addressed your concerns. We sincerely appreciate your constructive feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Brief Summary of Responses to Reviewer Area Chair

Dear ACs,

We sincerely appreciate your thoughtful effort in evaluating our submission. To assist your meta-review, we provide a concise summary of the reviewers' assessments and the rebuttal process below.

Our paper addresses the substantial modeling and generation efficiency bottleneck arising from CubiD-style dimension-wise quantization of high-dimensional semantic representations, a promising representation paradigm for unified multimodal architectures (Dxum S1, p8DF S1). As acknowledged by the reviewers, CUDiT significantly improves the quality–efficiency trade-off, achieving a gFID of 1.99 in only 64 sampling steps. Moreover, our theoretical analysis and controlled ablations provide valuable insights into the limitations of parallel generation and the importance of training on noisy content states (TDZs S1–S2, p8DF S1–S3, Dxum S2).

Building on these contributions recognized by the reviewers, we have further addressed their concerns through the following additional analyses, experiments, and manuscript revisions.

First, regarding empirical validation (TDZs W2, p8DF W2, Dxum W2), we completed the four-way controlled ablation and additionally evaluated cosine schedules, noise injection, confidence-based unmasking, and remasking. These experiments further substantiate full-support uniform corruption as the primary contributor to improved few-step generation.

Second, regarding practical efficiency and implementation (TDZs W1–W2, Dxum W3 and minor concerns), we demonstrated approximately 50× end-to-end speedup and 46% lower peak GPU memory at comparable generation quality, supplemented 8- and 16-step results, clarified sampling details, reported training costs, and improved figure readability.

Finally, regarding the framing of our contributions (p8DF W1, Dxum W1), we clarified that our contribution lies not in introducing new diffusion techniques, but in systematically analyzing the few-step generation bottleneck arising from CubiD's spatial-and-feature-wise factorization, addressing it with established techniques, and experimentally validating the underlying mechanism. We revised our contribution statements and expanded the related work accordingly.

We especially appreciate the follow-up feedback from reviewers p8DF and Dxum, both of whom explicitly confirmed that their concerns had been resolved, with reviewer Dxum raising their score to 3. The current ratings from all three reviewers are 3, 3, and 3.

We believe these revisions and supplementary experiments have further strengthened the technical soundness, empirical evidence, and practical significance of our work. We sincerely appreciate the reviewers' constructive feedback and your thoughtful consideration, and look forward to your positive recommendation.

Sincerely,

Authors of Submission #7162

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.