- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This work builds on extending drifting models to categorical or discrete features. Drifting models are designed specifically for one step generation. The key idea of this work is that (continuous) drifting models are designed for Euclidean space, and they are ill-suited for discrete categorical data over a probability simplex. The authors replace the Euclidean penalty terms in continuous drifting models with a Bregman divergence term, and from a mathematical view, the authors drive a closed-form algorithm for categorical drifting models. So the model constructs a local log-evidence field comparing real vs. generated neighbor distributions first, and then additively tilts the logits, and then trains the generator using cross-entropy against this frozen soft target. This proposed method is evaluated on discretized images, DNA sequences, SMILES strings, and good performance is shown.
Critical strengths
- This is a novel or the frist extension of drifting models to categorical data as far as I know (also with the help from AI to do a literature review)-
- The motivation is mathematically sound, the resulting closed-form softmax tilt is intuitive, simple to implement, and naturally preserves the simplex constraints for categorical data
Critical weaknesses
- I do have some questions, I won't frame them as weaknesses for now since it is possible that I misunderstood. Please refer to questions.
The paper positions CatDrift as an alternative to methods that require "expensive training pipelines, pretrained teachers, or additional adaptation stages." However, the local improvement field heavily relies on an external, pre-trained Masked Autoencoder (MAE) to compute the feature map () for the locality kernel. This somewhat undercuts the "trained from scratch" claim, as the success of the local neighborhood lookup is bottlenecked by the quality of this separate representation-learning phase.
Questions for authors
-
The authors claim CatDrift as a alternative to methods that require expensive training pipelines, pretrained teacher models, or additional adaptation stages. However, based on my understanding, the local improvement field in this work still relies on a pretrained MAE? Then how is this work an alternative?
-
Why is the eval on CIFAR-10 on only 1,000 examples? If computational cost is not a constraint (which I don't think it would be since CIFAR-10 is pretty small), can you provide an updated result with a larger eval size? Also, I understand that this work is not to compete with standard continuous diffusion/flow models on image generation, but it would be greatly helpful to provide some results just a reference to contextualize the results in this work
-
How critical is the pre-trained MAE for your proposed method? If you have a less well-trained MAE, would CatDriff fail?
Thank you so much for the thoughtful and constructive review, specifically your questions about the role of pretrained representations and the CIFAR-10 evaluation. We found these suggestions genuinely helpful and took some additional time to investigate them through new experiments. The resulting findings have helped us better understand CatDrift and strengthen the paper, so we're grateful that you raised these points.
Q1. Dependence on a pretrained MAE
We agree that our original wording could be more precise. By teacher-free generation, we mean that CatDrift requires no pretrained generative teacher, distillation, or subsequent generative adaptation. Our original implementation does use a separately trained MAE for locality estimation, as is also common in continuous Drifting Models (Deng et al., 2026).
However, learned representations are not required by the categorical update itself.
To investigate this, we removed the feature extractor entirely for promoter DNA generation. Using a direct locality kernel with temperatures 0.01, 0.05, and 0.1, and further tuning the kernel parameters, we obtained:
These results, evaluated over ten runs, remain in a comparable performance regime without a separate representation-learning stage, although the 6-mer correlation is somewhat lower.
We will clarify the role and computational cost of representation pretraining in the revised paper.
Q2. CIFAR-10 evaluation
We agree that evaluating FID on only 1,000 samples was limiting.
Our experiments use the categorical CIFAR-10 formulation considered by PairFlow (Park et al., 2026), where each RGB channel is modeled directly as a 256-way categorical variable, rather than generating continuous RGB values.
Following your suggestion, we substantially expanded our evaluation and investigated whether a separately trained MAE could be replaced by a general-purpose representation.
Using an off-the-shelf pretrained DINOv3 encoder (Siméoni et al., 2025), CatDrift now achieves FID 4.8 ± 0.85, over five independent training seeds, each evaluated using 50,000 argmax-decoded samples against CIFAR-10 training-set reference statistics. Generation requires only one network evaluation.
For comparison, the original MAE-based result was FID 58 ± 2.9 using 1,000 samples across ten seeds. Since both the feature representation and evaluation sample count changed, these values should not be interpreted as a controlled MAE-versus-DINOv3 comparison.
Nevertheless, the new result shows that CatDrift can achieve strong one-step categorical image generation using an existing, general-purpose encoder without separately training a task-specific MAE.
For context, continuous one-step generators on unconditional CIFAR-10 report FID 2.83 (iCT; Song & Dhariwal, 2024), 2.92 (MeanFlow; Geng et al., 2025), and 3.60 (ECT; Geng et al., 2024), using FID-50K with one network evaluation. These comparisons are contextual, as the continuous pixel-space formulations and training protocols differ from ours.
We will update the evaluation section to clarify these distinctions.
Q3. How critical is the pretrained MAE?
Our additional experiments suggest that representation quality matters, particularly for high-dimensional image generation, consistent with findings in continuous Drifting Models (Deng et al., 2026; Zhang et al., 2026).
However, the DNA experiments suggest that appropriate kernel tuning can make learned features unnecessary in some categorical domains. The DINOv3 experiment further shows that CatDrift is not tied to a specific MAE or task-specific representation-learning procedure.
These experiments do not establish insensitivity to MAE quality across all settings. They instead clarify the distinction between locality estimation, which depends on the kernel and representation, and the categorical update, whose geometry is defined by the Bregman potential.
We will incorporate these findings and make the corresponding limitations more explicit.
References:
Deng et al. (2026), Generative Modeling via Drifting;
Park et al. (2026), PairFlow;
Siméoni et al. (2025), DINOv3;
Zhang et al. (2026), Learning Discriminative Geometry for Drifting Models;
Geng et al. (2025), Mean Flows for One-step Generative Modeling.