- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes CM-QAT for quantizing flow-based models while preserving one-step generation quality. It trains the quantized student to match the teacher’s hierarchical feature-space regions using cell-margin and distributional losses. Experiments demonstrate improved low-bit generation, approaching full-precision performance on ImageNet-256 at W8A8.
Critical strengths
- The paper addresses an important problem: preserving generation quality when quantizing flow-matching models.
- The method is evaluated on several benchmarks and demonstrates promising results. In particular, the quantized SiT-L/2 MeanFlow model approaches full-precision performance on ImageNet-256 at W8A8.
Critical weaknesses
- Limited evidence of scalability. The evaluation focuses on Fashion-MNIST, CIFAR-10, and ImageNet-256, using relatively modest, sub-billion-parameter backbones. While quantization is useful at these scales, its practical benefits are particularly relevant for billion-parameter generative models with substantial memory and computational requirements. Evaluation on a larger text-to-image model, such as FLUX, would better establish the method’s scalability and practical relevance.
- Applicability to video generation remains unexplored. Video generation is a natural extension given its substantial computational and memory costs. Even a preliminary evaluation would strengthen the evidence for the method’s broader applicability.
- Dependence on a fixed feature-space partition raises generalization questions. The hierarchical k-means partition is constructed from a finite set of teacher-generated samples. It remains unclear how well the method handles conditions or semantic modes that are poorly represented during partition construction. An analysis of sensitivity to the size and composition of this sample set would help address this concern.
- The feature encoder is insufficiently specified. The paper does not clearly identify the encoder used for , including its architecture, pretraining checkpoint, and feature extraction layer. Since this encoder defines the geometry underlying both cell construction and training supervision, these details are essential for reproducibility.
- Hyperparameter sensitivity is insufficiently evaluated. The margin objective introduces several hyperparameters, including , , , and , but no sensitivity analysis is provided. Figure 3 removes the student-margin regularizer as a whole; it does not establish how robust the method is to the settings of these hyperparameters.
To the authors: I used an AI tool to help analyze the manuscript and improve my understanding. I formulated my own human-based concerns and used the tool to polish their wording. I carefully reviewed the revised text to ensure that it accurately reflected my original concerns.
Questions for authors
see weakness
4 comments
Second rebuttal
Feature space partition analysis.
We conducted additional experiments on CIFAR-10 W4A8 across 3 seeds, varying both the size and composition of the teacher sample set used to construct the partition. We kept the feature encoder, hierarchy size, Endpoint-QAT initializer, training pairs, and adaptation budget fixed across configurations. Each model received 50,000 adaptation updates and was evaluated using 50,000 generated images with 1-NFE. Results report mean ± standard deviation across seeds.
| Partition construction | Construction samples | FID ↓ |
|---|---|---|
| Original composition | 50,000 | 24.51 ± 1.79 |
| Approximately preserved composition | 25,000 | 23.74 ± 0.42 |
| Approximately preserved composition | 12,500 | 25.01 ± 0.70 |
| Underrepresented cat/dog groups | 50,000 | 24.94 ± 1.28 |
Halving the construction set did not worsen average FID, while reducing it to one quarter produced a modest increase relative to the half-size setting. These results suggest that performance does not depend critically on using the full construction set within this tested configuration, although they do not establish statistical equivalence between sample sizes.
Further, to examine poorly represented semantic groups, we retained approximately 10% of the original teacher-predicted cat/dog samples, replacing the removed samples with unique examples from the other groups while maintaining 50,000 construction samples. The QAT training data remained unchanged. This intervention yielded a small average FID increase, with effects varying across seeds.
We additionally measured paired teacher–student displacement in the normalized training feature space on 5,000 fresh held-out inputs per seed. Groups were assigned from teacher predictions. The table reports averages across the three seeds.
| Held-out group | Original partition: displacement ↓ | Skewed partition: displacement ↓ |
|---|---|---|
| Cat & dog | 0.7942 | 0.8055 |
| Other eight groups | 0.7945 | 0.7983 |
The affected groups show a larger average displacement increase, although this differential effect is not consistent across all seeds. These results provide evidence of tolerance to reduced construction data while indicating that partition composition can influence preservation of underrepresented groups. We will include these experiments in the revision and clarify that they assess sensitivity within the tested setting, rather than establish guarantees for unseen semantic modes.
Feature encoder (duplicate of Reviewer 3dBq's Q2)
We use ResNet-family feature encoders. For Fashion-MNIST, we use SmallConvResNet128, a compact residual CNN that produces 128-dimensional pooled features, followed by L2 normalization. For CIFAR-10, we use WideResNet-28-10, extracting 640-dimensional pooled features. The default training configuration applies PCA whitening to reduce these features to 256 dimensions, followed by L2 normalization. For ImageNet-256, we use a frozen ResNet-50 pretrained on ImageNet with torchvision's IMAGENET1K_V2 weights. Since the MeanFlow backbone outputs VAE latents, its terminal latents are decoded into RGB images using the frozen Stable Diffusion EMA-finetuned VAE before being passed through ResNet-50. The resulting 2,048-dimensional pooled features are L2-normalized for the cell and histogram objectives. Please also note that these training encoders are separate from the Inception-v3 network used for FID/IS evaluation. We will explicitly specify these encoders and feature-processing steps in the revised manuscript.
Third rebuttal
W3. Hyperparameter sensitivity.
We conducted a one-at-a-time sensitivity study on CIFAR-10 W4A8, multiplying each of the four hyperparameters by 0.5 or 2 while keeping the partition, initialization, training data, and adaptation budget fixed within each seed. The study comprises 27 runs across 3 seeds, evaluated using 50,000 generated images per run. The default achieves FID 24.51 ± 1.79; results below report mean ± standard deviation.
| Hyperparameter | Default value | FID at 0.5× | FID at 2× |
|---|---|---|---|
| Prototype temperature (τ_p) | 0.07 | 23.90 ± 0.54 | 25.30 ± 0.58 |
| Competitor smoothing (α) | 0.05 | 24.84 ± 0.68 | 24.90 ± 2.46 |
| Cell margin (γ) | 0.10 | 23.74 ± 2.39 | 24.30 ± 1.01 |
| Surrogate-temperature schedule (τ) | 0.50 → 0.05 | 25.80 ± 1.84 | 24.99 ± 1.09 |
For the schedule, both endpoints are scaled. The cell-margin sweep leaves the separate student-margin regularizer unchanged; prototype temperature affects both cell logits and soft-histogram assignments.
Mean FID ranges from 23.74 to 25.80, with no perturbation improving upon the default in every seed. These results show moderate aggregate variation but seed-dependent sensitivity, rather than universal insensitivity to hyperparameter choices. We will include this analysis and explicitly document the parameter settings in the revision.
Final remarks.
We thank the reviewer for the constructive feedback. We have addressed each concern and hope that our clarifications and additional experiments resolve the issues raised. We will incorporate all extended experimental results into the revised manuscript.
First rebuttal
On scalability and the scope of evaluation.
We appreciate the suggestion to evaluate larger text-to-image models. We respectfully clarify that our primary contribution is a theoretical framework motivating terminal-geometry preservation under low-bit, one-step generation, supported by guarantees under explicit assumptions. Our evaluation follows well-established benchmarks in this research area. For example, ImageNet-256 and CIFAR-10 serve as benchmark datasets in recent well-known papers such as MeanFlow [1] and Inductive Moment Matching [2], with ImageNet-256 serving as a primary benchmark. Earlier foundational papers also use smaller-scale datasets for image generation: OT-CFM [3] uses CIFAR-10, Flow Matching [4] uses CIFAR-10 and ImageNet-128, and OT-Flow [5] uses MNIST. Thereby, ImageNet-256 and CIFAR-10 provide recognized settings for evaluating principled advances in flow-based generation. We therefore respectfully ask the reviewer to assess the experimental scope in relation to this contribution, while distinguishing it from the broader question of scalability to billion-parameter models.
Moreover, quantization remains practically relevant for backbones with hundreds of millions of parameters, where storage and per-evaluation computation are already substantial. Our experiments evaluate generation quality across multiple weight and activation precisions at a fixed one-step inference budget. Although FLUX experiments would strengthen the evidence for larger-scale applicability, our central contribution is to provide practitioners with a theoretically grounded understanding of what quantized flow models should preserve, supported by empirical validation in established image-generation settings.
On applicability to video generation.
We agree that video generation is a valuable direction given its computational demands. However, our present claims concern image generation; extending terminal feature-cell supervision to video would additionally require an appropriate spatiotemporal representation and evaluation of temporal consistency. This constitutes a meaningful extension beyond the current experimental scope. Hence, we will discuss this direction in future work.
We will continue our rebuttal for the other weaknesses in the following comments.
[1] Geng et al. Mean Flows for One-step Generative Modeling. NeurIPS 2025.
[2] Zhou et al. Inductive Moment Matching. ICML 2025.
[3] Tong et al. Improving and generalizing flow-based generative models with minibatch optimal transport. TMLR 2024.
[4] Lipman et al. Flow Matching for Generative Modeling. ICLR 2023.
[5] Onken et al. OT-Flow: Fast and accurate continuous normalizing flows via optimal transport. AAAI 2021