acceptodds
Under review as a conference paper at ICLR 2027

Quantized One-Step Generative Modeling via Cell-Margin Quantization-Aware Training

Abstract

Low-bit quantization and one-step sampling can substantially reduce the cost of flow-matching models, yet existing quantization-aware training methods typically minimize velocity-field reconstruction error, although generation quality is ultimately determined by the terminal distribution after ODE integration. We introduce CM-QAT, which trains a quantized student for one-step sampling with one function evaluation (1-NFE) to preserve the teacher's terminal geometry in a frozen feature space. Specifically, CM-QAT constructs a public nested partition and uses a differentiable prototype-margin objective to encourage each source sample to reach the correct terminal cell. Our counterexamples show that small velocity- error is neither necessary for terminal cell correctness nor sufficient for a distribution-free cell-risk guarantee without boundary-margin control. We derive conditional feature-space bounds from recursive cell-histogram discrepancies and relate cell-crossing probability to endpoint displacement and boundary margins. Experiments on Fashion-MNIST, CIFAR-10, and class-conditioned ImageNet-256 show that CM-QAT outperforms recent low-bit flow-generation methods and enables generation with models quantized to as low as 2-bit & 3-bit weights and sampled in a single step (1-NFE).

Primary area:
generative models
Keywords:
Flow Matching, Quantization-Aware Training, Generative Modelling, Optimal Transport

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3RCmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CM-QAT for quantizing flow-based models while preserving one-step generation quality. It trains the quantized student to match the teacher’s hierarchical feature-space regions using cell-margin and distributional losses. Experiments demonstrate improved low-bit generation, approaching full-precision performance on ImageNet-256 at W8A8.

Critical strengths
  • The paper addresses an important problem: preserving generation quality when quantizing flow-matching models.
  • The method is evaluated on several benchmarks and demonstrates promising results. In particular, the quantized SiT-L/2 MeanFlow model approaches full-precision performance on ImageNet-256 at W8A8.
Critical weaknesses
  • Limited evidence of scalability. The evaluation focuses on Fashion-MNIST, CIFAR-10, and ImageNet-256, using relatively modest, sub-billion-parameter backbones. While quantization is useful at these scales, its practical benefits are particularly relevant for billion-parameter generative models with substantial memory and computational requirements. Evaluation on a larger text-to-image model, such as FLUX, would better establish the method’s scalability and practical relevance.
  • Applicability to video generation remains unexplored. Video generation is a natural extension given its substantial computational and memory costs. Even a preliminary evaluation would strengthen the evidence for the method’s broader applicability.
  • Dependence on a fixed feature-space partition raises generalization questions. The hierarchical k-means partition is constructed from a finite set of teacher-generated samples. It remains unclear how well the method handles conditions or semantic modes that are poorly represented during partition construction. An analysis of sensitivity to the size and composition of this sample set would help address this concern.
  • The feature encoder is insufficiently specified. The paper does not clearly identify the encoder used for ϕ\phi, including its architecture, pretraining checkpoint, and feature extraction layer. Since this encoder defines the geometry underlying both cell construction and training supervision, these details are essential for reproducibility.
  • Hyperparameter sensitivity is insufficiently evaluated. The margin objective introduces several hyperparameters, including τp\tau_p, α\alpha, γ\gamma, and τ\tau, but no sensitivity analysis is provided. Figure 3 removes the student-margin regularizer as a whole; it does not establish how robust the method is to the settings of these hyperparameters.

To the authors: I used an AI tool to help analyze the manuscript and improve my understanding. I formulated my own human-based concerns and used the tool to polish their wording. I carefully reviewed the revised text to ensure that it accurately reflected my original concerns.

Questions for authors

see weakness

4 comments
AuthorsOpenReview ↗

First rebuttal

On scalability and the scope of evaluation.

We appreciate the suggestion to evaluate larger text-to-image models. We respectfully clarify that our primary contribution is a theoretical framework motivating terminal-geometry preservation under low-bit, one-step generation, supported by guarantees under explicit assumptions. Our evaluation follows well-established benchmarks in this research area. For example, ImageNet-256 and CIFAR-10 serve as benchmark datasets in recent well-known papers such as MeanFlow [1] and Inductive Moment Matching [2], with ImageNet-256 serving as a primary benchmark. Earlier foundational papers also use smaller-scale datasets for image generation: OT-CFM [3] uses CIFAR-10, Flow Matching [4] uses CIFAR-10 and ImageNet-128, and OT-Flow [5] uses MNIST. Thereby, ImageNet-256 and CIFAR-10 provide recognized settings for evaluating principled advances in flow-based generation. We therefore respectfully ask the reviewer to assess the experimental scope in relation to this contribution, while distinguishing it from the broader question of scalability to billion-parameter models.

Moreover, quantization remains practically relevant for backbones with hundreds of millions of parameters, where storage and per-evaluation computation are already substantial. Our experiments evaluate generation quality across multiple weight and activation precisions at a fixed one-step inference budget. Although FLUX experiments would strengthen the evidence for larger-scale applicability, our central contribution is to provide practitioners with a theoretically grounded understanding of what quantized flow models should preserve, supported by empirical validation in established image-generation settings.

On applicability to video generation.

We agree that video generation is a valuable direction given its computational demands. However, our present claims concern image generation; extending terminal feature-cell supervision to video would additionally require an appropriate spatiotemporal representation and evaluation of temporal consistency. This constitutes a meaningful extension beyond the current experimental scope. Hence, we will discuss this direction in future work.

We will continue our rebuttal for the other weaknesses in the following comments.

[1] Geng et al. Mean Flows for One-step Generative Modeling. NeurIPS 2025.

[2] Zhou et al. Inductive Moment Matching. ICML 2025.

[3] Tong et al. Improving and generalizing flow-based generative models with minibatch optimal transport. TMLR 2024.

[4] Lipman et al. Flow Matching for Generative Modeling. ICLR 2023.

[5] Onken et al. OT-Flow: Fast and accurate continuous normalizing flows via optimal transport. AAAI 2021

AuthorsOpenReview ↗

Second rebuttal

Feature space partition analysis.

We conducted additional experiments on CIFAR-10 W4A8 across 3 seeds, varying both the size and composition of the teacher sample set used to construct the partition. We kept the feature encoder, hierarchy size, Endpoint-QAT initializer, training pairs, and adaptation budget fixed across configurations. Each model received 50,000 adaptation updates and was evaluated using 50,000 generated images with 1-NFE. Results report mean ± standard deviation across seeds.

Partition constructionConstruction samplesFID ↓
Original composition50,00024.51 ± 1.79
Approximately preserved composition25,00023.74 ± 0.42
Approximately preserved composition12,50025.01 ± 0.70
Underrepresented cat/dog groups50,00024.94 ± 1.28

Halving the construction set did not worsen average FID, while reducing it to one quarter produced a modest increase relative to the half-size setting. These results suggest that performance does not depend critically on using the full construction set within this tested configuration, although they do not establish statistical equivalence between sample sizes.

Further, to examine poorly represented semantic groups, we retained approximately 10% of the original teacher-predicted cat/dog samples, replacing the removed samples with unique examples from the other groups while maintaining 50,000 construction samples. The QAT training data remained unchanged. This intervention yielded a small average FID increase, with effects varying across seeds.

We additionally measured paired teacher–student displacement in the normalized training feature space on 5,000 fresh held-out inputs per seed. Groups were assigned from teacher predictions. The table reports averages across the three seeds.

Held-out groupOriginal partition: displacement ↓Skewed partition: displacement ↓
Cat & dog0.79420.8055
Other eight groups0.79450.7983

The affected groups show a larger average displacement increase, although this differential effect is not consistent across all seeds. These results provide evidence of tolerance to reduced construction data while indicating that partition composition can influence preservation of underrepresented groups. We will include these experiments in the revision and clarify that they assess sensitivity within the tested setting, rather than establish guarantees for unseen semantic modes.

Feature encoder (duplicate of Reviewer 3dBq's Q2)

We use ResNet-family feature encoders. For Fashion-MNIST, we use SmallConvResNet128, a compact residual CNN that produces 128-dimensional pooled features, followed by L2 normalization. For CIFAR-10, we use WideResNet-28-10, extracting 640-dimensional pooled features. The default training configuration applies PCA whitening to reduce these features to 256 dimensions, followed by L2 normalization. For ImageNet-256, we use a frozen ResNet-50 pretrained on ImageNet with torchvision's IMAGENET1K_V2 weights. Since the MeanFlow backbone outputs VAE latents, its terminal latents are decoded into RGB images using the frozen Stable Diffusion EMA-finetuned VAE before being passed through ResNet-50. The resulting 2,048-dimensional pooled features are L2-normalized for the cell and histogram objectives. Please also note that these training encoders are separate from the Inception-v3 network used for FID/IS evaluation. We will explicitly specify these encoders and feature-processing steps in the revised manuscript.

AuthorsOpenReview ↗

Third rebuttal

W3. Hyperparameter sensitivity.

We conducted a one-at-a-time sensitivity study on CIFAR-10 W4A8, multiplying each of the four hyperparameters by 0.5 or 2 while keeping the partition, initialization, training data, and adaptation budget fixed within each seed. The study comprises 27 runs across 3 seeds, evaluated using 50,000 generated images per run. The default achieves FID 24.51 ± 1.79; results below report mean ± standard deviation.

HyperparameterDefault valueFID at 0.5×FID at 2×
Prototype temperature (τ_p)0.0723.90 ± 0.5425.30 ± 0.58
Competitor smoothing (α)0.0524.84 ± 0.6824.90 ± 2.46
Cell margin (γ)0.1023.74 ± 2.3924.30 ± 1.01
Surrogate-temperature schedule (τ)0.50 → 0.0525.80 ± 1.8424.99 ± 1.09

For the schedule, both endpoints are scaled. The cell-margin sweep leaves the separate student-margin regularizer unchanged; prototype temperature affects both cell logits and soft-histogram assignments.

Mean FID ranges from 23.74 to 25.80, with no perturbation improving upon the default in every seed. These results show moderate aggregate variation but seed-dependent sensitivity, rather than universal insensitivity to hyperparameter choices. We will include this analysis and explicitly document the parameter settings in the revision.

AuthorsOpenReview ↗

Final remarks.

We thank the reviewer for the constructive feedback. We have addressed each concern and hope that our clarifications and additional experiments resolve the issues raised. We will incorporate all extended experimental results into the revised manuscript.

Reviewer_gPDyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge that relying on velocity matching in low-bit one-step generation is insufficient to guarantee the quality of the final generation distribution. To address this challenge, the authors propose CM-QAT. This method partitions the frozen feature space into hierarchical cells and uses cell-margin supervision with regularization to ensure the student model preserves the teacher's terminal cell assignments. Evaluation on diverse datasets demonstrates improvement in generation quality compared to baseline QAT methods.

Critical strengths
  • The connection between the problem definition and the proposed solution is clear. The study provides a theoriteical analysis of the limitations of velocity matching and introduces a hierarchical cell,margin-based learning approach aimed at terminal cell preservation. It offers a compelling perspective on what exactly should be preserved during the quantization process.

  • The method yields significant improvements in generation quality on the ImageNet dataset. This serves as strong evidence for the practical efficacy of the proposed approach.

Critical weaknesses
  • The training objectives and conditions for the baselines, including MeanFlow-QAT, are not sufficiently explained.

  • Some details of the evaluation protocol used for ImageNet and the settings for the baselines are not clearly outlined. it is difficult to assess comparability with the baseline results reported in external papers.

Questions for authors
  • Could you specify the training objectives for MeanFlow-QAT and explain which training conditions were aligned with those of CM-QAT?

  • Could you also specify the number of generated samples and the guidance settings used for the ImageNet evaluation, and summarize the key differences in evaluation conditions between your work and the baselines?

  • Furthermore, since the comparisons within the same backbone primarily focus on controlled baselines, comparisons with or a discussion of recent QAT methods such as RobuQ [1] would help clarify the practical performance advantages of CM-QAT

[1] Kaicheng Yang et al., "RobuQ: Pushing DiTs to W1.58A2 via Robust Activation Quantization," ICML 2026.

1 comment

I appreciate the author's rebuttal and the additional experiments.

The explanations regarding the training objective of MeanFlow-QAT and the ImageNet evaluation protocol have largely resolved the ambiguities.

However, some limitations remain in how experimental conditions were controlled across methods. Please ensure that the revision clearly documents the training settings and includes a discussion of additional baselines. I will maintain my rating.

Reviewer_3dBqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper shows that QAT for flow matching models should use the terminal distribution as the target instead of using velocity field error L2 and proposes CMßAQT for one-step quantized generation. The quantized student is trained with a multi-scale prototype-margin loss to match with teacher endpoint. The theory shows that velocity-L2 error is neither necessary nor sufficient for cell correctness, bounds feature-space W₁ by cell diameter and multi-scale TV, and links cell-crossing probability to endpoint displacement and boundary mass.

Critical strengths

S1. Big improvement at low bits on ImageNet. On the same MeanFlow model, CM-QAT gets FID 5.99 at W4A4 while Endpoint-QAT gets 29.07, and 5.04 vs 11.09 at W3A8 in Table 2, with around 8 to 10 times smaller model.

S2. Interesting new view. Instead of only matching the velocity, the paper looks at where the final output lands, which make sense for 1-step generation. The counterexamples in Prop. 1 show clearly that a small velocity error alone is not enough.

S3. Careful theory. The authors clearly say which bound works for the real quantized model Prop. 3 and which one does not Lemma 4.

S4. Detailed and honest experiments. There are three datasets and several bit settings, and most baseline algorithms and the training configs are given in the appendix. The authors also report the negative results, such as W2A8 failing on ImageNet and the lower recall.

Critical weaknesses

W1. Table 1 baselines are not trained for 1-step. OT-PTQ, Q-Sched and Q-Drift only calibrate the 128-step teacher, and Velocity-QAT only matches the teacher velocity. So they get very bad FID, and only Endpoint-QAT is a fair comparison. A distill-then-quantize baseline would be more fair.

W2. Table 2 comparison may not be fair. Endpoint-QAT and MeanFlow-QAT are implemented by the authors, but CM-QAT starts from a W4A8 checkpoint for W4A4/W3A8 while Endpoint-QAT starts from the teacher in App. E, Alg. 4. This could explain part of the big gap at W4A4. Also SA-PTQ, EfficientDM and SA-QLoRA are copied from another paper with a different model.

W3. Not clear why it works. Some ablations in Fig. 3a are much worse than Endpoint-QAT (FID ≈201 and ≈134 vs 32.40), and no seeds are reported. The bounds in the theory are also never measured in the experiments, so it is hard to see if the theory really explains the gains.

W4. The 2-bit claim is too strong. The abstract says the method works down to 2-bit weights, but W2A8 failed on ImageNet and gets FID 108.38 on CIFAR-10 in Tab 1. It would be better to claim 3-bit for ImageNet.

Minor weakness: overall presentation is very difficult to follow, especially for the abstract and intro, I would recommend the author to double proofread the whole manuscript and try to improve the presentation for making it easier to understand.

Questions for authors
  1. For W4A4 and W3A8 on ImageNet, were Endpoint-QAT and MeanFlow-QAT also started from the W4A8 checkpoint, with the same training steps?
  2. Which feature encoder ϕ\phi is used for CIFAR-10 and ImageNet?
  3. Why are the ablations in Fig. 3a much worse than Endpoint-QAT?
  4. The gains at W8A8 and W4A8 on ImageNet are small (0.09 and 0.60 FID), can you report results over several seeds?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.