acceptodds
Under review as a conference paper at ICLR 2027

DriftXpress: Faster Drifting Models via Projected RKHS Fields

19 views

Abstract

Drifting Models have emerged as a new paradigm for one-step generative modeling, achieving strong image quality without iterative inference. The premise is to replace the iterative denoising process in diffusion models with a single evaluation of a generator. However, this creates a different trade-off: drifting reduces inference cost by moving much of the computation into training. We introduce DriftXpress, an accelerated formulation of drifting models based on projected RKHS fields. DriftXpress approximates the drifting kernel in a low-rank feature space. This preserves the attraction-repulsion structure of the original drifting field while reducing the cost of field evaluation. Across image-generation benchmarks, DriftXpress achieves comparable FID to standard drifting while reducing wall-clock training cost. These results show that the training-inference trade-off of drifting models can be pushed further without giving up their one-step inference advantage.

Primary area:
generative models
Keywords:
Drifting Models, One-Step Generative Modeling, Reproducing Kernel Hilbert Spaces (RKHS), Nyström Approximation, Low-Rank Kernel Approximation, Generative Modeling, Kernel Methods, Efficient Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tcUtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DriftXpress to reduce the training cost of Drifting Models. Since the training data distribution is fixed, the attraction component is computed with these fixed landmarks, and the summaries can be precomputed. The repulsion component is kept the same as the original because the generated distribution changes over time. While projecting to the landmark set can introduce error, it makes attraction evaluation faster and speeds up convergence thanks to the full positive support represented by the landmark set. There are two versions: shared and non-shared. Sharding is more memory-efficient because it computes the summaries per class. The method is tested on 4 datasets: CIFAR10, CIFAR100, SVHN, ImageNet. Its performance is on par with standard drifting models, with faster training time and convergence. With large batch sizes, the benefit is mainly computational efficiency, and with small batch sizes, it’s faster convergence.

Critical strengths

The method is sound and simple. Projecting only for the attraction component makes sense because the data distribution does not change during training.

Critical weaknesses
  • From the provided implementation, all models seem to be unconditional (the number of classes in Unet is set to 0, the label is dropped in the dataloader and is not used when calling model() to generate data), but in Appendix C1, Imagenet is still mentioned as class-conditional. Unconditional generation means that the attraction needs to be compared against full training samples rather than samples from a single class. This also links to the complexity of the two methods. In Drifting Models, the N+N^+ (NposN_{pos} in the original Drifting Models paper) is relatively small compared to the full training set size, so the running time may differ, depending on the number of positive samples for each class. If the ImageNet experiment is class-conditional generation, please specify the number of classes and positive samples. In the implementation, data is gathered across GPUs in standard Drifting Models, while it’s not the case for Driftxpress (drift_global_query_gather=True, nystrom_global_query_gather=False). How much does this contribute to the reported running time and to the performance?

  • Figure 2 shows FID vs elapsed time. FID may decrease faster because each epoch is trained faster, so more epochs are completed, and this might not be because of a larger effective support set. To show the benefit of the larger effective support set and the smoother, more stable field estimate, Figures 1C–D, 3, and 5 are more convincing, since they compare performance at matched training steps rather than matched training time.

Questions for authors
  • Please address my concern regarding the experimental setup and implementation.
  • Optional: It would be nice to run an experiment on the class-conditional ImageNet dataset, where the standard Drifting Models use 64 positive samples per class
6 comments

Thanks for the additional experiments.

1a. If “class-conditional” referred to landmarks selected per class, why is it mentioned in the FID evaluation together with “50 generated images per class”?

1b. Can you describe the setting used in the table? From the table, disabling global-query gathering reduces the mean step time by 9.4–12.5%, but DriftXpress is only about 1.5× faster per training step than Drifting Models on CIFAR10 and SVHN, while Table 1 in the main paper reports a speedup of at least 6×. Can you clarify this?

  1. Can you give more details about the reference set used in the experiment?

We thank the reviewer for their continued engagement with our work.

The timing table in our response uses two H100 GPUs with a per-GPU batch size of 1,000, while Table 1 uses eight H100 GPUs with a per-GPU batch size of 1,500. These correspond to global batch sizes of 2,000 and 12,000, respectively. The speedup depends on the batch size, as also illustrated in Figure 3. In particular, larger batches increase the baseline’s computational cost, which is why these numbers differ.

To clarify, the ImageNet results in Table 1 were evaluated using 50,000 images generated without class conditioning, compared against the 50,000 real images in the ILSVRC2012 validation split. The reference split contains 50 real images per class across 1,000 classes. This class balance applies to the real reference set. We will revise the appendix to make this explicit.

For the new class-conditioned ImageNet experiment, we compute FID between 50,000 generated images, with 500 per class generated using the corresponding class labels, and 5,000 held-out ILSVRC2012 validation images, with 50 per class from the same 100 selected classes.

AuthorsOpenReview ↗

Gentle follow-up on class-conditioned generation and experimental setup

Dear Reviewer tcUt, thank you again for the constructive discussion and for engaging with our responses. We wanted to check in as the discussion period draws to a close to confirm whether our latest clarifications on class-conditioned generation and experimental setups have fully resolved your remaining concerns. If everything looks satisfactory, we hope you will consider raising your score. Please let us know if any further questions remain!

Reviewer_ESmqOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to lower the training cost of Drifting Models by approximating the attractive term with projection and low-rank. The summary terms are then pre-computed using the training set. A sharded variant is proposed as a further optimisation. Experiments are conducted on SVHN, CIFAR and a small-resolution ImageNet setting.

Critical strengths
  1. The idea is reasonable and a natural fit. The summary pre-computing and the sharded version are smart implementation optimisations. The code is provided.

  2. The performance generally matches the FID of standard drifting with higher training throughput. (But does not evaluated on high-resolution images). Ablation studies validate some design choices.

Critical weaknesses
  1. Training cost is a secondary priority compared to inference cost. Also, the experiments are conducted on settings with relatively short training time, where the training cost is not a bottleneck. It is unclear whether the method is still useful for larger-scale training where the training cost is indeed the bottleneck.

  2. DriftXpress evaluates only small-resolution datasets, and the paper does not state the ImageNet resolution (which seems to be a small resolution). It is unknown whether the FIDs are still comparable and whether the memory overhead is still worth it when scaled to higher resolution.

  3. The speedup shrinks as the number of classes grows. This is another concern about whether the method can be scaled up. This is critical since the training cost reduction is more meaningful on large-scale training.

  4. DriftXpress uses more peak GPU memory than standard drifting. The method is a space-time trade-off. The overhead is small at small resolution, but may not be small at higher resolution / more classes.

  5. The precomputation time should be included in the speed reporting. It may also be longer for larger-scale settings.

Questions for authors

See above.

Reviewer_QpxLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces DriftXpress, aiming to reduce the training cost of Drifting Models. The method employs Nyström projection to approximate the attraction field, compressing the entire training set information into computable statistics while retaining the precise repulsion terms. It also enables the use of sharding to accommodate a larger set of landmarks. The paper reports that on multiple image generation datasets, this method can maintain a FID close to the baseline while improving the training throughput rate.

Critical strengths
  1. This research question is interesting and the method is quite practical.
  2. A series of experiments provided relatively comprehensive initial evidence for the effectiveness of the method.
Critical weaknesses
  1. This paper attributes the more stable and faster convergence to the sufficient data support provided by the statistics of the entire training set. However, the existing main comparison has simultaneously changed the kernel estimation method and the positive sample support set, and has not directly verified this explanation. It lacks a comparison using minibatch and the full data statistics under the same low-rank approximation setting.
  2. This method comes at a cost. Firstly, it consumes more GPU memory than the baseline Drifting model. Additionally, this method lacks more validation on datasets with higher resolutions.
Questions for authors

Please refer to the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.