Mean-Shifted Intermediate Sampling: Training-Free Acceleration for Few-Step Diffusion Generation

Few-step diffusion sampling suffers from severe discretization errors due to large step sizes. Some existing methods reduce the average step size by starting reverse sampling from an intermediate distribution, but they often require training additional models or rely on real samples during inference. In this paper, we propose Mean-Shifted Intermediate Sampling (MSIS), a simple training-free strategy that approximates the high-noise intermediate distribution with mean-shifted Gaussian noise. The mean shift is determined by the forward noising formula and the dataset itself, so it can be computed once offline and reused across samplers, timestep schedules, and models sharing the same input space, with negligible inference overhead. We further show that near the Gaussian endpoint, the mean shift is the dominant data-dependent deviation, while covariance and higher-order effects are weaker, justifying mean-shifted Gaussian noise as a first-order approximation. Extensive experiments demonstrate the effectiveness and generality of MSIS across datasets, samplers, schedules, and architectures. On conditional ImageNet \(256\times256\), MSIS requires only \(2.22\) minutes of one-time preprocessing on a single RTX 3090 and reduces FID by \(35.18%\)–\(55.33%\) across the evaluated DDIM step budgets. Our code will be made publicly available upon acceptance.